Can't add/edit quantitative data in Adjudicate Extraction for Dual Extraction#485

There seems to be an issue with Adjudicate Extraction where quantitative data can’t be added. Timepoint, Time, and Units can be added/edited, but important quantitative data for extraction disappears after entering values. Also, it looks like Interventions are greyed out and can’t be edited in Final Adjudication (would be great if intervention labels can be changed). See examples attached.

This was specifically for Nest #2774 (https://nested-knowledge.com/gather/2774) and was using Windows at the time.

On a side note, it could also be useful to allow export of data collected from each of the individual reviewers. This could be useful for both Extraction and Risk of Bias (sometimes important to show results of adjudication and for calculating inter-rater agreement metrics)

Woah, that’s no good. We’re on it.

Thanks!!!

Hey John - sorry this issue fell through the cracks. Our UX is extremely subtle here, to the point that I forgot how it worked, and it took me 30s to identify. Yikes.

If you hit the tiny + button under final arms, you can add/confirm reviewer-level arms. Then fields will become properly editable (pictured is a development copy of your nest, we did not touch your data on production). Please give this try and let me know if it works.

There are so many UX problems at play here, I’m going to hold off on telling you how we will fix - I am marking this down for a revisit from our designers.

Thanks for the update Karl! I did notice the + button under final arms, but once you add the arms for adjudication, I still can’t enter data without it disappearing (no data is added to cells). The only editable fields seem to be the timepoints/units in adjudicate extraction. Fortunately, this project is not pressing but will be excited to see the update. Let me know if I can assist in any way!

I’m guessing you didn’t finish creating the arm (adding a size)?

Created a video since there’s a lot of moving parts here:

Oh my goodness… Thanks for the detailed reply! It certainly is working - my apologies for the confusion and for not looking into this more closely.

Some things to consider may be:

1. Allow user to directly select from a dropdown of interventions rather than add an intervention from the “+” button.

2. Autofill adjudicated cells when reviewers have identical responses. Adjudicator will still need to confirm results, but this could still be very helpful

1. Agreed!
2. Yes, auto-adjudication would be so nice - although this is less trivial to build while still making QA/manual review possible. A UI/UX problem to solve :)

I’ll also add:

1. Cells that aren’t editable need to be disabled
2. Rows that aren’t complete should not show a green checkmark (this is an outright bug).

Here’s what we have planned to address manual adjudication issues:

• Arms should be prepopulated with the intervention selection (clicking the + button is weird & hard to notice)
• Green checkmarks, for both arms & metrics, should not be present unless the data is populated, & correct
• Cell inputs should be disabled, as expected, if that data is not yet ready to be extracted.

Auto-adjudication has less obvious solutions than ^, so we’re giving it more thought, and a longer timeline.

Thanks Karl! Definitely agree with the three main objectives above - that will be an awesome update. Totally understand that auto-adjudication will come at a later time. Auto-adjudication is certainly not a must have, but will be a really nice addition

Jeff is already designing it, but definitely best to prioritize fixing these usability issue!

We’ve corrected all 3 issues identified above in release 1.57.1. We plan to revisit extraction auto-adjudication (in design right now), but hopefully this improves usability in the mean time :)