Inspector displays in dual modes#60

Currently, when a nest is in dual screening mode, inspector only displays + allows filtering on its finalized (adjudicated) screening status. This is quite confusing to all users (especially new ones) and limits inspectors usefulness in dual screening.

A thought: when in dual screening, could we have two columns, one for reviewer and one of finalized screening decision.

If dual screening is activated for the nest, it would be cool if you could choose what “mode” you want study inspector to be in as needed (e.g., if you choose reviewer 1 mode, you see screening status and exclusion/inclusion reasons for reviewer 1 only; if you choose reviewer 2 mode, you only see screening status and exclusion/inclusion reasons for reviewer 2). If you put study inspector in an “overview” mode, maybe there could be 3 columns: reviewer 1, reviewer 2, and the final screening decision.

To protect screening independence, if reviewer 1 is logged in, study inspector could be set up so that if reviewer 1 puts it in reviewer 2 mode, reviewer 1 can only see whether reviewer 2 has screened articles or not and can’t see reviewer 2’s exclusion/inclusion reasons. If reviewer 1 puts it in overview mode, they could see their reviewer 1 column, but the screening decisions in the reviewer 2 column would be hidden/grayed out unless they matched with reviewer 1’s screening decisions; in that case, the final decision column would automatically populate with the matching screen decision.

I like where you’re going with this! It’s a little arbitrary to designate which of the screens would be 1/2 (especially if the viewing user performed one of the screens). Regardless, I think breaking out non-final screening decisions into a separate column would be ideal; you also raise a valid point about protecting independence.

Perhaps we could get away with only showing your screening decision (as in whichever user is viewing the page) and the final screening decision? I’m not sure why, outside of adjudication, which you typically wouldn’t do via inspector, you’d need to see anyone’s screening determinations other than your own? Perhaps to pinpoint records that still require a second reviewer?

I guess I assumed that whoever started screening first would be designated screener 1 and the other would automatically be screener 2. But maybe designating screener #s could also be a setting in NK? Like where you designate someone admin/user etc. there could be a separate dropdown for within-the-nest roles where you can select “screener 1” and “screener 2” options. There could even be other labels like “tagger 1” or “extractor 1”; from a project management standpoint, it would be nice to have a record of who was assigned/who performed which tasks.

And yes, that was definitely my train of thought––essentially, study inspector could just show you screening status for the other screener (i.e., whether the other screener had given a screening decision for a record) but not show you screening decision for the other screener. The only exception would be if both screeners had a “completed” screening status AND their screening decision matched for a given record. However, if showing the other screener’s screening status is tricky or not optimal for some reason, I definitely still think it would be useful to differentiate in overview mode whether a record doesn’t have an adjudicated screening decision because of incomplete screening status, or because of mismatching screening decisions.

Will also apply in the future to Dual Extract

Thanks Kevin! To be less cryptic, we were just discussing in detail how dual extraction will work. We’re going to try to create a generalized solution that more clearly delineates final & reviewer-level results (for screening, extraction, and in the future ROB).

I really like @Alex Mebane‘s suggestion of having a mode toggle for inspector. In the name of leak-prevention & simplicity, I’m going to propose two modes:

• Reviewer Mode:
  • When a module is in dual mode, view your (current user) decisions/gathered data. We should also make it possible to see the number of reviewers who have completed a given module (e.g. “0 screens”, “1 data extraction”), but without showing their name/results.
  • When a module is in single mode, view the finalized decisions/gathered data.
• Overview Mode:
  • When a module is in dual mode, view finalized decisions/gathered data, including the adjudication interface.
  • Single mode is handled the same as ^.

These modes would apply to everything in inspector; filters, sorts, data shown in the table, viewing of individual records, etc.

Is this too clunky? I like that it completely separates the two modes of review, but also see that it could come with frustrations (e.g. when a nest is configured for dual screening and you want to update a single record’s screening status, it’s a couple more clicks away).

I’m basically aligned here– categorical starts to look super messy with three columns…

Good point RE: status! Our goal here will be to show very simply in Reviewer mode how many times a study has been extracted; that information is already inherent in Overview mode since you see what has been gathered by 2 reviewers. Thanks, and we may reach out with specific questions about how you want that status to ‘look’ in Reviewer mode!

Sounds great! Reach out whenever.

@Kimberley.bennett30 I’m adding you to this issue so you can track our changes related to your request (https://nested-knowledge.nolt.io/352)!

With release 1.36.0, we’ve added explicit differentiation between “Your Screening Status” (as in the current user’s reviewer-level screen) and “Final Screening Status” (as in the adjudicated screening status) for both columns and filters. “Your Screening Status” will only be made available to nests with dual screening configured; single screening nests will only see “Final Screening Status” (prior behavior).

We’ve also rearranged & simplified inspector options. You can now find inspector column selection in the far right “…” table header.

Please drop a line with any questions or feedback!

Here’s what this display looks like on a dual configured nest: