Hierarchical Exclusion Reasons#136

When screening, not all exclusion reasons are ‘equal’. In some cases, studies excluded for certain exclusion reasons need to be scrutinized more, and in others, studies may be excluded but still of interest. It would be helpful to categorize exclusion reasons (user-customized), such that they can be filtered by exclusion ‘category’ or possibly color-coded.

Help me understand how this will be used? I see two scenarios:

• filtering in inspector, we don’t allow ORing filters together, so building+ filtering on categories of exclusion reasons could be more efficient way of finding records of interest. The current suggested way of doing this would be separate queries on individual exclusion reasons, which is clunky.
• Drawing the reviewer’s eye to certain reasons when screening

Those are the scenarios we envisioned– users wanting to filter by a certain type of exclusion on Inspector, and users wanting to prioritize or further scrutinize based on exclusion reason. This would also serve as a de-facto way for groups like ECRI to separate out which articles were screened by searchers vs analysts if they have workflows where both types of users are screening!

Notes from NK-> We see complications with categorizing Exclusion Reasons, but the Full Text check box could be an example of a similar feature that we could expand if there are workflows that involve categorizing studies. Also, couldn’t using “process-focused” bulk action tags (i.e. adding a tag like ‘ready for secondary screening’) solve this problem as effectively as categorizing the reasons?

Harmonizing some discussion– Alex brought up the idea of HIERARCHICALLY organized exclusion reasons: “To make the set of exclusion reasons created less arbitrary/disconnected, I know some studies set up exclusion hierarchies (see here: https://www.ncbi.nlm.nih.gov/books/NBK531721/table/book-app5.tabb2/) where a study with multiple exclusion reasons is assigned to the highest applicable exclusion reason. I wonder if setting up a hierachy maybe through a dropdown mechanism or clicking and dragging mechanism could be an optional feature of exclusion reason configuration?”

^I have thought about this before too. In other programs like Disteller RS they have a cascade of exclusion reasons, where it prompts users to answer “Is it RTC” before moving on to “is it related to X drug” and so on. This helps since when reviewing exclusions you can focus on the ones that are further down the cascade and so more relevant.

@Annie Steffenson We just had an internal thread about exclusion reason hierarchies, so tagging you here!

Just wanted to note that an exclusion hierarchy would be particularly helpful when doing dual screening. Might save time at the adjudicator stage.

I really appreciate the recent feedback & enthusiasm on this issue, and it did cause us to investigate it. In the interest of transparency, we internally discussed tonight how we’d like to implement this (vs. hacking together a version of it). In short, it would look similar to a tagging hierarchy (possibly even being part of it). We would similarly include this structure in our exclusion reason templating, and, yes, as Annie is pointing out, use it in several other locations to allow users to work faster.

The development effort for this big vision and potential for confusion (even if limited) of what’s currently a simple concept did not allow us justify implementing this feature yet. That’s probably not what you want to hear, but I wanted to make sure you knew where this actually stands. We’d need to see more votes to justify implementing this, when we have e.g. dual extraction still on our plate.

Related, with release 1.61.0, ordering exclusion reasons is now possible using drag and drop.

Completed in release 1.73.0. Exclusion reasons may now be hierarchically structured (/grouped) & reordered under their parent. All existing reasons will appear flatly, as before, but may be structured as you wish. We have a couple of optimizations using this structure in the pipeline (AB/FT reason differentiation, PRISMA exclusion reason summarization). Thanks for your patience!