Advice about excluding post hoc studies or follow-upstudies?#308

When you are screening for a meta-analysis or systemic review, would you say that it is generally okay to exclude what are called post hoc studies or follow-up studies? What if a study like this matches your inclusion criteria really well?

I generally exclude post-hoc studies and instead include the primary studies they are based on. I then review the excluded post-hoc studies to see if there is additional data that is useful and then add that data to the primary studies. I always note on the primary studies if I pull data from another source besides the manuscript. I hope this helps!

There are times where the post-hoc study is more relevant than the primary study, in that case I add the post-hoc and make a comment on it saying that there is a primary but the post-hoc is more relevant since it’s on the subpopulation of interest.

Thanks for your answers, Nicole– they are exactly what I was going to say, but with one extra warning: make sure not to duplicatively collect any patient’s outcomes. So, best practices with respect to secondary studies are:

(1) generally exclude post hoc/secondary studies in favor of primary studies, BUT

(2) if there is data of interest in the post hoc study that is not present in the primary study, you can either add that data to the primary study or replace the primary study. If you do so, make sure that…

(3) You only collect data on each patient once.

Thanks, Kevin! I should have clarified in my initial response that we don’t want to duplicate data. Thanks for highlighting that.

@Tiffanyyesavage I hope these answers are helpful! I’ll also note that a feature was requested to help differentiate and link primary and secondary publications in the software (https://nested-knowledge.nolt.io/142), so we are working on helping with this practice :)

Thiis is great advice, Niicole and Kevin.! Let me talk to @Kathryn Cowie about whether we should include some of this in the wiki section of best practices. @Kevin.kallmes, I wasn’t able to see the link from above. Here is a paper that talks about this issue. It definitely implies that post-hoc studies are not as good. https://pubmed.ncbi.nlm.nih.gov/24950694/

@Tiffanyyesavage , It would be great to add this information to the Wiki Guide! …It might fit with the screening best practices, or it could be a page for advanced-users under Miscellaneous: https://wiki.nested-knowledge.com/doku.php?id=wiki:guide:tips. Open to other ideas as well!

Yes, @Kathryn Cowie. I add some of the tips from Nicole and Kevin into the Best Practices for Screening page.

I think the paper is correct! Mostly, post hoc studies should be excluded; if used, you have to be careful about recording how you did so and preventing duplication. I saw the Wiki addition and I’m a huge fan!

Good to know!I hope to add some tips about investigating the feasibility of a review as well. Something I need to now more about myself. Erin has a good article on the wiki that we are trying to place in a more prominent spot.