Diagnostic Concordance: Key to Improving Pathology Training

by | Apr 7, 2017 | Education, Residents | 0 comments

Take Pathology Training to the Next Level

Are you a pathology resident or fellow who is looking to get the most out of your training?

Are you a program director who wants a more effective way to evaluate your residents and fellows?

Would you like a better idea of where your program can improve?

4 Great Reasons Why Pathology Training Programs Should Track Diagnostic Concordance

  1. Insanely easy to start tracking without any extra resources
  2. Easy scalable to an entire program for even more benefits
  3. Emphasizes diagnostic skills in evaluations
  4. Strengthens current evaluation systems by adding objective data

Why Else?

Undeniably, the most important thing to learn as a pathology trainee is the right diagnosis.  We could focus on hundreds of other attributes that a pathology trainee should learn but if the “top line” is wrong then what else matters?

Past experience: All throughout pathology training, I had a very rough idea of how often my preliminary diagnosis matched that of my attending.  Rarely would I ever have the time to check back in the computer to verify that my preliminary diagnosis was correct.  The attending pathologist was also very busy and wouldn’t have time to inform me if I missed a diagnosis unless we were able to “double scope”.  Without a formal way of tracking my concordance rate with my attending, I had no real accounting of my accuracy.

Bottom line: If you don’t track concordance as a trainee, you don’t truly know if you’re improving

What Is Diagnostic Concordance in Pathology Education?

It’s when one pathologist’s diagnosis agrees with another pathologist’s diagnosis.  In this article and in educational terms, “diagnostic concordance” refers to whether a trainee’s preliminary diagnosis agrees with the final diagnosis of the attending pathologist.

Current Resident Evaluations Not Sufficient

Current evaluations are based on standardized tests and the opinions of faculty.

Standardized Tests Not the Only Objective Method

Programs that overemphasize standardized tests in resident evaluations will favor residents who are better test takers but not necessarily better pathologists.  Don’t get me wrong, there is definitely a place for standardized tests in pathology education.  Standardized tests are like an in vitro test in the lab while tracking diagnostic concordance is testing the resident in vivo.  It’s a truer representation of the residents abilities as a pathologist inside the lab.

Standardized tests are like an in vitro test in the lab while tracking diagnostic concordance is testing the resident in vivo.

Subjective Measures Reward Personality and Punish Quiet Achievers

Besides standardized tests, nearly everything else in evaluations is based on someone’s opinion.  Many decisions are made that both reward and discipline residents based on how they are evaluated.  If the evaluation is subjective, then rewards are merely popularity contests.  Likewise, disciplinary actions are more likely to be the result of misunderstandings and conflicting personalities rather than poor performance.

Past experience: During training I saw numerous examples in which subjective evaluations punished intelligent residents unjustly and rewarded underperforming residents.

Contrary to popular belief, interpersonal skills are important to have in pathology but they shouldn’t supersede or be used in isolation to evaluate a resident.

So, how does this whole “diagnostic concordance” thing work?  I’m glad you asked…

4 Simple Steps to Track Concordance During Pathology Training

1. Find a Place to Record Your Data

1.1. Plain Paper: Initial tracking of concordance can be as easy as writing down how often the preliminary diagnosis agrees with the attending’s diagnosis.  This shouldn’t be confused with jotting down notes on the requisition paperwork.  You should have a specific space to compare the preliminary and final diagnosis.  You can download a basic concordance worksheet to help you get started.

Paper Diagnostic Concordance Tracker

Example of Filled Diagnostic Concordance Tracker

1.2. Department LIS: Several Laboratory Information Systems (LIS) will allow a cytotechnologist to record their diagnosis prior to a case being seen by a pathologist.  Trainees have access to this method while working on cytology cases but I have never seen this carried over into general anatomic pathology, let alone clinical pathology.  Is your program able to track concordance on the LIS (i.e. CoPath, Beaker, Cerner, etc.)? Are you able to record preliminary diagnoses?  Are you able to pull records comparing the preliminary and the final?  Please comment below if your training program has used the LIS to compare preliminary and final diagnoses.

1.3. Automatic Spreadsheet: You can also download this spreadsheet that keeps track of your concordance over time.  This spreadsheet allows you to sort data based on time, attending, specialty, and subspecialty.  The spreadsheet automatically charts your data so you can quickly see positive and negative trends, weak spots, and strengths.

Automatically chart diagnostic concordance

Download (Mac Edition)

Windows Edition Coming Soon!

2. Use Four Categories

1. Agree, 2. Minor Discrepancy, 3. Major Discrepancy, 4. Challenging Case

More categories is not necessarily better.  If you prefer, you can also include subcategories of AP/CP or a subspecialty (i.e. cytology, dermatopathology, blood bank) in each data point.

3. Define Each Category

  • Agree: All major components of the final diagnosis are the same, including any additional comments in the final report.  Minor wording changes are allowed but should only be stylistic and have limited clinical consequences.
  • Minor discrepancy: A component of the final diagnosis or comment is different.  The difference is worth noting because of minor clinical consequences or because of the difference infers some lack of understanding by the trainee.  No major clinical consequences are seen between the two diagnoses.
  • Major discrepancy: A component of the final diagnosis or comment section is significantly different.  The clinical outcome would be significantly different between the two diagnoses.  Even if the clinical difference is not substantial, if the two diagnoses are so different that it implies are large gap in pathology understanding by the trainee, this category also applies.
  • Challenging Case: In rare circumstances, a specific diagnosis cannot be reached.  This is usually after the case has been seen by more than one attending and reviewed at the consensus conference.  In this case, the trainee should not be discredited for missing the diagnosis.  This category shouldn’t exceed 10 percent of total cases over several months of tracking.  It may rise higher than 10% briefly when rotating with faculty who have challenging consult services.

When each category is defined, the results can be carried over from attending to attending with less variability in results.

4. Track Concordance Over Time

The real value of tracking concordance during training is demonstrated after tracking the results over time. It’s encouraging when you see an upward trend and it’s also easy to spot areas of weakness.  When tracked over time, diagnostic concordance is written as a percentage of the total number of cases for each time period tracked.

 

Benefits of Tracking Concordance

There are many benefits to tracking diagnostic concordance as a trainee.  To summarize, objective data always holds more weight.  When a trainee is able to show actual numbers demonstrating past weaknesses compared to current abilities, this means much more than just a verbal or written “good job” that are often seen in current residency evaluations.

Benefits for Residents and Fellows

You can easily track your own diagnostic concordance without waiting for it to be implemented by your program.  Use a paper worksheet or download the automatic spreadsheet to get started. Here are a few reasons why you should:

  • Helps you track your progress during training
  • Highlights your strengths and weaknesses
  • Gives direction and focus to your board study preparations
  • Helps you take personal responsibility for your diagnostic accuracy
  • Helps you down play or defend yourself against poor evaluations from faculty (for those times when your diagnostic skills are high but their personal opinion of you is low)
  • Supports you when asking for appropriate and graduated responsibilities from program director or department chair

Benefits for Program Directors (PRODS)

There is a broader amount of helpful information that can be learned when the program as a whole tracks trainee to attending concordance.  Here are a few other benefits to tracking concordance as a program:

  • Provides objective data when giving feedback to trainees
  • Supports decisions to reward or discipline residents
  • Measures quality in education in the department
  • Measures quality at the subspecialty level
  • Provides objective feedback to faculty who receive poor resident evaluations (residents may be trained well but rate faculty poorly because of conflicting personalities)
  • Provides objective data to hospital executives when advocating for increased responsibilities for your trainees at the hospital level

Conclusions

Tracking diagnostic concordance is an easy, objective, data driven solution that can improve pathology training.  Admittedly, it’s not without its own challenges.  Regardless, when improving patient safety is the goal and advances in medicine are moving at a break-neck pace, it’s a model of pathology education that’s worth exploring.

Acknowledgments

Big thanks to these great pathologists for their help!

Yasmeen Butt, MD, David A. Cohen, MD, Whitney McCarthy MD, Rashna Meunier, MD

Feedback

Does your program already track concordance between residents and faculty? Have you had success?  If you have any comments, questions, concerns, or personal experiences, please comment below.