The ACGME Survey Can Teach Milestones a Lot About Evaluations!

by | Feb 21, 2017 | Education | 0 comments

The ACGME Survey…

The ACGME Survey is a confidential questionnaire that helps the ACGME evaluate training programs. The ACGME Milestones, on the other hand, is a way for the training programs to evaluate their trainees.  Here’s how the ACGME defines the survey:

The ACGME’s Resident/Fellow and Faculty Surveys are an additional method used to monitor graduate medical clinical education and provide early warning of potential non-compliance with ACGME accreditation standards.

While filling out the semiannual ACGME survey recently, I found it very ironic that the Survey questions were strikingly better than questions found in the Milestones.  Wouldn’t it be nice if the qualities that make the Survey good could also be used to make the Milestones good as well?  Here are a few qualities about the Survey that make it better than the Milestones:

Positive qualities of the ACGME Survey:

  1. Appropriate options for the question being asked: The ACGME Survey asked various types of questions about duty hours, fear of retaliation, the learning environment, etc.  Some questions could be answered by Yes or No, while others had more answer choices.  Those with more answer choices used several scales based on the words like Satisfactory/Unsatisfactory, Adequate/Inadequate, or Always/Sometimes/Never, depending on the question that was being asked.  In stark contrast, the milestones only use a single scale 1-10 scale regardless of what is being asked.
  2.  Objective questions: With more Yes or No questions, the survey is inherently more objective. Other questions asked about specific numbers (i.e. 8 hour break between work periods) that are also more objective.  I realize there is recall bias when answering some questions but when the question is defined by a simple Yes/No or a specific number, it’s easier to give a more objective answer.
  3. Easier to understand: (See Figure 1).  With only a quick glance, you’ll easily notice how much simpler each question is.  There is a FAQ section on the ACGME website regarding the survey so there is room for interpretation in some questions but overall, it’s much easier to answer a question without any prior knowledge about the survey.
  4. More questions isn’t always a bad thing: I don’t have an exact number but there were more question in the ACGME Survey than there are in the milestones.  You may think this would take more time but when the questions are short, simple, and easy to understand, answering them goes a lot faster.

All in all, I think the writers of the Milestones could learn a lot from the writers of the ACGME Survey.  Given that these two questionnaires come from the same organization, this shouldn’t be too hard should it?

Milestones vs Survey

Figure 1: The milestones are so complex compared to the simplicity of the survey, but both are created under the direction of the ACGME!

If you, or someone you know, works with the ACGME Milestones or Survey, please let me know what you think!  You can also follow the links on the side to see previous posts about creating a resident evaluation system.