Accreditation Council for Graduate Medical Education Core Competencies at a Community Teaching Hospital: Is There a Gap in Awareness?

Tweet

Mohammed Al-Temimi, MD, MPH; Michael Kidon; Samir Johna, MD, MACM

Perm J 2016 Fall;20(4):16-067 [Full Citation]

https://doi.org/10.7812/TPP/16-067

E-pub: 10/14/2016

ABSTRACT

Context: Reports evaluating faculty knowledge of the Accreditation Council for Graduate Medical Education (ACGME) core competencies in community hospitals without a dedicated residency program are uncommon.
Objective: Faculty evaluation regarding knowledge of ACGME core competencies before a residency program is started.
Design: Physicians at the Kaiser Permanente Fontana Medical Center (N = 480) were surveyed for their knowledge of ACGME core competencies before starting new residency programs.
Main Outcome Measures: Knowledge of ACGME core competencies.
Results: Fifty percent of physicians responded to the survey, and 172 (71%) of respondents were involved in teaching residents. Of physicians who taught residents and had complete responses (N = 164), 65 (39.7%) were unsure of their knowledge of the core competencies. However, most stated that they provided direct teaching to residents related to the knowledge, skills, and attitudes stated in each of the 6 competencies as follows: medical knowledge (96.3%), patient care (95.7%), professionalism (90.7%), interpersonal and communication skills (86.3%),  practice-based learning (85.9%), and system-based practice (79.6%). Physician specialty, years in practice (1-10 vs > 10), and number of rotations taught per year (1-6 vs 7-12) were not associated with knowledge of the competencies (p > 0.05); however, full-time faculty (teaching 10-12 rotations per year) were more likely to provide competency-based teaching.
Conclusion: Objective assessment of faculty awareness of ACGME core competencies is essential when starting a residency program. Discrepancy between knowledge of the competencies and acclaimed provision of competency-based teaching emphasizes the need for standardized teaching methods that incorporate the values of these competencies.

INTRODUCTION

It is estimated that the number of graduate medical education residency positions for medical school graduates has grown by 1.7% per year over the past decade.¹ However, this trend is not expected to fulfill the actual projected need of physicians, with an anticipated shortfall of 41,000 to 90,000 physicians by 2025.² As the demand for new training positions³ continues to exist, the Accreditation Council for Graduate Medical Education (ACGME) instituted standard requirements for accrediting new residency programs.⁴ These requirements were designed to ensure that each resident will develop the knowledge, skills, and attitudes required to enter unsupervised practice by the end of their training.⁴

The ACGME mandated that all residency programs realign their medical education and residents’ evaluation around six core competencies: medical knowledge, patient care, professionalism, interpersonal and communication skills (ICS), practice-based learning and improvement (PBLI), and system-based practice.⁵ In 2013, the ACGME went a step further by implementing the Next Accreditation System.⁶ In the Next Accreditation System, the evaluation of every residency program is shifted from a process-oriented procedure into an outcome-based procedure that is based on the six ACGME core competencies.^5-7 Because the milestones in the Next Accreditation System are considered the natural progression of the work on the six competencies, faculty awareness of those competencies is considered paramount. In fact, the ACGME expects program faculty to “administer and maintain an educational environment conducive to educating residents in each of the ACGME competency areas.”⁵ However, faculty evaluation in regard to the knowledge and delivery of competency-based education is left to the discretion of the Program Director^5,6 and, more recently, residents’ evaluation of faculty.⁶

This study focuses on the assessment of faculty knowledge of the six ACGME core competencies in a single community teaching hospital. We conducted the survey as part of needs assessment before establishing new residency programs in general surgery, internal medicine, and psychiatry. We hypothesize that at the time of starting a new residency program, objective evaluation of faculty knowledge of the ACGME core competencies is essential. This assessment can help to identify gaps in knowledge that potentially could be addressed before a residency program is started.

METHODS

We surveyed 480 physicians specializing in family medicine, internal medicine, sports medicine, geriatric medicine, psychiatry, pediatrics, general surgery, plastic surgery, obstetrics and gynecology, orthopedics (listed under other specialties), and neurosurgery at the Kaiser Permanente (KP) Fontana Medical Center in CA. The purpose of this survey was to explore faculty knowledge of the ACGME core competencies and to assess their implementation of competency-based teaching. At the time of this evaluation, the general surgery, neurosurgery, plastic surgery, and obstetrics and gynecology services were already involved in teaching residents rotating from affiliated residencies; however, the affiliated residency programs were planning to withdraw their residents from our hospital. As such, the KP Fontana Medical Center was planning to start a general surgery residency program integrated with Arrowhead Regional Medical Center, in Colton CA. The Fontana Medical Center also was going to be the sponsoring institution for the internal medicine and psychiatry residency programs. These programs were added to the family medicine residency program, which has been established since 1975.⁸

Pilot testing of the survey was performed among eight students and four faculty members from the University of Southern California Master of Academic Medicine program. The survey questions were revised on the basis of feedback from the participants. The standardized survey was sent electronically. The survey was resent twice, for a total of three times, to improve the participation rate. A copy of the survey can be accessed online at www.thepermanentejournal.org/files/16-067.pdf.

We categorized the participants by their specialty (medical vs surgical), years in practice (1-10 vs > 10 years), and the number of monthly rotations taught per year (1-3, 4-6, 7-9, and 10-12). Full-time faculty was defined as a faculty member who teaches 10 to 12 rotations per year. Surgical specialties included general surgery, plastic surgery, obstetrics and gynecology, orthopedics (listed as “other”), and neurosurgery. Medical specialties included family medicine, internal medicine, sports medicine, psychiatry, pediatrics, and geriatric medicine.

The six competencies as they appeared on the survey included medical knowledge, patient care, professionalism, ICS, PBLI, and system-based practice. We asked faculty members to define the degree of their own level of knowledge of the ACGME core competencies as “not sure,” “know the names of the six competencies,” or “well versed.” The last category included response choices for faculty who “have worked with the subcompetencies/milestones” or “have developed objectives or curricula based on the competencies.” Additionally, we asked faculty physicians about their perception of whether they had incorporated each individual core competency in their teaching of residents. Finally, faculty members were asked to assess their residents’ performance across the six core competencies. The residents’ performance was evaluated as a “strength,” “weakness,” or “neither a strength nor a weakness.”

A c² test was used to test the association of physicians’ characteristics with their knowledge of the competencies and whether they provided resident teaching that is pertinent to the objectives stated in each core competency. The study was approved by the institutional review board of KP Southern California. We used STATA/IC Version 11.2 for all statistical analyses (StataCorp, College Station, TX).

RESULTS

Among 480 hospital physicians who were surveyed, 242 (50.4%) responded to the survey. Among the respondents, 172 faculty physicians actively instructed residents, of whom 164 had complete responses. Seventy physicians who responded to the survey did not teach residents. The data of those who taught residents with complete survey responses were analyzed. Fifty-three (32.3%) faculty physicians taught 7 or more rotations per year. Only 33 (20.1%) physicians were full-time teaching faculty (teaching 10-12 rotations per year). One hundred thirty-three (81.1%) of the faculty physicians were teaching medical residents, and 31 (18.9%) were involved in teaching surgical residents (Table 1). Figure 1 demonstrates a comparison of faculty knowledge of ACGME core competencies. Teaching (faculty) physicians were more likely to know the names of (33.5% vs 27.9%) or to be well versed in (26.8% vs 4.4%, p < 0.01) the core competencies compared with nonteaching (nonfaculty) physicians.

Most of the faculty physicians surveyed indicated that they teach the knowledge, skills, and attitudes that make up each of the core competencies as follows: medical knowledge (96.3%), patient care (95.7%),  professionalism (90.7%), ICS (86.3%), PBLI (85.9%), and system-based practice (79.6%; Figure 2). Faculty characteristics, such as specialty (medical vs surgical), years in practice (1-10 vs >10), and number of rotations taught per year (1-6 vs 7-12) were not associated with the level of knowledge of the core competencies (p > 0.05). Regardless of the faculty characteristics, the percentage of faculty who were “unsure” of their knowledge of the competencies was high (n = 65, 39.6%; Table 2).

Junior (< 10 years in practice) and full-time (teaching 10-12 rotations per year) faculty were more likely than senior faculty and faculty who teach less than 10 rotations per year to endorse competency-based education. These findings were particularly demonstrated for junior faculty who emphasized medical knowledge in residents’ education (100% vs 93.2% for senior faculty, p = 0.021), and for full-time faculty endorsement of professionalism (p = 0.003) and ICS (p = 0.006; Table 3).

Faculty perception of residents’ performance in relation to the 6 core competencies is demonstrated in Table 4. Most of the faculty thought that residents’ performance in relation to the 6 core competencies was not a weakness of the program. When considering residents’ performance, 11.6% of faculty thought that medical knowledge was a program weakness. Such weakness was less commonly perceived for system-based practice (6.7%), PBLI (6.1%), patient care (2.4%), ICS (2.4%), and professionalism (2.4%).

DISCUSSION

Competency-Based Teaching

Our study shows that at the time of starting new residency programs (in general surgery, psychiatry, and internal medicine) at a community teaching hospital, only 60% of faculty physicians had some knowledge (well versed or knew the names) of the ACGME core competencies. However, most (80%-96%) of those faculty indicated their dedication and active participation in teaching the principles and goals outlined in the competencies. In addition, the competencies were not equally ingrained in the faculty members’ teaching methods, with PBLI and system-based practice being the least commonly emphasized. The lack of awareness of the core competencies came in contrast to the acclaimed implementation of competency-based teaching and led our program to establish faculty development educational seminars to improve faculty awareness of the standards of medical education. If the numbers from our study were generalizable to community hospitals without established residency programs, they should alert residency program directors to the importance of conducting needs assessment before starting a residency program. Such assessment should objectively clarify faculty knowledge of the 6 core competencies and competency-based teaching. Such assessment would also be useful for the ACGME in assessing whether the faculty members of a newly proposed residency program have the necessary information regarding educational goals that are set by the ACGME.

Our findings that some physicians from community hospitals without dedicated teaching programs have had gaps in knowledge of the standards of medical education (as set by the ACGME) are important for hospitals that plan to start a new residency program. Faculty members are expected to be well versed in the six core competencies to be able to deliver competency-based education to residents as required by ACGME. A needs assessment survey is potentially an easy and efficient tool to identify such gaps before the start of a residency program. One approach to improve faculty knowledge of the ACGME standards of education is using faculty development seminars to improve competency-based teaching knowledge and skills.^9,10 Our program adopted this strategy to address the gap in knowledge of the six core competencies. Gaps in physicians’ knowledge of the educational goals can be far more than expected, and we hypothesize that using educational seminars might become an effective tool to address such deficiency in knowledge.

Our study also shows that individual components of the ACGME core competencies were variably emphasized during residents’ education, suggesting the need for systematic and standardized teaching methods to emphasize all six competencies with equal importance. Our data show that among the six competencies, PBLI and system-based practice were the least likely to be emphasized competencies in residents’ education. It is not clear whether these findings are related to lack of conceptual understanding of those competencies or to inadequate practical opportunities to apply them. In fact, these competencies were shown to be conceptually difficult for residents and faculty.¹¹ Therefore, residency programs that instituted resident and faculty teaching that specifically target those competencies seem to have success in improving PBLI and system-based practice for residents and faculty.^11,12

Certain faculty characteristics seemed to affect their implementation of specific competency-based teaching. Full-time faculty members were more likely to emphasize each of the individual competencies compared with faculty who taught fewer than 6 rotations per year. This was particularly true for teaching professionalism and ICS. Because the understanding and teaching of professionalism and ICS can be challenging,¹³ their emphasis by full-time faculty (compared with part-time faculty) might be coming from their more extensive interaction with residents and appreciation of the importance of those competencies. Full-time status might also reflect the faculty interest in resident education with subsequent self-commitment to understanding the educational requirements for residents’ education.

Junior faculty members (< 10 years since graduation) were also more likely to emphasize the core competencies. This finding might be explained by ACGME’s enforcement of competency-based education, resulting in early and extensive exposure of junior faculty to the values of those competencies during their training. These findings will help identify faculty characteristics that are important for new community-hospital residency programs in meeting the ACGME requirements for residents’ education.

Study Limitations

Our study has many limitations. First, the results of this study may not be generalizable because of the limited subject population from a single community hospital. Furthermore, our pool of physicians may not simulate a true faculty distribution at a community teaching hospital, given that only 20% of the surveyed physicians were full-time faculty. Second, we did not objectively assess the effect of the educational seminars on the faculty teaching behaviors and knowledge. Third, the study evaluated faculty perception of implementing competency-based education to achieve residents’ mastery of the competencies; however, those answers are subjective and may be biased. Unfortunately, objective assessment of faculty teaching methods can be challenging and difficult to measure.¹⁴ Fourth, the response rate to our survey was 50%, which can affect the validity of our results; however, the response rate in our study is comparable to that of other studies in the literature.¹⁵ One potential explanation of this response rate is the high number of nonteaching physicians in our hospital. Even in our survey responses, nonteaching physicians formed 28% (n = 70) of the responding physicians.

Fifth, we do not have information about the characteristics of those who did not respond to the survey, which could have biased our results. Sixth, our survey spanned multiple specialties, whereas our intended new residency programs were limited to a few of them. This might limit generalizing the final findings to the faculty from specialties with the newly proposed residency programs. However, our findings were not different between medical and surgical specialties overall. In addition, faculty knowledge of the competencies is not expected to be different by specialty. Seventh, although associations between faculty characteristics and knowledge of the competencies were noticed, these associations do not necessarily imply causality.

Despite these limitations, we presented our experience of conducting a needs assessment before starting new residency programs, which might be helpful for community hospitals planning to begin a new residency program. We highlighted the important findings, lessons learned, and the methods used to address these findings.

CONCLUSION

Awareness of the ACGME core competencies among faculty of new residency programs in community teaching hospitals cannot be assumed without objective assessment; therefore, needs assessment is essential to identify deficits in requirements for starting a new residency program. Although most teaching physicians actively enforce teaching the knowledge surrounding the core competencies, their knowledge of those competencies could be improved. The use of explicit directions during workshops is one of the approaches that has a tangible effect on instilling the virtues of the core competencies. The lack of faculty orientation to the core competencies could be further investigated to include their source of current medical educational guidelines as well as their graduating institutions’ degree of raising awareness of the competencies. Further research also is needed to objectively evaluate physicians’ teaching methods as they relate to the goals stated for each competency.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Acknowledgements

Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.

How to Cite this Article

Al-Temimi M, Kidon M, Johna S. Accreditation council for graduate medical education core competencies at a community teaching hospital: Is there a gap in awareness? Perm J 2016 Fall;20(4):16-067. DOI: https://doi.org/10.7812/TPP/16-067.

References

1.    Salsberg E. The 2014 GME residency match results: is there really a “GME squeeze”? Health Aff Blog [Internet] 2014 Apr 24 [cited 2016 Apr 7]:[about 3 p]. Available from: http://healthaffairs.org/blog/2014/04/24/the-2014-gme-residency-match-results-is-there-really-a-gme-squeeze/.
    2.    Dall T, West T, Chakrabarti R, Iacobucci W; IHS Inc. The complexities of physician supply and demand: projections from 2013 to 2025 [Internet]. Washington, DC: Association of American Medical Colleges; 2015 Mar [cited 2016 Apr 7]. Available from: www.aamc.org/download/426248/data/thecomplexitiesofphysiciansupplyanddemandprojectionsfrom2013to2.
    3.    Meagher AD, Beadles CA, Sheldon GF, Charles AG. Opportunities to create new general surgery residency programs to alleviate the shortage of general surgeons. Acad Med 2016 Jun;91(6):833-8. DOI: https://doi.org/10.1097/ACM.0000000000001005.
    4.    ACGME common program requirements [Internet]. Chicago, IL: Accreditation Council for Graduate Medical Education; 2007 Feb 11 [revised 2016 Jul 1; cited 2016 Feb 7]. Available from: www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/CPRs_07012016.pdf.
    5.    Derstine P. Improvement through the application of the competencies. Guest editor’s introduction: turning the challenge into opportunities. ACGME Bulletin [Internet] 2006 Apr [cited 2016 May 28]:[p 1, 3]. Available from: www.acgme.org/Portals/0/PFAssets/bulletin/bulletin04_06.pdf.
    6.    Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med 2012 Mar 15;366(11):1501-6. DOI: https://doi.org/10.1056/NEJMsr1200117.
    7.    Kavic MS. Competency and the six core competencies. JSLS 2002 Apr-Jun;6(2):95-7.
    8.    Welcome to the Kaiser Permanente family medicine training program at Fontana Medical Center [Internet]. Fontana, CA: Kaiser Permanente Southern California; 2016 [cited 2016 May 29]. Available from: https://residency-scal-kaiserpermanente.org/residency/familymedicine-fontana/.
    9.    Lee SM, Lee MC, Reed DA, et al. Success of a faculty development program for teachers at the Mayo Clinic. J Grad Med Educ 2014 Dec;6(4):704-8. DOI: https://doi.org/10.4300/JGME-D-14-00139.1.
    10.    Conforti LN, Ross KM, Holmboe ES, Kogan JR. Do faculty benefit from participating in a standardized patient assessment as part of rater training? A qualitative study. Acad Med 2016 Feb;91(2):262-71. DOI: https://doi.org/10.1097/ACM.0000000000000838.
    11.    Ziegelstein RC, Fiebach NH. “The mirror” and “the village”: a new method for teaching practice-based learning and improvement and systems-based practice. Acad Med 2004 Jan;79(1):83-8.
    12.    Wilper AP, Smith CS, Weppner W. Instituting systems-based practice and practice-based learning and improvement: a curriculum of inquiry. Med Educ Online 2013 Sep 16;18:21612. DOI: https://doi.org/10.3402/meo.v18i0.21612.
    13.    Gaiser RR. The teaching of professionalism during residency: why it is failing and a suggestion to improve its success. Anesth Analg 2009 Mar;108(3):948-54. DOI: https://doi.org/10.1213/ane.0b013e3181935ac1.
    14.    Shah D, Goettler CE, Torrent DJ, et al. Milestones: the road to faculty development. J Surg Educ 2015 Nov-Dec;72(6):e226-35. DOI: https://doi.org/10.1016/j.jsurg.2015.06.017.
    15.    Pololi LH, Evans AT, Civian JT, et al. Faculty vitality—surviving the challenges facing academic health centers: a national survey of medical faculty. Acad Med 2015 Jul;90(7):930-6. DOI: https://doi.org/10.1097/ACM.0000000000000674.