From Culture to Testing: ABMS Member Boards Highlight DEI Efforts

Speakers at ABMS Conference 2022 offered a glimpse of how American Board of Medical Specialties (ABMS) Member Boards are addressing diversity, equity, and inclusion (DEI) and a broader view of enhancing justice and health equity.

DEI a Top Priority at ABIM

Pamela Browner White, Senior Vice President of Communications and Chief DEI Officer for the American Board of Internal Medicine , and Lorna Lynn, MD, ABIM’s Vice President for Medical Education Research and Staff Administrator for the Board of Directors’ DEI Committee, discussed AIM’s strategic vision and workplan to address DEI. The foundation of this work is rooted in ABIM’s racial justice pledge, created in summer 2020 following the murder of George Floyd. The pledge acknowledges that “it’s simply not enough to say passively we will ‘do no harm.’” It continues, “We pledge actively to do our part in opposing and dismantling systems and policies that cause harm to our patients and disproportionately affect those in Black and Brown communities.” Furthermore, ABIM commits to analyzing its programs for potential disparate impact on candidates because of race, ethnicity, gender, or country of birth; be transparent about its findings; and address any inequity to which ABIM may be contributing.

Among the initiatives ABIM has undertaken is developing health equity content for its exams. Additionally, ABIM is learning from and working collaboratively with other Member Boards and specialty societies to advance DEI efforts. Dr. Lynn credits the Association of American Medical Colleges (AAMC) for helping ABIM obtain race and ethnicity data on physicians who are board certified by ABIM, which enabled the board to develop its first-ever reports on the internal medicine workforce that include race and ethnicity. These reports will be updated periodically and help to monitor progress in widening the pathway into the profession for individuals underrepresented in internal medicine.

ABIM is moving beyond engaging with its society partners and other ABMS boards, which has largely been specialty centric, to develop partnerships with the National Medical Association , National Hispanic Medical Association (NHMA) , American Medical Women's Association (AMWA) , and American Medical Student Association (AMSA) , among others, to incorporate the voices of those with unique perspectives as they move through their medical careers. The ABIM Foundation is funding projects that address health care disparities. Browner White noted that 90 percent of the ABIM Foundation’s work is in the DEI realm. ABIM has made tremendous progress in developing more diversity among its specialty boards, Dr. Lynn said. As they become more diverse, the decisions they make about ABIM programs and policies become smarter and wiser. Browner White added, “Internally, ABIM is working to create a diverse and inclusive workplace culture that attracts and retains top talent.” Along those lines, it has created a new employee orientation with a DEI focus and conducts ongoing staff and leadership training in DEI. ABIM recently launched a DEI webpage that highlights these and other DEI initiatives.

Bias, Fairness, and Validity in Testing

As certifying bodies, it’s incumbent upon the ABMS Member Boards to construct tests that are fair and unbiased, stated Linda Althouse, PhD, Vice-President of Assessment at the American Board of Pediatrics (ABP)

Treating test-takers equally is the goal in standardized testing, noted John Weiner, Chief Science Officer of PSI, an assessment services company that provides test development and test delivery for high-stakes licensing, certification, qualifications, and exams. But that approach often results in different pass rates for different demographic groups, triggering claims of unfairness. He reviewed four general views of fairness: being treated equally during the testing process, lack of measurement bias, access to constructs measured, and validity of score interpretations for intended uses. Scientifically speaking, group differences in outcomes alone are not evidence of bias or unfairness, Weiner said. Still, when group differences are manifested, they should be investigated.

Distinguishing between fairness and bias, Dr. Althouse explained that bias is one component of fairness. If a test is statistically biased, the testing process can’t be fair. However, a testing process can still be unfair even if there is no statistical bias present as other factors play into that determination. Bias is any preference or inclination, even if it’s implicit, that prevents an impartial judgment and results in an inaccurate measurement. Basically, this occurs when a test score is not interpreted the same for all subgroups.

Dr. Althouse reviewed the main approaches for finding and addressing measurement bias, noting that ABP uses a differential item functioning (DIF) analysis and a bias/sensitivity review panel for its general pediatrics exam. The DIF analysis uses statistics to determine whether any items performed differently for different subgroups while the bias/sensitivity review panel determines if the identified items are potentially biased based on content. The preventive approach involves following best practices/standards from item writing training through analysis following exam administration. The strongest approach is to combine all three, she stated.

Weiner concluded by reviewing solutions to address fairness, including measuring additional relevant constructs. Research has shown that when non-cognitive and behavioral competencies are also measured that validity increases and demographic subgroup differences tend to decrease. He noted that organizations are investigating competency modeling approaches and incorporating behavioral and non-cognitive aspects in their assessments. Another approach is to adjust scoring. How different components of an assessment are weighted, how decisions in a multi-stage assessment are ordered, and where the passing score is set, can all impact validity and subgroup differences in test performance. An additional effective approach is to use alternative assessment methods and modes, such as multi-media and interactive methods, in addition to written exams as part of the assessment. These alternative methods may reduce barriers and extraneous factors that are unrelated to the competencies being measured.

Consciously Embracing Unconscious Bias

Everybody has unconscious bias. It’s just a matter of being aware of one’s bias in order to address and mitigate it, explained Jennifer Hopp, SHPR, SHRM-CP, CDE, Senior Human Resources Manager at the The American Board of Anesthesiology . Bias originates in the brain, she explained. The brain creates mental shortcuts to handle the large amount of information people process daily. The brain creates assumptions and generalizations to help make decisions quickly. Individuals make lightning-fast generalizations based on these shortcuts, which are drawn from previous experiences. Acting on their biases also helps humans survive. For example, a child who was bitten by a dog may develop a fear of dogs and avoid them to prevent getting bitten again.

An individual’s unconscious bias, however, is different than a company’s systemic bias, the latter of which is perpetuated by unfair policies, recruiting methods, compensation and benefits, worksite accessibility, meeting formats, and dress codes. Even hybrid work environments create a disparity between employees’ experiences based on whether they work from home or go into the office. Hopp discussed common forms of bias in the workplace, noting that if bias goes unchecked, it could lead to disengagement, turnover, stagnation, burnout, and conflict.

Among the suggestions Hopp gave to address an organization’s culture were to bring awareness to bias and how it shows up daily, as well as manage behaviors by creating cultural norms/code of conduct and holding everyone accountable. Defining the “cultural fit” is important because it helps people know what it looks like, she said. ABA convened a task force to create a code of conduct. Most importantly, create psychological safety in the organization, enabling people to feel comfortable talking about bias. That starts at the leadership level, she noted.

To address systemic bias, Hopp recommended examining organizational policies. Striving for diverse candidate pools, ABA uses panel interviews because it allows everyone who will be working with the candidate to provide feedback. Evaluate compensation annually and look specifically for pay inequities that lack an explanation. ABA uses a pay scale for its salary structure, which enables her to explain objectively why an employee is getting a specific salary. Throw out the dress code as it creates a gender bias and excludes non-binary employees. Ensure policy language is inclusive and examine promotion policies to ensure that opportunities to advance are fair for all employees. After identifying gaps in policies and practices that are not inclusive, set goals to work toward closing those gaps. This work should involve everyone, both leaders and staff as it is a team effort. Finally, understand that change takes time and commitment, and mistakes will happen.?

Modeling Effective Change Strategies

Dr. Amit Joshi, MD, Editor-in-Chief for the SCORE Surgical Council on Resident Education (SCORE), discussed how the American Board of Surgery SCORE curriculum is an ideal vehicle for providing DEI content and self-assessment to surgical trainees. It also has helped promote DEI among surgical leaders. SCORE is a national surgical curriculum to which 95 percent of all residency programs across the country subscribe, representing 500 residency programs and approximately 10,000 trainees. All trainees have equal access and usage of SCORE. When DEI was built into ABS’ 2022 Strategic Plan, SCORE began to develop DEI content, such as principles of disparities in surgical care, principles of DEI in the surgical workforce, and allyship for surgeons. DEI concepts also were added to some existing modules and images were reviewed to ensure they represent the diversity of patients.

Greg Laszakovits, MDiv, Chief Executive Officer and Founder of GDL Insight , highlighted three strategies that can close the gap between DEI gestures and real investment in DEI efforts. Unfortunately, even well-intentioned DEI efforts are falling flat at organizations. According to Harvard Business Review , only 33 percent of employees rate their organization’s DEI efforts “successful,” even fewer say DEI is a high priority. Many organizations indicate that lack of leadership commitment hinders these efforts. The first strategy is to include members of non-privileged groups in the conversation. The second is to provide support and accountability for leaders, who typically are among the privileged group. These leaders are good people who don’t want to make a mistake or offend somebody, he said. But the reality is that mistakes will be made, Laszakovits added, and the alternative is remaining paralyzed, which is worse. Third, engage a firm to help drive the process and keep it moving forward. Expecting a group of volunteers to address such a complex and sometimes touchy subject as DEI is unrealistic.

Kamry Goodwin, Team Lead of Assessment Development at The American Board of Anesthesiology , discussed using assessments to promote systemic change, enhance DEI, and minimize health care disparities. In 2021, new content on health care disparities and DEI in the workplace were added to the Maintenance of Certification in Anesthesiology (MOCA) Minute?. The ABA Board of Directors expressed interest in adding these topics to the content outlines for written examinations. Twelve volunteers were selected to form the Content Outline Advisory Task Force for DEI/Health care Disparities. The task force submitted 29 items and 134 references. Of those, 14 items were approved by the Exams Chairs group and will appear on written exams, including the In-Training Exam, beginning in 2023. Sexual orientation, gender, and gender identity will be added to the list of DEI barriers. The task force will remain active and continue developing new items.

Using Education to Enhance Justice, Health Equity

It’s important to “unlearn” fallacies from within medical education and society before being able to develop and implement equitable education, stated Jodi Abbott, MD, MHCM, Medical Director of the American Medical Association Ed Hub. As an example, race is a socially constructed way of viewing different groups of individuals, often based on skin color or other physical qualities, explained Emily Cleveland Manchanda, MD, MPH, Director for Social Justice Education and Implementation at the AMA Center for Health Equity. In the United States, people are racialized into one of five categories based on the current census list, a list that has evolved over time. Despite the fact that race has no genetic or biologic basis, it has been and continues to be used in problematic ways, including in the measurement of lung function or recommendations for which medications to use. Ingrained beliefs about racial differences lead to false beliefs, such as “Blacks don’t feel pain as much as whites.” These fallacies have ongoing real-world implications for individuals who are marginalized or minoritized. According to data compiled by the Racial Equity Institute, Black people have worse outcomes than white people when it comes to systems of health, education, criminal/legal, child welfare, and finances, she said. The corollary of racism is privilege. In the U.S., the dominant group within society are white people. White privilege does not mean that one’s life has not been hard, it just means it has not been harder because of one’s skin color.

Dr. Cleveland Manchanda provided numerous examples of race-based medicine, noting that some of these practices are being recognized and fixed. For example, the equations to measure kidney function have been recalculated because they were based on a flawed premise that led to delayed treatment for Black patients. These inequities extend to the medical profession for physicians and trainees whether they are from a historically marginalized or minoritized racial or ethnic group, or women practicing in a historically or contemporarily male specialty, she said.

Dr. Abbott discussed efforts that AMA has made as part of its organizational strategic plan to move forward out of its racist past. She highlighted the Advancing Health Equity: A Guide to Language, Narrative and Concepts, which provides guidance for physicians and other health care professionals and promotes a deeper understanding of equity-focused, person-first language and why it matters. The guide, which AMA partnered with the AAMC to publish, also encourages physicians to address the upstream social causes of health inequities and to center care around their patients’ lived experiences. AMA also developed tools to help embed equity into medical educational content. Embedding equity in organizations requires educating teams for equity, developing equitable structures, processes, and accountabilities; integrating trauma informed lens into management and leadership; and assessing organizational change over time, Dr. Abbott concluded.

To subscribe to future issues of ABMS Insights, visit: https://www.abms.org/newsletter-sign-up/.