Patient-reported outcome measures (PROMs) are questionnaires that assess how patients feel and function. PROMs should be developed and validated using a mixed methods, multistep approach with extensive patient input to ensure that they are easy to understand, comprehensive, and relevant. PROMs that are specific to gender-affirming care (including surgery), such as the GENDER-Q, can be used to educate patients, align patients’ goals and preferences with realistic expectations about the surgical procedures’ purposes and outcomes, and conduct comparative effectiveness research. PROM data can contribute to evidence-based, shared decision making and just access to gender-affirming surgical care.
The Importance of Asking Patients
Gender-affirming surgery includes a range of individualized and medically necessary procedures that are performed to align an individual’s physical characteristics with their gender identity. Demand for gender-affirming surgery has grown exponentially in recent years,1 with 25% of transgender and gender diverse (TGD) individuals reporting in a 2015 survey that they had undergone some type of gender-affirming surgery.2 In parallel, there has been an upsurge in gender-affirming surgical options and technical variations.3,4 Gender-affirming surgeries are often complex, as they can involve multiple specialties, and might be irreversible. They are also associated with high costs to the health care system and the patient (eg, copays).5,6 Consequently, to provide the highest-quality and evidence-based care, it is crucial to measure and longitudinally evaluate outcomes of gender-affirming procedures and to conduct comparative effectiveness research.
To date, the measurement of outcomes in the gender-affirming surgery literature has largely focused on the clinician perspective (ie, clinical judgment or interpretation of a patient’s observable signs or physical manifestations of a condition). These clinician-reported outcomes are impairment focused and include, for example, wound healing, bleeding, nerve injury, and flap loss. However, only collecting and reporting clinician-reported outcomes overlooks the impact of gender-affirming surgeries and related complications on patients and their health-related quality of life. Patient-reported outcomes (PROs) are unobservable or latent outcomes known only by the patient and cannot be assessed using clinical observation or physical examination. PROs are symptom and function focused and may include physical symptoms (eg, pain, fatigue), functions (eg, activities of daily living, sleep, work), psychosocial well-being, and sexual well-being. These outcomes are measured using standardized and validated questionnaires (also called scales, surveys, or instruments) without the data being interpreted by a health care professional or anyone else and are called patient-reported outcome measures (PROMs).7 PROMs, including for gender-affirming care, have a number of benefits and should be developed and validated using a mixed methods, multistep approach with extensive patient input to ensure that they are easy to understand, comprehensive, and relevant.
At its core, the use of PROMs allows for systematic and meaningful inclusion of patient voice in treatment decision making and enhances patient-centered care. However, collecting and utilizing PROM data may have a multilevel impact on how health care is planned, organized, delivered, and reimbursed (see Figure).7,8,9
Figure. Multilevel Uses of PROMs Data
Previous studies have shown that completing a PROM can result in patients’ improved awareness of their health status or treatment-related effects and provide patients with relevant terminology (nano level), enabling them to better communicate with their health care team.10,11,12 At the level of patients and health care professionals (micro level), PROM data can be used to set expectations or align a treatment approach with the preferences of the patient, educate the patient, facilitate clinician-patient communication, identify pre- or postoperative concerns, prioritize health outcomes, and measure changes in health over time.13 At the level of a health care organization (meso level), systematically or routinely collected PROM data can be used to assess health outcomes over time. More specifically, patient data can be used to predict health outcomes for clinical and sociodemographic subgroups and to evaluate the comparative clinical effectiveness of treatment interventions. The PROM data also can be used to evaluate clinician performance and for peer benchmarking.14,15 The organization may use these data to evaluate program effectiveness and efficiency as well as quality assurance and improvement initiatives and to identify gaps in health care services. Lastly, PROM data are useful to health care systems (macro level) in comparing health outcomes across different organizations or jurisdictions for the purpose of informing health care reimbursement and policy decisions, ultimately providing the basis for value-based reimbursement.16,17
Broadly, there are 2 main types of PROMs: (1) generic PROMs that measure overall health or well-being or general aspects of health status and (2) condition- or treatment-specific PROMs that measure symptoms and symptom interference for a specific condition or treatment. Both generic and condition- or treatment-specific PROMs are required to meet PROM development and validation guidelines that have been put forth by the US Food and Drug Administration (FDA),18 COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments),19 the Professional Society for Health Economics and Outcomes Research (ISPOR; formerly, the International Society for Pharmacoeconomics and Outcomes Research),20,21 and similar organizations.22,23 At a minimum, the guidelines recommend that the development of a PROM should begin by defining the construct, target population, and context of use. As part of this process, extensive qualitative input should be sought from the people who experience the construct—and for whom the measure is intended—to establish the PROM’s face validity (what the PROM appears to measure from patients’ perspectives). Additionally, patient data should be used to develop questions (ie, items) for the PROM. The items should include words used by patients as much as possible, and any double-barreled, technical, or value-laden terms should be avoided. Once the items are developed, appropriate response options, recall duration, and instructions should be defined. The PROM should be piloted among patients using cognitive debriefing interviews, and expert feedback should be sought to establish the PROM’s content validity (ie, comprehensibility, comprehensiveness, and relevance). A field test study should be conducted with a large, heterogeneous sample of patients to assess the PROM’s reliability (ie, internal consistency, measurement error), construct or criterion validity (whether the PROM measures what it intended to measure), as well as its responsiveness (whether the PROM captures change over time in health status or condition).19 Scoring algorithms should be established based on the theoretical approach guiding PROM development and validation. Following this process, the PROM should be made available for clinical care and research. The PROM may be translated into other languages and culturally adapted for increased uptake using ISPOR’s best practice guidelines.24
PROM Data Collection and Implementation Considerations
PROM data collection should always start with W5H questions—why, who, what, when, where, and how (see Table). Establishing concordance between what matters to the target population (the construct of interest) and what the PROM is intended to measure is of utmost importance for a successful PROM data collection program. A core team of key stakeholders—patients, clinicians, researchers, payers, regulators, and, where applicable, caregivers, hospital administrators, or community organizations—should be established and their feedback integrated into the planning, design, implementation, and evaluation of the program. The feasibility (ease of implementation, practicality, integration with information technology such as electronic health records, and scalability) and acceptability (face validity, content validity, ethics, burden, opportunity cost) of the PROM(s) should be examined in a pilot study prior to scaling PROM data collection at the meso and macro levels.25,26
Implementation of PROMs in gender-affirming surgery at the hospital, program, system, or national level should be grounded in implementation science frameworks (deterministic and evaluative) with a focus on intersectionality (eg, the Consolidated Framework for Implementation Research27,28,29 enhanced for intersectionality30). Prior to implementation, extensive input should be sought from all stakeholders on factors that affect implementation success and scalability, including barriers to and enablers of PROM data collection, such as staff and organizational preparedness. The clinic workflows should be refined to ensure minimal logistical burden to clinic staff and patients, and the clinic staff should be trained on the collection, interpretation, and use of PROM data. Information technology-related resources (eg, data reporting, analytics) should be harnessed or developed to ensure accessible and equitable data collection. An iterative feasibility evaluation should be conducted to ensure that the preset quality indicators (eg, program fidelity, PROM completion rates) are met and that there are no gaps in the efficient and effective scaling of the PROM data collection program. Elements of PROM programs that have been linked to long-term success include identifying clinical champions, dedicated staff members and resources, ensuring stakeholders’ commitment to integrate and use PROM data, accessibility of PROM data for clinical care, and actionable feedback to patients and clinicians based on PROM data.31 Guidance is available for planning PROM implementation and selecting PROMs,22,32,33,34 implementing and evaluating PROM initiatives,29 integrating PROMs into electronic health systems,35 and visualizing PROM data.36,37
A key consideration in gender-affirming surgery is that PROs research, as it expands, should aim to reduce health- and health care-related disparities at the policy level. Efforts should be taken at the micro, meso, and macro levels to ensure that PROMs are designed and implemented in fully accessible ways and without the unintended exclusion or inundation of patient subgroups. PROMs should be made available in languages spoken by patients, require no more than a sixth-grade reading level,38,39 and employ hybrid modes and methods of data collection (eg, during a clinic visit or remotely, on mobile devices or on paper). The environment in which PROMs are administered or used should foster inclusiveness by ensuring that the staff are culturally competent, by providing accessible spaces (eg, gender-neutral washrooms), and by using intake forms that include a variety of gender and sexual identities. Data collected at the hospital system (meso) or jurisdiction (macro) level should be analyzed to identify ways to improve care quality and cost effectiveness to promote value-based health care. The analysis, use, and dissemination of PROM data at all levels should thoroughly and thoughtfully consider the impact of the data on extant health policies that fund and regulate access to gender-affirming care.
PROMs in Gender-Affirming Surgery
Although PROMs have been used to assess gender-affirming surgery outcomes for the last few decades, recently the shortcomings related to the development and psychometric properties of existing PROMs have been called to attention. Converging evidence from recent systematic reviews40,41,42 on PROMs used in gender-affirming surgery highlight 4 key issues. First, most PROMs identified in the literature were developed to be used for a specific study and therefore lack validation. Second, several PROMs that are used in the gender-affirming care literature were developed to evaluate outcomes in cisgender groups and have not been rigorously validated in gender diverse individuals (eg, the Female Genital Self-Image Scale, the International Prostate Symptom Score).40,41,42 Third, the number of PROMs used in the gender-affirming surgery literature are limited by their content or by failing to follow international guidelines for PROM development. Lastly, PROMs that comprehensively assess outcomes of specific types of treatment interventions or procedures (eg, scrotoplasty, labiaplasty) or a single body part or region (eg, forehead, jaw, facial hair) are lacking. An urgent need for a comprehensive, rigorously designed, and validated PROM to assess outcomes of gender-affirming care was identified. Our international team of clinicians, quality-of-life researchers, and psychometricians responded to this call to action by developing the GENDER-Q—a PROM for assessing outcomes in gender-affirming care.43
The GENDER-Q consists of a comprehensive set of unidimensional scales (questionnaires) that assess the domains of appearance (hair, face, neck, body, breasts, chest, genitals, donor site), health-related quality of life (physical, psychosocial, sexual, voice, practices), the experience of care (health professional, clinic, preoperative information, and outcome), and devices (catheter, testicular implants, erectile devices).43,44 To develop the GENDER-Q, our team followed international guidelines for PROM development.18,19,20,21,22,23 We conducted in-depth interviews with 84 TGD individuals from 4 countries (Canada, Denmark, the Netherlands, and the United States) who were seeking or had undergone gender-affirming treatment(s).44 The data were used to create the preliminary versions of a set of independently functioning scales. The scales were shown to 7 to 14 TGD individuals (depending on the scale) and 50 clinicians and research experts and iteratively refined, resulting in the field test version of the scales.44 The field test version was piloted in a sample of 602 English-speaking TGD individuals from 28 countries who were recruited using an online crowdsourcing platform.44 An international field test study to establish the measurement properties of the GENDER-Q is underway. The data collected will be used to refine the scales, assess their reliability and validity, and develop a common scoring algorithm for each scale for international use. Once the field test is completed, the scales and scoring will be made available for not-for-profit clinical research and care at no charge.
The GENDER-Q represents a positive ethical shift in the measurement of PROs for gender-affirming surgery, as it lays the foundation for a patient-centered health care culture that promotes the notion of “nothing about us without us,” as opposed to the current, fundamentally flawed practice of using PROMs in gender-affirming surgery that were developed for the cisgender population.
Empirically and systematically integrating rigorously developed and validated gender-affirming surgery-specific PROMs (eg, GENDER-Q) that capture what matters to patients are indispensable to patient-centered, shared treatment decision making; improving care quality; and expanding access to and funding of surgical procedures.
- Scott KB, Thuman J, Jain A, Gregoski M, Herrera F. Gender-affirming surgeries: a national surgical quality improvement project database analyzing demographics, trends, and outcomes. Ann Plast Surg. 2022;88(5)(suppl 5):S501-S507.
James SE, Herman JL, Rankin S, Keisling M, Mottet L, Anafi M. The Report of the 2015 US Transgender Survey. National Center for Transgender Equality; 2016. Accessed December 5, 2022. https://transequality.org/sites/default/files/docs/usts/USTS-Full-Report-Dec17.pdf
- Nolan IT, Dy GW, Levitt N. Considerations in gender-affirming surgery: demographic trends. Urol Clin North Am. 2019;46(4):459-465.
- Canner JK, Harfouch O, Kodadek LM, et al. Temporal trends in gender-affirming surgery among transgender patients in the United States. JAMA Surg. 2018;153(7):609-616.
- Downing J, Holt SK, Cunetta M, Gore JL, Dy GW. Spending and out-of-pocket costs for genital gender-affirming surgery in the US. JAMA Surg. 2022;157(9):799-806.
- Defreyne J, Motmans J, T’sjoen G. Healthcare costs and quality of life outcomes following gender affirming surgery in trans men: a review. Expert Rev Pharmacoecon Outcomes Res. 2017;17(6):543-556.
Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
Greenhalgh J. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18(1):115-123.
Al Sayah F, Lahtinen M, Bonsel GJ, Ohinmaa A, Johnson JA. A multi-level approach for the use of routinely collected patient-reported outcome measures (PROMs) data in healthcare systems. J Patient Rep Outcomes. 2021;5(suppl 2):98.
- Kane PM, Ellis-Smith CI, Daveson BA, et al. Understanding how a palliative-specific patient-reported outcome intervention works to facilitate patient-centred care in advanced heart failure: a qualitative study. Palliat Med. 2018;32(1):143-155.
- Kotronoulas G, Papadopoulou C, Simpson MF, McPhelim J, Mack L, Maguire R. Using patient-reported outcome measures to deliver enhanced supportive care to people with lung cancer: feasibility and acceptability of a nurse-led consultation model. Support Care Cancer. 2018;26(11):3729-3737.
Mejdahl CT, Schougaard LMV, Hjollund NH, Riiskjær E, Thorne S, Lomborg K. PRO-based follow-up as a means of self-management support—an interpretive description of the patient perspective. J Patient Rep Outcomes. 2018;2:38.
Field J, Holmes MM, Newell D. PROMs data: can it be used to make decisions for individual patients? A narrative review. Patient Relat Outcome Meas. 2019;10:233-241
- Graupner C, Kimman ML, Mul S, et al. Patient outcomes, patient experiences and process indicators associated with the routine use of patient-reported outcome measures (PROMs) in cancer care: a systematic review. Support Care Cancer. 2021;29(2):573-593.
Hughes S, Aiyegbusi OL, Lasserson D, Collis P, Glasby J, Calvert M. Patient-reported outcome measurement: a bridge between health and social care? J R Soc Med. 2021;114(8):381-388.
- Greenhalgh J, Dalkin S, Gibbons E, et al. How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis. J Health Serv Res Policy. 2018;23(1):57-65.
- Squitieri L, Bozic KJ, Pusic AL. The role of patient-reported outcome measures in value-based payment reform. Value Health. 2017;20(6):834-836.
US Food and Drug Administration. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. US Department of Health and Human Services; 2009. Accessed February 23, 2023. https://www.fda.gov/media/77832/download
- Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539-549.
- Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2—assessing respondent understanding. Value Health. 2011;14(8):978-988.
- Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 1—eliciting concepts for a new PRO instrument. Value Health. 2011;14(8):967-977.
Montgomery N, Howell D, Ismail Z, et al. Selecting, implementing and evaluating patient-reported outcome measures for routine clinical use in cancer: the Cancer Care Ontario approach. J Patient Rep Outcomes. 2020;4(1):101.
- Aaronson N, Alonso J, Burnam A, et al. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193-205.
- Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94-104.
- Bowen DJ, Kreuter M, Spring B, et al. How we design feasibility studies. Am J Prev Med. 2009;36(5):452-457.
Brusco NK, Atkinson V, Woods J, et al. Implementing PROMS for elective surgery patients: feasibility, response rate, degree of recovery and patient acceptability. J Patient Rep Outcomes. 2022;6(1):73.
- Breimaier HE, Heckemann B, Halfens RJ, Lohrmann C. The consolidated framework for implementation research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nurs. 2015;14(1):1-9.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
- Stover AM, Haverman L, van Oers HA, Greenhalgh J, Potter CM. Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings. Qual Life Res. 2021;30(11):3015-3033.
Intersectionality-enhanced consolidated framework for implementation research. Knowledge Translation Program. February 2020. Accessed February 22, 2023. https://knowledgetranslation.net/wp-content/uploads/2020/02/Intersectionality_Enhanced_CFIR_Table_final.pdf
- Sokas C, Hu F, Edelen M, Sisodia R, Pusic A, Cooper Z. A review of PROM implementation in surgical practice. Annals Surg. 2022;275(1):85-90.
Aaronson N, Elliott T, Greenhalgh J, et al. User’s guide to implementing patient-reported outcomes assessment in clinical practice. Version 2. International Society for Quality of Life Research; 2015. Accessed February 23, 2023. https://www.isoqol.org/wp-content/uploads/2019/09/2015UsersGuide-Version2.pdf
van der Wees PJ, Verkerk EW, Verbiest MEA, et al. Development of a framework with tools to support the selection and implementation of patient-reported outcome measures. J Patient Rep Outcomes. 2019;3(1):75.
- Chan EKH, Edwards TC, Haywood K, Mikles SP, Newton L. Implementing patient-reported outcome measures in clinical practice: a companion guide to the ISOQOL user’s guide. Qual Life Res. 2019;28(3):621-627.
Snyder C, Wu AW, eds. Users’ Guide to Integrating Patient-Reported Outcomes in Electronic Health Records. Johns Hopkins University; 2017. Accessed April 21, 2023. https://www.pcori.org/sites/default/files/PCORI-JHU-Users-Guide-To-Integrating-Patient-Reported-Outcomes-in-Electronic-Health-Records.pdf
- Snyder CF, Smith KC, Bantug ET, Tolbert EE, Blackford AL, Brundage MD. What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer. 2017;123(10):1848-1859.
- Snyder C, Smith K, Holzner B, Rivera YM, Bantug E, Brundage M. Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data. Qual Life Res. 2019;28(2):345-356.
Weiss BD. Health literacy: a manual for clinicians. American Medical Association Foundation; American Medical Association; 2003. Accessed February 23, 2023. http://lib.ncfh.org/pdfs/6617.pdf
Clear communication: clear and simple. National Institutes of Health. Updated July 7, 2021. Accessed February 23, 2023. https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/clear-simple
- Dy GW, Nolan IT, Hotaling J, Myers JB. Patient reported outcome measures and quality of life assessment in genital gender confirming surgery. Transl Androl Urol. 2019;8(3):228-240.
- Andréasson M, Georgas K, Elander A, Selvaggi G. Patient-reported outcome measures used in gender confirmation surgery: a systematic review. Plast Reconstr Surg. 2018;141(4):1026-1039.
- Barone M, Cogliandro A, Di Stefano N, Tambone V, Persichetti P. A systematic review of patient-reported outcome measures following transsexual surgery. Aesthetic Plast Surg. 2017;41(3):700-713.
Klassen AF, Kaur M, Johnson N, et al. International phase I study protocol to develop a patient-reported outcome measure for adolescents and adults receiving gender-affirming treatments (the GENDER-Q). BMJ Open. 2018;8(10):e025435.
Gender-Q. Accessed December 5, 2022. https://qportfolio.org/gender-q/gender-q/