Equitable Research PRAXIS: A Framework for Health Informatics Methods

Summary

Objectives: There is growing attention to health equity in health informatics research. However, the literature lacks a comprehensive framework outlining critical considerations for health informatics research with marginalized groups.

Methods: Literature review and experiences from nine equity-focused health informatics conducted in the United States and Canada. Studies focus on disparities related to age, disability or chronic illness, gender/sex, place of residence (rural/urban), race/ethnicity, sexual orientation, and socioeconomic status.

Results: We found four key equity-related methodological considerations. To assist informaticists in addressing equity, we contribute a novel framework to synthesize these four considerations: PRAXIS (Participation and Representation, Appropriate methods and interventions, conteXtualization and structural competence, Investigation of Systematic differences). Participation and representation refers to the necessity for meaningful participation of marginalized groups in research, to elevate the voices of marginalized people, and to represent marginalized people as they are comfortable (e.g., asset-based versus deficit-based). Appropriate methods and interventions mean targeting methods, instruments, and interventions to reach and engage marginalized people. Contextualization and structural competence mean avoiding individualization of systematic disparities and targeting social conditions that (re-)produce inequities. Investigation of systematic differences highlights that experiences of people marginalized according to specific traits differ from those not so marginalized, and thus encourages studying the specificity of these differences and investigating and preventing intervention-generated inequality. We outline guidance for operationalizing these considerations at four research stages.

Conclusions: This framework can assist informaticists in systematically addressing these considerations in their research in four research stages: project initiation; sampling and recruitment; data collection; and data analysis. We encourage others to use these insights from multiple studies to advance health equity in informatics.

Keywords

Health equity - health disparities - research methods - marginalized populations - conceptual frameworks

1 Introduction

There has been growing international attention to health equity in health informatics research. Journals have published special issues on the topic [[1], [2]], and there have been panels, workshops, and a published research agenda [[3]]. Equity is a worthy focus and corrective for our field, where there have been poorly representative study samples [[4]], differential intervention uptake [[5]], and biases in deployed technologies [[6]]. Moreover, there is an expanding yet dispersed literature on equity-focused research methods. However, the literature lacks a comprehensive framework outlining critical considerations for research with marginalized groups, which is a step towards achieving equity in technology-enabled health care and public health. This paper outlines and synthesizes these considerations and developments while providing practical examples and citations for further reading.

We found four key equity-related methodological considerations based on our experiences with nine equity-focused research projects ([Table 1] and [Online Appendix A]). We contribute a new framework to synthesize these four considerations: PRAXIS. This framework refers to (acronym parts are bold and underlined): (1) Participation and Representation; (2) Appropriate Methods and Interventions; (3) ConteXtualization and Structural Competence; and (4) Investigation of Systematic Differences ([Figure 1]).

1. Participation and Representation: Marginalized groups must participate meaningfully in research that affects them [[7]]. Meaningful participation increases the likelihood that research will benefit marginalized communities through the research process, such as through skills development and increased resources for community-based organizations. Equitable participation in research projects requires decision-making transparency and researcher reflection [[8]]. Representation is needed since we cannot develop appropriate interventions for marginalized groups without data. We should also elevate the voices and perspectives of marginalized participants since their knowledge is crucial for equity initiatives. Moreover, ethical considerations involve portraying marginalized people as they are comfortable. For example, technology researchers focusing on older adults and rural communities have advocated for “assets-based” versus solely “deficit-based” analyses [[9], [10]].

2. Appropriate Methods and Interventions: If not developed with marginalized groups, primary research methods and novel informatics interventions may fail to reach or alienate marginalized people. For example, surveys that require people to disclose their gender but offer only “male” and “female” as options often alienate transgender and non-binary people [[11]]. However, a recent systematic review of informatics interventions for lesbian, gay, bisexual, transgender, intersex, or queer (LGBTIQ+) youth showed the value of finding appropriate population-level targeting. Too much or too little can lead to interventions that feel superficial, stereotypical, or patronizing [[12]].

3. ConteXtualization and Structural Competence: Equity-oriented health informatics research requires attention to context; this means settings in which people are “born, live, learn, work, play, worship, and age” [[13]]. Borrowing “structural competence” terminology [[14]], we assert that informaticists must avoid individualizing systematic disparities through contextualization. For example, rather than approaching cardiovascular disease disparities among African-Americans as a product of individual “poor choices”, we should trace inequities to their roots in social determinants of health shaped by racism. Residents of segregated African-American neighborhoods often live in food deserts with poor walkability and limited green space [[15]-[18]], and racial discrimination increases cardiovascular risk [[19]]. Failure to acknowledge racism within cardiovascular research will thus distort understanding, alienate affected communities, and may lead to less effective interventions. Interventionists should target social conditions that (re-)produce inequities in meso-level or multi-level interventions [[20]].

4. Investigation of Systematic Differences: Experiences of people marginalized according to specific traits differ systematically from those not so marginalized. For instance, African-American, Native-American, Hispanic/Latino, and Native Hawaiian/Pacific Islander Americans receive lower-quality healthcare than non-Hispanic Whites across many quality indicators [[21]]. Thus, researchers should embed the potential for racial differences in any study of healthcare quality improvement. Furthermore, researchers can learn from intersectionality theory [[22]], which encourages investigating the specificity of experiences of multiply marginalized people, such as Indigenous women. Finally, health informatics risks intervention-generated inequalities (IGI) [[5]]. For instance, the racial “adjustment” that alters diagnostic algorithms or clinical practice guidelines based on race often directs more healthcare resources to Whites [[23]]. Health informatics researchers should investigate and prevent IGI to avoid harm.

Table 1 Example of health equity informatics studies.

Fig. 1 Equitable research PRAXIS framework.

Next, we detail PRAXIS considerations across four research phases, from project initiation to data analysis. We also outline operationalizing these considerations across different phases of nine health equity-focused informatics studies ([Table 1] and [Online Appendix A].

2 Initiating Projects

2.1 Participation and Representation

Informaticists should seek diverse representation “at the table” when a study starts. In the Dialysafe study, we ensured the involvement of women hemodialysis patients on study committees since more men initially volunteered. We asked partner organizations to identify women candidates; this achieved better gender balance. Those represented also need equitable opportunities to participate and power to influence project directions to help prioritize research important to marginalized groups. We now briefly outline three leading participatory research approaches to illustrate how we can pursue participation and representation from the outset of studies. See [Online Appendix B] for further comparison of these approaches.

Community-based Participatory Research (CBPR) originated in public health. It has “community” at its core unit for addressing inequities and involves members at all study stages [[8]]. CBPR requires that researchers: (i) build and sustain trusting relationships; (ii) develop access strategies; (iii) involve community members beyond a token role and compensate them fairly; (iv) establish realistic expectations; and (v) co-develop sustainable next steps.

The Shared Mobility study demonstrates the CBPR approach. Discussions with a Detroit Federally Qualified Health Center (FQHC)’s staff revealed that the high rate of no-show appointments was a significant concern, as were shortcomings of previous FQHC initiatives to provide healthcare transportation. Based on this dialogue and pilot data [[24]], we are developing and pilot-testing technology-enabled ride-sharing for FQHC patients’ healthcare transportation. In line with the FQHC’s priorities, no-show appointments were an evaluation outcome for the intervention pilot. For a partnering Detroit Timebank, transportation challenges were salient since rides were part of services already exchanged, and members explained that transportation was a major unmet need. These early discussions allowed us as researchers to align our interests in informatics interventions for healthcare access with their priorities

Transformative Mixed Methods (TMM) originated in education and focuses on values [[25]]. TMM prioritizes social justice and challenges marginalization—a suitable foundation for health equity research. TMM recognizes the importance of building capacity for communities with limited voice. Sequential mixed methods align with the transformative approach, and the mixed design and successive stages provide time to develop relationships with populations without established communities [[25]].

The Digital Technologies for Chronic Obstructive Pulmonary Disease (DT-COPD) study used TMM to elevate the voices of people with COPD [[17], [107]]. The research question targeted social isolation among COPD patients. Capacity building through sequential design [[26]] provided a way to target isolation. We met with support groups that helped build trust with COPD patients and, during meetings, gave a presentation on how to get involved in patient-oriented research. Regular check-ins at each stage built strong relationships.

Action Research has roots in education and anthropology and focuses on a “look-think-act” inquiry process to develop practical strategies to address social issues [[108]]. Action researchers facilitate stakeholders through an iterative process, which involves collecting data, reflecting on the knowledge shared, and theorizing to understand the study issue [[108]]. Action research may involve business, government, and professions [[108]], accommodating intersectoral collaboration.

The Gender, Sex and Sexual Orientation (GSSO) project was an action research project [[27]] focused on healthcare inequities for Canadian sexual and gender minorities. A pan-Canadian organization, Canada Health Infoway, hosted monthly working group meetings with community members, policymakers, industry, practitioners, and researchers. The research team facilitated monthly consultation sessions and completed a series of reviews that revealed issues and possible strategies to move towards inclusive digital health systems [[27]]. Subsequently, the team created a theoretically-guided action plan [[28]].

2.2 Appropriate Methods and Interventions

When initiating research, we must target data collection methods, instruments, and interventions to marginalized groups. For interventional studies, this includes considering communities’ technological contexts. For example, the BPMED intervention in Detroit, which served as the foundation for the mHealth to Improve Blood Pressure (MI-BP) trial, provided text messages for hypertension self-management [[29]]. We learned that government-issued cellular phones blocked text messages using short code services (shortened 5- or 6-digit phone numbers) as “premium” services at the project’s start. Owning a cellphone and having continuous functional service were separate issues. MI-BP pivoted to an App-based approach to avoid carrier-specific text message issues.

2.3 Contextualization and Structural Competence

Contextualization and structural competence is vital in framing research questions. For example, scholars have drawn attention to the folly of studying racial disparities by measuring individual race [[30]]. Countering this, informatics researchers have investigated stigmatizing language about African-American patients in clinical notes [[31], [32]] and provided feedback to healthcare providers about biases in patient interactions [[33]]. Structural competence aids in selecting the “level” of society at which interventions should operate [[20]]. In Dialysafe, structural competence involves comparing the effectiveness of meso-level informatics interventions focused on changing the practice patterns of hemodialysis facility staff [[34]] to a patient-facing informatics intervention for individual behavior change [[35]].

2.4 Investigating Systematic Differences

Investigating systematic differences should occur in studies with broad questions alongside subgroup analyses (e.g., [[36]]). We also need comparative studies to “detect, understand, and/or reduce” disparities [[1]]. Relatedly, the Telehealth Uptake study uses electronic health record (EHR) data to detect and explain potential disparities by assessing associations between telehealth visit modality (phone/video), patient demographics (e.g., race/ethnicity, age), and patient residence (e.g., broadband Internet access).

3 Sampling and Recruitment

3.1 Participation and Representation

Non-academic collaborators can: provide expert advice regarding how to reach participants; circulate study information through their communication vehicles; allow in-person recruitment to occur at events; and provide direct referrals. Such approaches may favor diversity in study samples [[37]]. For Project Simplify, we piloted an intervention to support telehealth video visits [[38]]. Based on advice from our FQHC partner, we first attempted to reach potential recruits by phone and made calls through a clinic phone number. As desired, with this approach, people who participated in the intervention had less internet experience and formal education than other FQHC patients [[38]].

Study inclusion and exclusion criteria may undermine representation. An informatics study that excluded potential study participants due to a lack of technology also excluded those of lower socioeconomic status (SES) and with more comorbidities [[39]]. An early health human-computer interaction (HCI) study required that participants be regular phone users, resulting in an advantaged sample [[40]]. We addressed this in our MI-BP trial, with eligibility criteria requiring only compatible smartphone ownership and not ongoing cellular service.

Representation is a challenge in research that uses secondary data sets [[41]]. In the Neighborhood Effects study, which uses social media data for population health research, we improved representation by oversampling based on Twitter users’ geographic location, targeting marginalized census tracts.

Researchers need to select appropriate methods and interventions when recruiting marginalized populations. One barrier can be online recruitment methods. For example, recruiting participants via patient portals may bias a sample since portal uptake is differential [[42]]. It is necessary to meet marginalized people “where they are.” We recruited from emergency departments (EDs) in our BPMED [[29]], and MI-BP [[43]] studies since the target population of African-American people with uncontrolled hypertension often sought care at EDs for both urgent and non-urgent issues.

3.2 Contextualization and Structural Competence

Contextualization and structural competence can aid study recruitment. When conducting surveys or interventions, sampling can proceed at contextual and individual levels, as in the Dialysafe cluster randomized controlled trial. Further, social network-based sampling can recruit marginalized participants due to social network homophily [[44]]— the tendency of people to interact with others similar to themselves. Respondent-driven sampling asks members of marginalized groups to recruit one another [[45]]. However, homophily can undermine diversity when researchers recruit from their social networks (e.g., [[46]]). In some studies, researchers use chain referral sampling to study social networks [[47]]. In the Families study [[48], [49]], we used chain referral sampling to recruit marginalized family units where an index person with HIV/AIDS or diabetes invited their family members into the study. Researchers may choose venue-based sampling [[50]] when locations are associated with health.

3.3 Investigating Systematic Differences

Sample sizes should allow for sufficient statistical power to facilitate subgroup analyses, including powering for planned moderation and/or stratified analyses as applicable. Oversampling of marginalized groups may also be justified in population health research, as we did in the Neighborhood Effects Study, when we oversampled health behavior-related tweets from census tracts with high neighborhood disadvantage [[51]]. Qualitative studies should plan for sufficient sample sizes to reach data saturation among subgroups within a heterogeneous sample. Furthermore, as described elsewhere, methods such as maximum variation sampling and quota sampling may help to ensure diverse samples [[5]]. We also encourage embedding evaluation of the impact of recruitment and retention methods on study sample representativeness within studies [[3]]. For example, researchers have compared the impact of recruitment venues on the demographics of study samples [[37]].

4 Data Collection

4.1 Participation and Representation

Participatory methods, such as participatory design (PD), elicit perspectives, experiences, and design requirements. PD methods include design workshops [[52]], focus groups, prototyping, and arts-based methods in which participants generate artifacts as data. Arts-based methods can elevate marginalized voices and communicate about contexts [[53]]. Examples include photo-voice [[54]], photo-elicitation [[55]], drawing, information world and journey mapping [[56], [57]], production of design artifacts [[58]], and digital storytelling [[59]]. Generated visual data like drawings may prompt discussion on sensitive topics [[59]] and flatten hierarchies between participants and researchers [[59]]. In Dialysafe, hemodialysis patients provided drawings responding to prompts about a desired intervention’s characteristics. We conducted thematic analyses of participants’ drawings to propose design recommendations [[35]]. We can also prioritize the voices of marginalized groups by treating their online content as study data. For example, a study used gender transition blogs to generate disclosure timelines by audiences, identifying the typical order in which transgender people disclosed their identities [[60]].

For representation, equity considerations arise when using secondary data sources like EHRs for research. There can be data completeness challenges with EHR data—perhaps due to more fragmented care receipt due to frequent moving or health insurance gaps among marginalized groups. In one study, children of low socioeconomic status had more missing data concerning asthma severity and diagnosis than those of higher SES [[61]]. Furthermore, an essential function of US EHRs is managing billing. This may impede equity-oriented analyses. In the Telehealth Uptake study, we attempted to analyze associations between patient health insurance type and healthcare visit modality as a proxy for SES. However, insurance coverage data were present at the encounter level instead of the patient level. Thus, it was initially only possible to track insurance coverage associated with billing, which led to large amounts of missing data.

4.2 Appropriate Methods and Interventions

Researchers must ensure that data collection instruments have appropriate language, readability, and self-description opportunities. The use of appropriate methods also involves treating marginalization as appropriately complex. For instance, sexual orientation is a multidimensional construct consisting of identity, behavior, and attraction. [Online Appendix C] summarizes measurements for marginalized groups to assist readers in their selection.

PD methods may assist in designing or adapting informatics interventions to marginalized groups. Researchers have extended PD methods for marginalized groups, intending to make the resulting technologies more appropriate for them as well. For example, researchers have extended PD methods to children using age-appropriate elicitation [[62]] and reflection methods [[63], [64]].

We conducted PD sessions remotely in the Shared Mobility study due to the COVID-19 pandemic. However, we facilitated participation via telephone due to technology access issues in Detroit. We created graphic novel-like “activity packets” to share common scenarios and generate ideas for group discussion [[65]]. Completed activity packets were both data and an “agenda” for the sessions.

4.3 Contextualization and Structural Competence

Study measures should capture person-person and person-environment interactions. Social network analysis conveys patterns of interaction and resource access (“social capit al”). For network data, we may gather social media interactions [[66]], email or instant message communication [[67]], co-location data [[68]], and surveys [[69]]. “Go-alongs” and “home-tours” are also relevant to equity-oriented research as they involve researchers following participants as they move through their daily lives. As [Online Appendix C] shows, health equity studies focusing on disabled people increasingly focus on the mismatch between environments and a person’s capacities. Researchers have collected data using wearable motion sensors to study real-time interactions between function and environment [[70]]. Crowd-sourced data on the accessibility of establishments, or virtual audits conducted with Google Street View, can also provide metrics on environmental accessibility [[71]]. Another critical aspect of context is history; the life-course perspective highlights individuals’ exposures to multiple contexts over time, which shape risks and resilience [[72]].

We have extended data collection methods to gather contextual information, often through interconnected samples. In the Families study, we borrowed the “whole family methodology” [[73]] for individual interviews in which we investigated the perspectives of each interviewee about the family unit, and we extended the focus group method to include family members (“family group interviewing”) [[74]]. Thus, we could observe family behaviors and their underlying dynamics [[75]-[77]].

There is also value in extending technology evaluation to naturalistic settings that mimic “real life” use constraints. In Dialysafe, we conducted a usability evaluation of a novel tablet-based intervention for hemodialysis patients among those who were dialyzing during the test [[78]]. Results revealed limitations in patient movement and device positioning when dialyzing to consider when re-designing the intervention. In Project Simplify, we held remote user tests in which FQHC patients used videoconferencing technologies at home. Although we provided patients with all necessary technology, they faced challenges they might encounter in an actual telehealth visit, such as: accepting phone settings, internet connectivity issues, and interruptions.

4.4 Investigating Systematic Differences

We must know who is in our study samples; thus, we must gather information about marginalized participants ([Online Appendix C]). Secondary data must be sufficiently granular for analyses, which may be challenging for small groups. For instance, in the United States, the PCORNet common data model classifies American Sign Language as “other” language, making it hard to identify culturally Deaf patients in a database of 80 million patients [[79]]. For interventional studies, we advocate using CONSORT’s equity reporting guidelines for clinical trials [[80]].

Furthermore, we advocate gathering data to advance understanding of inequities. Patient addresses can be geocoded and linked to data describing census tracts or zip codes. Many publicly-available US data sources offer georeferenced contextual data (e.g., [[81]]). Healthcare screening for patient social risks may also identify factors driving outcomes, like food insecurity [[82]]. However, we must collect data systematically to be useful. In the Telehealth Uptake study, we could not include social risk information from the EHR in analyses due to data sparsity. Another approach is to conduct parallel studies. In the Shared Mobility study, we collect data in tandem in urban Detroit and rural Indiana to compare technology design requirements in rural and urban settings [[82]].

5 Data Analysis

5.1 Participation and Representation

Participation can help interpretations to remain faithful to marginalized groups’ perspectives. We have involved marginalized people in developing qualitative codebooks and coding [[7]]. Community members can also interpret and label unstructured data to capture their meaning [[84]]. Extending qualitative member checking, we often present preliminary results for partner feedback, and community partners are co-authors of many manuscripts.

For representation, we recommend analyses that estimate the effects of missing groups on estimates of parameters from samples. A goal of the Neighborhood Effects study is to estimate demographic and selection bias in Twitter posts concerning health behavior. To investigate such biases, we survey Twitter users who have tweeted about health behaviors and compare social media-based to survey-based estimates of these behaviors.

5.2 Appropriate Methods and Interventions

Gathering feedback can enhance appropriateness. In the Shared Mobility study, we held three PD sessions with community partners to gather feedback on intervention design recommendations from a first round of PD sessions with potential riders and drivers. We incorporated this feedback into low-fidelity prototypes for second-round PD sessions.

Appropriate methods also require researchers to address potential limitations based on our perspectives. Establishing explicit reflexivity methods can assist researchers in maintaining a focus on marginalized voices [[85]]; this is critical for arts-based data since visuals elicit multiple worldviews [[86]]. Strategies to center marginalized voices in analyses include continuously revisiting data [[87], [88]] or regular analytic team discussions [[87]]. In Project Simplify, we used a form with questions to prompt research assistants to write daily reflections throughout an intervention, which we analyzed thematically [[38]]. Other approaches include using templates to prompt researchers’ assumptions [[87]].

5.3 Contextualization and Structural Competence

Quantitative approaches include multi-level and longitudinal statistical models, spatial analysis, and social network analysis. Multi-level models account for the clustering of observations within contextual data units (e.g., census tracts) that violate the assumption of independence in statistical models [[89]]. Multi-level models partition the variance in an outcome into that which is both within and between contexts.

Longitudinal statistical analyses also aid in contextualizing disparities. Cohort effects may be evident in matches between biographical time and socio-historical events, revealing disparity dynamics. For instance, rapid changes in social acceptance in Western countries may have produced generational differences in the prevalence of hardships like family rejection among bisexual men [[90]]. We can use growth curve models to examine life course health trajectories and incorporate spline terms to capture critical life stages when using age as the time indicator [[91]].

When operationalizing context spatially, we may use mobile technologies to gather mobility data and then use analytical methods to identify exposures, as in “activity space” analysis [[92]]. We can use spatial clustering methods such as hotspot analyses to identify areas with high concentrations of adverse health outcomes or risks [[93]]. Furthermore, analytical methods such as geographically-weighted regressions can explore how relationships between predictors and outcomes vary in space [[94]].

We may also use social network analysis methods to illuminate access to equity-relevant social capital [[95]] and related outcomes. For example, in health-harming natural disasters, people with higher social capital recover faster [[96]]. People with higher bridging capital are more likely to evacuate [[97]].

Qualitative contextual approaches include multi-sited ethnography and field observation in homes, doctor’s offices, and clinics [[98], [99]]. Researchers have examined how technologies within clinical consultations may convey clinical authority [[100], [101]], reduce clinicians’ visual focus on patients, and reduce questioning about psychosocial issues [[101]] important for equity. In the Families study, we used home tours to study illness self-management technologies by taking photos of objects in the home. We then analyzed the material content and spatial arrangements in photographs [[76]].

5.4 Investigating Systematic Differences

We can investigate systematic differences in varied study designs, including informatics trials employing moderation or stratified analyses. For example, in Dialysafe, analyses involve testing for moderation effects based on patient sex, race, and health literacy. We should also investigate biases in machine learning algorithms. Biases may emerge because supervised machine learning methods rely on human-generated annotations or human activities; thus, human biases can emerge in training data (e.g., [[102]]). Training data may also have insufficient representation of certain groups (e.g., [[103]]). Measurable variables that serve as proxies for harder-to-measure variables can also introduce biases (e.g., [[104]]). Methods to evaluate and mitigate biases seek fairness [[105]], which we may define as individual fairness, such that for a specific task, the model classifies any two individuals similarly. Alternatively, group fairness requires similar classification for protected and unprotected groups [[106]].

In the Neighborhood Effects study, we developed a machine learning model for a binary classification task on textual, geotagged data mined from Twitter to identify health behavior discussion [[51]]. We evaluated model performance (accuracy, precision, recall, and F1 score) within demographic groups based on the American Community Survey census tract information. We identified representativeness issues; thus, we collected and labeled additional tweets from tracts with more marginalized residents to re-train the model.

6 Conclusion

We have contributed the PRAXIS framework to guide informaticists in systematically considering equity issues in research. The PRAXIS framework highlights four considerations: (1) Participation and Representation; (2) Appropriate Methods and Interventions; (3) ConteXtualization and Structural Competence; and (4) Investigation of Systematic Differences. Drawing from nine informatics studies as examples, we outlined guidance for operationalizing these considerations at four research stages: (i) project initiation; (ii) sampling and recruitment; (iii) data collection; and (iv) data analysis. Although not this paper’s focus, we also stress the importance of dissemination. In the example projects, participatory dissemination strategies were often part of the study design. We hope that this framework will assist informatics researchers in systematically addressing equity considerations and that this article will be a resource for training new researchers. We encourage other researchers to use these insights to advance health equity in informatics.

No conflict of interest has been declared by the author(s).

Acknowledgements

The studies described in this paper were funded by: Agency for Healthcare Research and Quality (1R21HS019092-01), Canadian Institutes for Health Research (MOP 130248, Planning Grant), Google Health, Health Endowment Fund (R-2001-145243), Institute of Museum and Library Services (Grant #RE-04-10-0038-10), Michael Smith Foundation for Health Research (RA-2020-1383) National Institutes of Health (1R01HL127215-01A1), National Science Foundation (#1910281 and #2031662), Patient-Centered Outcomes Research Institute (IHS-1503-278481), the BC Support for People & Patient-Oriented Research & Trials Unit, and the Sloan & Moore Foundations. No conflict of interest has been declared by the author(s).

• References

• 1 Veinot TC, Ancker JS, Bakken S. Health informatics and health equity: Improving our reach and impact. J Am Med Inform Assoc 2019 Aug 1;26(8-9):689-95
• 2 Johnson KB, Bright TJ, Clark CR. Overview of the issue. J Health Care Poor Underserved 2021;32(2):1-4
• 3 Siek K, Veinot TC, Mynatt E. Research opportunities in sociotechnical interventions for health disparity reduction Washington, DC: CRA; 2019 Available from: https://arxiv.org/abs/1908.01035
• 4 Gibbons MC, Wilson RF, Samal L, Lehmann CU, Dickersin K, Lehmann HP, et al. Consumer health informatics: Results of a systematic evidence review and evidence based recommendations. Transl Behav Med 2011 Mar;1(1):72-82
• 5 Veinot TC, Mitchell H, Ancker JS. Good intentions are not enough: How informatics interventions can worsen inequality. J Am Med Inform Assoc 2018 Aug 1;25(8):1080-8.
• 6 Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 2019 Oct 25;366(6464):447-53
• 7 Unertl KM, Schaefbauer CL, Campbell TR, Senteio C, Siek KA, Bakken S, et al. Integrating community-based participatory research and informatics approaches to improve the engagement and health of underserved populations. J Am Med Inform Assoc 2016 Jan;23(1):60-73.
• 8 Liang CA, Munson SA, Kientz JA. Embracing four tensions in human-computer interaction research with marginalized people. ACM Transactions on Computer-Human Interaction (TOCHI) 2021;28(2):Article 14
• 9 Hardy J, Wyche S, Veinot TC. Rural HCI research: Definitions, distinctions, methods, and opportunities. Vancouver, BC: CSCW Conf Comput Support Coop Work. 2019
• 10 Vines J, Pritchard G, Wright P, Olivier P, Brittain K. An age-old problem: Examining the discourses of ageing in HCI and strategies for future research. ACM Trans Comput Hum Interact 2015;22(1):1-27
• 11 Spiel K, Haimson OL, Lottridge D. How to do better with gender on surveys. Interactions 2019;26(4):62-5.
• 12 Gilbey D, Morgan H, Lin A, Perry Y. Effectiveness, acceptability, and feasibility of digital health interventions for LGBTIQ+ young people: Systematic review. J Med Internet Res 2020 Dec 3;22(12):e20158.
• 13 US Department of Health and Human Services. Healthy People 2030: Social determinants of health Washington: DHHS; 2020. Available from: https://health.gov/healthypeople/priority-areas/social-determinants-health
• 14 Metzl JM, Hansen H. Structural competency: theorizing a new medical engagement with stigma and inequality. Soc Sci Med 2014 Feb;103:126-33
• 15 Larson NI, Story MT, Nelson MC. Neighborhood environments: disparities in access to healthy foods in the U.S. Am J Prev Med 2009 Jan;36(1):74-81.
• 16 Diez Roux AV, Merkin SS, Arnett D, Chambless L, Massing M, Nieto FJ, et al. Neighborhood of residence and incidence of coronary heart disease. N Engl J Med 2001 Jul 12;345(2):99-106
• 17 Conderino SE, Feldman JM, Spoer B, Gourevitch MN, Thorpe LE. Social and economic differences in neighborhood walkability across 500 U.S. cities. Am J Prev Med 2021 Sep;61(3):394-401
• 18 Kephart L. How racial residential segregation structures access and exposure to greenness and green space: A review. Environ Justice 2021.
• 19 Lewis TT, Cogburn CD, Williams DR. Self-reported experiences of discrimination and health: Scientific advances, ongoing controversies, and emerging issues. Annu Rev Clin Psychol 2015;11:407-40
• 20 Veinot TC, Ancker JS, Cole-Lewis H, Mynatt ED, Parker AG, Siek KA, et al. Leveling up: On the potential of upstream health informatics interventions to enhance health equity. Med Care 2019 Jun;57 Suppl 6 Suppl 2:S108-S114
• 21 Agency for Healthcare Research and Quality. 2019 National healthcare quality and disparities report. Rockville, MD: AHRQ; 2021
• 22 Bauer GR, Churchill SM, Mahendran M, Walwyn C, Lizotte D, Villa-Rueda AA. Intersectionality in quantitative research: A systematic review of its emergence and applications of theory and methods. SSM Popul Health 2021 Apr 16;14:100798
• 23 Vyas DA, Eisenstein LG, Jones DS. Hidden in plain sight - Reconsidering the use of race correction in clinical algorithms. N Engl J Med 2020 Aug 27;383(9):874-82.
• 24 Dillahunt TR, Veinot TC. Getting there: Barriers and facilitators to transportation access in underserved communities. ACM Trans Comput Hum Interact 2018;25(5):1-39
• 25 Mertens DM. Transformative paradigm: Mixed methods and social justice. J Mixed Methods Res 2007;1(3):212-25
• 26 Mertens DM. Transformative mixed methods: Addressing inequities. Am Behav Sci 2012;56(6):802-13
• 27 Antonio M, Lau F, Davison K, Devor A, Queen R, Courtney K. Toward an inclusive digital health system for sexual and gender minorities in Canada. J Am Med Inform Assoc 2022 Jan 12;29(2):379-84
• 28 Lau F, Devor A, Antonio M, et al. A proposed action plan to modernize gender, sex and sexual orientation information practices in Canadian electronic health record systems; 2021. Available from: https://infocentral.infoway-inforoute. ca/en/resources/docs/sex-gender/sex-gender-action-plan/3496-gsso-action-plan-full-document
• 29 Buis L, Hirzel L, Dawood RM, Dawood KL, Nichols LP, Artinian NT, et al. Text messaging to improve hypertension medication adherence in African Americans from primary care and emergency department settings: Results from two randomized feasibility studies. JMIR Mhealth Uhealth 2017 Feb 1;5(2):e9
• 30 Hardeman RR, Homan PA, Chantarat T, Davis BA, Brown TH. Improving the measurement of structural racism to achieve antiracist health policy. Health Aff (Millwood) 2022 Feb;41(2):179-86.
• 31 Sun M, Oliwa T, Peek ME, Tung EL. Negative patient descriptors: documenting racial bias In The electronic health record. Health Aff (Millwood) 2022 Feb;41(2):203-11
• 32 Himmelstein G, Bates D, Zhou L. Examination of stigmatizing language in the electronic health record. JAMA Netw Open 2022 Jan 4;5(1):e2144967
• 33 Dirks LG, Beneteau E, Sabin J, Pratt W, Lane C, Bascom E, et al. Battling bias in primary care encounters: Informatics designs to support clinicians. Ext Abstr Hum Factors Computing Syst 2022 Apr;2022:386
• 34 Kuo PY, Saran R, Argentina M, Heung M, Bragg-Gresham JL, Chatoth D, et al. Development of a checklist for the prevention of intradialytic hypotension in hemodialysis care: Design considerations based on activity theory. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems; 2019. p. 1-14
• 35 Willis MA, Hein LB, Hu Z, Saran R, Argentina M, Bragg-Gresham J, et al. Feeling better on hemodialysis: User-centered design requirements for promoting patient involvement in the prevention of treatment complications. J Am Med Inform Assoc 2021 Jul 30;28(8):1612-31
• 36 Iott BE, Loveluck J, Benton A, Golson L, Kahle E, Lam J, et al. The impact of stigma on HIV testing decisions for gay, bisexual, queer and other men who have sex with men: A qualitative study. BMC Public Health 2022 Mar 9;22(1):471
• 37 Iott BE, Veinot TC, Loveluck J, Kahle E, Golson L, Benton A. Comparative analysis of recruit­ment strategies in a study of men who have sex with men (MSM) in metropolitan Detroit. AIDS Behav 2018 Jul;22(7):2296-311
• 38 Antonio, MG, Williamson, AK, Kameswaran, V, Ankrah, E., Goulet, S., Wang, I, et al. Reducing patients’ cognitive load for telehealth video visits through student-delivered helping sessions at a United States Federally Qualified Health Center: A pilot intervention study. JMIR [submitted 12 Sep 2022]
• 39 Toscos T, Drouin M, Pater J, Flanagan M, Pfafman R, Mirro MJ. Selection biases in technology-based intervention research: Patients’ technology use relates to both demographic and health-related inequities. J Am Med Inform Assoc 2019 Aug 1;26(8-9):835-9
• 40 Consolvo S, McDonald DW, Toscos T, Chen MY, Froehlich J, Harrison B, et al. Activity sensing in the wild: A field trial of ubifit garden. In: Proceedings of the 2008 SIGCHI conference on human factors in computing systems; 2008. p. 1797-806
• 41 Puri R. Mitigating bias in AI models. IBM Research Blog. 2018. Available from: https://www.ibm.com/blogs/research/2018/02/mitigating-bias-ai-models/
• 42 Grossman LV, Masterson Creber RM, Benda NC, Wright D, Vawdrey DK, Ancker JS. Interventions to increase patient portal use in vulnerable populations: a systematic review. J Am Med Inform Assoc 2019 Aug 1;26(8-9):855-70
• 43 Buis LR, Dawood K, Kadri R, Dawood R, Richardson CR, Djuric Z, et al. Improving blood pressure among african americans with hypertension using a mobile health approach (the MI-BP app): Protocol for a randomized controlled trial. JMIR Res Protoc 2019 Jan 25;8(1):e12601
• 44 McPherson M, Smith-Lovin L, Cook JM. Birds of a feather: Homophily in social networks. Ann Rev Sociol 2001;27(1):415-44
• 45 Heckathorn DD. Respondent-driven sampling: a new approach to the study of hidden populations. Soc Probl 1997;44(2):174-99
• 46 Lin JJ, Mamykina L, Lindtner S, Delajoux G, Strub HB. Fish’n’Steps: Encouraging physical activity with an interactive computer game. In: International conference on ubiquitous computing 2006. Berlin, Heidelberg: Springer; 2006. p. 261-78.
• 47 Veinot TC. Regional HIV/AIDS information environments and information acquisition success. The Information Society 2013;29(2):88-112
• 48 Veinot TC, Kim YM, Meadowbrooke CC. Health information behavior in families: Supportive or irritating? Proceedings of the American Society for Information Science and Technology 2011;48(1):1-10
• 49 Brown LK, Veinot TC. Information behavior and social control: Toward an understanding of conflictual information behavior in families managing chronic illness. J Assoc Inf Sci Technol 2021;72(1):66-82
• 50 Muhib FB, Lin LS, Stueve A, Miller RL, Ford WL, Johnson WD, et al; Community interven­tion trial for youth study team. A venue-based method for sampling hard-to-reach populations. Public Health Rep 2001;116 Suppl 1(Suppl 1):216-22
• 51 Vydiswaran VGV, Romero DM, Zhao X, Yu D, Gomez-Lopez I, Lu JX, et al Uncovering the relationship between food-related discussion on Twitter and neighborhood characteristics. J Am Med Inform Assoc 2020 Feb 1;27(2):254-64
• 52 Harrington CN, Borgos-Rodriguez K, Piper AM. Engaging low-income African American older adults in health discussions through community-based design workshops. In: Proceedings of the 2019 chi conference on human factors in computing systems; 2019. p.1-15
• 53 Majid U, Kandasamy S, Ramlakhan J. How to design an arts-based health services research study: A participatory qualitative study on the determinants of telehealth adoption. Thousand Oaks, CA: SAGE; 2020
• 54 Sutton-Brown CA. Photovoice: A Methodological Guide. Photography & Culture 2015;7(2):169-85
• 55 Bugos E, Frasso R, FitzGerald E, True G, Adachi-Mejia AM, Cannuscio C. Practical guidance and ethical considerations for studies using photo-elicitation interviews. Prev Chronic Dis 2014 Oct 30;11:E189
• 56 Greyson D, O’Brien H, Shoveller J. Information world mapping: A participatory arts-based elicitation method for information behavior interviews. Libr Inf Sci Res 2017;39(2):149-57
• 57 Howard T. Journey mapping: A brief overview. Communication Design Quarterly Review 2014;2(3):10-3
• 58 Demirbilek O, Demirkan H. Universal product design involving elderly users: A participatory design model. Appl Ergon 2004;35(4):361-70
• 59 Khoshkesht S, Nikbakht Nasrabadi A, Mardanian Dehkordi L. Digital storytelling: The new arts-based research method. Iran J Public Health 2020 Jul;49(7):1395-6
• 60 Haimson OL, Veinot TC. Coming out to doctors, Coming out to “everyone”: Understanding the average sequence of transgender identity dis­closures using aocial media data. Transgend Health 2020 Sep 2;5(3):158-65
• 61 Juhn YJ, Ryu E, Wi CI, King KS, Malik M, Romero-Brufau S, et al. Assessing socioeconomic bias in machine learning algorithms in health care: A case study of the HOUSES index. J Am Med Inform Assoc 2022 Jun 14;29(7):1142-51
• 62 Yip J, Clegg T, Bonsignore E, Gelderblom H, Rhodes E, Druin A. Brownies or bags-of-stuff? Domain expertise in cooperative inquiry with children. In: Proceedings of the 12th International Conference on Interaction Design and Children, 2013. New York: ACM; 2013. p. 201-10.
• 63 Guha ML, Druin A, Chipman G, Fails JA, Simms S, Farber A. Mixing ideas: A new technique for working with young children as design partners. In: Proceedings of the 2004 conference on Interaction design and children: building a community; 2004. p. 35-42
• 64 Walsh G, Druin A, Guha ML, Foss E, Golub E, Hatley L, et al. Layered elaboration: A new technique for co-design with children. In: Proceedings of the 2010 SIGCHI Conference on Human Factors in Computing Systems; 2010. p. 1237-40
• 65 Dillahunt TR, Maestre JF, Kameswaran V, Poon E, Osorio Torres J, Gallardo M, et al. Trust, reciprocity, and the role of timebanks as inter­mediaries: Design implications for addressing healthcare transportation barriers. In: 2022 CHI Conference on Human Factors in Computing Systems; 2022. p. 1-22
• 66 Lewis K, Kaufman J, Gonzalez M, Wimmer A, Christakis N. Tastes, ties, and time: A new social network dataset using Facebook.com. Soc Networks 2008;30(4):330-42
• 67 Romero DM, Uzzi B, Kleinberg J. Social networks under stress. In: Proceedings of the 25th International Conference on World Wide Web 2016. p.9-20
• 68 Lu S, Zhao J, Wang H. Academic failures and co-location social networks in campus. EPJ Data Sci 2022;11(1):10
• 69 Burt RS. Network items and the general social survey. Soc Networks 1984;6(4):293-339.
• 70 Duchowny K, Clarke P, Gallagher NA, Adams R, Rosso AL, Alexander NB. Using mobile, wearable, technology to understand the role of built environment demand for outdoor mobility. Environ Behav 2019; 51(6):671-88
• 71 Clarke P, Gallagher NA. Optimizing mobility in later life: The role of the urban built environment for older adults aging in place. J Urban Health 2013 Dec;90(6):997-1009.
• 72 Ferraro KF, Shippee TP. Aging and cumulative inequality: how does inequality get under the skin? Gerontologist 2009 Jun;49(3):333-43.
• 73 Handel G. Family Worlds and Qualitative Family Research. Marriage Fam Rev 1997;24(3-4):335-48.
• 74 Eggenberger SK, Nelms TP. Family interviews as a method for family research. J Adv Nurs 2007 May;58(3):282-92
• 75 Barbarin AM, Klasnja P, Veinot TC. Good or bad, ups and downs, and getting better: Use of personal health data for temporal reflection in chronic illness. Int J Med Inform 2016 Oct;94:237-45
• 76 Barbarin A, Veinot TC, Klasnja P. Taking our time: chronic illness and time-based objects in families. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing 2015. p. 288-301
• 77 Wolf CT, Veinot TC. Struggling for space and finding my place: An interactionist perspective on everyday use of biomedical information. J Assoc Inf Sci Technol 2015;66(2):282-96
• 78 Willis M, Brand Hein L, Hu Z, Saran R, Argentina M, et al. IUsability evaluation of a tablet-based intervention to prevent intradialytic hypotension in dialysis patients during in-clinic dialysis: Mixed methods study. JMIR Hum Factors 2021 Jun 14;8(2):e26012.
• 79 James TG, Sullivan MK, Butler JD, McKee MM. Promoting health equity for deaf patients through the electronic health record. J Am Med Inform Assoc 2021 Dec 28;29(1):213-6.
• 80 Welch VA, Norheim OF, Jull J, Cookson R, Sommerfelt H, Tugwell P; CONSORT-Equity and Boston Equity Symposium. CONSORT-Equity 2017 extension and elaboration for better reporting of health equity in randomised trials. BMJ 2017 Nov 23;359:j5085
• 81 Clarke P. National Neighborhood Data Archive (NaNDA). Ann Arbor, MI: ICPSR; 2000-2020. Available from: https://www.openicpsr.org/openicpsr/nanda
• 82 Berkowitz SA, Karter AJ, Corbie-Smith G, Seligman HK, Ackroyd SA, Barnard LS, et al. Food insecurity, food “deserts,” and glycemic control in patients with diabetes: A longitudinal analysis. Diabetes Care 2018 Jun;41(6):1188-95
• 83 Maestre JF, Dillahunt T, Theisz A, Furness M, Kameswaran V, Veinot TC, et al. Examining mobility among people living with HIV in rural areas. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama. ACM; 2021. p. 1-17
• 84 Frey WR, Patton DU, Gaskell MB, McGregor K. Artificial intelligence and inclusion: Formerly gang-involved youth as domain experts for analyzing unstructured Twitter data. Soc Sci Comput Rev 2020;38(1):42-56
• 85 Corlett S, Mavin S. Reflexivity and researcher positionality. In: Cassell C, Cunliffe A, Grandy G. The SAGE handbook of qualitative business and management research methods. Thousand Oaks, CA: SAGE; 2018. p. 377-98
• 86 Wang C, Burris MA. Photovoice: Concept, methodology, and use for participatory needs assessment. Health Educ Behav 1997 Jun;24(3):369-87
• 87 Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res 1999 Jan;9(1):26-44.
• 88 Caretta MA, Pérez MA. When participants do not agree: Member checking and challenges to epistemic authority in participatory research. Field Methods 2019;31(4):359-74
• 89 Bryk AS, Raudenbush SW. Hierarchical linear models: Applications and data analysis. Newbury: Sage; 1992
• 90 McCormack M, Anderson E, Adams A. Cohort effect on the coming out experiences of bisex­ual men. Sociology 2014;48(6):1207-23
• 91 Clarke P, Wheaton B. Mapping social context on mental health trajectories through adulthood. Adv Life Course Res 2005;9:269-301
• 92 Goodspeed R, Yan X, Hardy J, Vydiswaran VGV, Berrocal VJ, Clarke P, et al. Comparing the data quality of global positioning system devices and mobile phones for assessing relationships between place, mobility, and health: Field study. JMIR Mhealth Uhealth 2018 Aug 13;6(8):e168.
• 93 Lee G, Choi B, Ahn CR, Lee S. Wearable biosensor and hotspot analysis–based framework to detect stress hotspots for advancing elderly’s mobility. Eng Manag J 2020;36(3):04020010
• 94 Arif Khan M, Shahmoradi A, Etminani-Ghasrodashti R, Kermanshachi S, Michael Rosenberger J. A Geographically weight regression approach to modeling the determinants of on-demand ride services for elderly and disabled. International Conference on Transportation and Development 2021. p.385-96
• 95 Burt RS. The network structure of social capital. Res Organ Behav 2000;22:345-423
• 96 Sadri AM, Ukkusuri SV, Lee S, Clawson R, Aldrich D, Nelson MS, et al. The role of social capital, personal networks, and emergency responders in post-disaster recovery and resilience. Nat Hazards (Dordr) 2018;90(3):1377-406
• 97 Metaxa-Kakavouli D, Maas P, Aldrich DP. How social ties influence hurricane evacuation behavior. Proc ACM Hum Comput Interact 2018;2(CSCW):1-16
• 98 Hartel J, Thomson L. Visual approaches and photography for the study of immediate information space. J Assoc Inf Sci Technol 2011;62(11):2214-24
• 99 Veinot TC, Pierce CS. Materiality in information environments: Objects, spaces, and bodies in three outpatient hemodialysis facilities. J Assoc Inf Sci Technol 2019;70(12):1324-39
• 100 Veinot TC, Zheng K, Lowery JC, Souden M, Keith R. Using electronic health record systems in diabetes care: emerging practices. In: Proceedings of the 1st ACM International Health Informatics Symposium 2010. p. 240-9
• 101 Shachak A, Reis S. The impact of electronic medical records on patient-doctor communication during consultation: A narrative literature review. J Eval Clin Pract 2009 Aug;15(4):641-9
• 102 Vasserman L, Li J, Adams CJ, Dixon L. Unintended bas and identity terms 2018. Available from: https://medium.com/jigsaw/unintended-bias-and-names-of-frequently-targeted-groups-8e0b81f80a23
• 103 Buolamwini J, Gebru T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In: 2018 Conference on fairness, accountability and transparency. PMLR. p. 77-91.
• 104 Chouldechova A, Benavides-Prado D, Fialko O, Vaithianathan R. A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In: 2018 Conference on fairness, accountability and transparency. PMLR. p. 134-48
• 105 Hovy D, Spruit SL. The social impact of natural language processing. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics 2016 (Volume 2: Short Papers). p. 591-8
• 106 Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R. Fairness through awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference 2012. Cambridge, MA: ACM. p.214-26
• 107 Antonio, M. Exploring the role of digital technologies for social connectedness, outcomes and experiences with the chronic obstructive pulmonary disease (COPD) community: A transformative mixed methods research study. Doctoral thesis], Victoria, BC: University of Victoria; 2021
• 108 Stringer ET. Action research: a handbook for practitioners. Thousand Oaks, CA: SAGE Publications; 1996.

Publication History

Article published online:
04 December 2022