3 Testimonies
Researchers from 11 countries accepted the invitation to contribute on this topic. Contributions are provided by country, listed in alphabetical order. Appendix 1 provides a structured overview of the organization of the IRB review process, the planning of the process, and the challenges faced country per country.
3.1 Argentina (I. Jauregui, D. Luna, C. Otero)
In Argentina, every study involving human participants is required to have an IRB approval prior to its initiation, and to deliver a follow-up documentation at least once a year until the study is completed. This process is regulated by a national law that although mainly focused on pharmacological, diagnostic, and therapeutic protocols, also includes HOF studies in healthcare because of their intrinsic nature of studying human beings and healthcare. This national law, which subscribes to the Declaration of Helsinki, delegates the surveillance of the protection of participants’ rights to the jurisdictional and institutional IRBs where the studies are carried out, by controlling and supervising the studies. Local ethical boards are autonomous: they decide about the approval of the studies, and they report to the jurisdictional (city) level only for statistical purposes.
IRBs have to be composed of multidisciplinary staff, evenly distributed across age, gender, and scientific and non-scientific members. The Hospital Italiano de Buenos Aires has an IRB that oversees all studies involving human subjects. It has a main procedure for approving protocols, in which investigators are required to present a study plan including a detailed methodology, clinical impact, data protection measures and funding sources, an informed consent of the participants to the study, a good clinical practices certificate, a letter of compliance with the Declaration of Helsinki, and the CV of the main investigator. The committee meets every 15 days, and in each session, it analyzes the new protocols and notifies the investigators of its decision, or asks for supplementary information or amendments for the protocol to be approved. This process takes around one to two months, and after approval investigators have to ill a form for the Health secretary that keeps track of all clinical protocols within its jurisdiction.
The IRB has also an expedited procedure for studies that investigate aspects related to the clinical management of the organization, or for studies in which no sensitive data (i.e. demographics, diagnosis, vital signs, procedures, etc.) are handled, no therapeutic or diagnostic instrument is used, and no possible harm is done to the patient. In this procedure, the statement from the committee is issued more rapidly. This procedure is sometimes used to get IRB approval for HOF studies, since no possible harm is done to participants, and findings help the organization to better engage with patients and staff information needs, and interaction with computer systems.
We believe that legislation about ethics in HOF and generally in Health Informatics studies is lacking in our country. This type of studies arises different risks for participants than pharmacological or diagnostic test studies, and they should be regulated and monitored in a more specific way.
3.2 Australia (M. Baysari, W. Y. Zheng)
As human factors researchers, we undertake studies that focus on understanding how work is done in practice and how information technology supports or hinders that work. For example, we have run multiple qualitative observational and interview studies in hospitals where we examined how computerized decision support influenced medication decision-making by doctors[6]
[7]
[8]
[9]. Although this research is non-interventional, to collect and publish data of any kind requires researchers to obtain ethics approval. In Australia, this is a two-step process. Initially, ethics approval must be obtained from a National Medical and Health Research Council (NHMRC)-approved Human Research Ethics Committee (HREC). These committees, which typically meet monthly, assess research to ensure it meets the requirements of the National Statement on Ethical Conduct in Human Research and is ethically acceptable[10]. Following HREC approval, site-specific governance approval must be obtained from each participating site (e.g. hospital). Site-specific assessment (SSA) allows the site to consider whether they have the capacity to conduct the research (e.g. physical resources, staff, insurance, and indemnity requirements). This can be particularly time-consuming when multi-site research is being done. If you are examining an IT system in three hospitals, you would need to obtain HREC approval once, but SSA approval three times. We see value in obtaining ethics approval to undertake research, including non-interventional HOF research, but believe there are some key problems with the process.
Our biggest concern is with the lack of standardization in the application and approval process across sites and jurisdictions. For example, additional forms are required for interstate researchers wishing to conduct research at sites in Queensland (Public Health Act) and Victoria (Victorian Specific Module).
Second, up until recently, the processes and application form required for ethics approval was dependent on the risk level of a project. Projects deemed to be at “low or negligible risk” were expedited through a streamlined process and the form required to be completed by researchers was shorter and simpler than the standard ethics form. Most HOF research comprises low risk research and so was processed through this expedited process. In 2019, Australia's national ethics form was revised and now a single application form is used for all projects, regardless of risk level.
Finally, in order to gain access to a site for research, researchers are required to go through a series of authorizations and checks including vaccinations, police checks, and employment checks. Unfortunately, once a researcher is approved to collect data at one site, this does not carry across to other sites, even if in the same Australian State. New forms and checks are required to be completed at each data collection site.
Overall, we do not dispute the value of ensuring HOF research is undertaken ethically, posing minimal risk and inconvenience to participants (in our case, users of technology). Currently, the ethics approval process is viewed by HOF researchers as a barrier to completing research and can act as a deterrent to conducting multi-site research within public and private health organizations. This is problematic, as much can be learned from conducting research at multiple sites (i.e. context-specific factors influencing IT uptake or success). We are confident that the availability of a low effort, streamlined and consistent process for applications which pose a minimal risk to users and organizations, would be welcomed by all Australian HOF researchers.
3.3 Canada (C. Kuziemsky)
In Canada, the Tri-Council Policy Statement (TCPS 2[1]) is the national policy for ethical research involving humans. The TCPS 2 provides definitions for types of research studies and provides guidance for more complicated studies such as multi-jurisdictional research. There are three general categories of review. First is full board review, which is used for clinical trials or other intervention studies. Second is minimal-risk review, which is done when the risk of adverse outcomes is no more than the risk from everyday life. Minimal risk is typically done by a delegated review where a selected set of review ethical board (REB) members review the file. Third is expedited review, done in cases when a project already has ethics approval from a TCPS 2-compliant Canadian university or hospital or if a project is using anonymized non-public secondary data. An example of this would be using anonymized data from a hospital. HOF studies involving human subjects where data is being collected to answer a research question would always require ethics approval. Studies involving patient recruitment for observation or evaluation such as a usability study would be classified as minimal risk review and typically have turnaround times of up to six weeks. Expedited review would take up to three weeks.
However, while general guidelines for ethics approval are quite clear, the actual process of review can vary significantly depending on several factors. One factor is the affiliation of the researcher and where the study is being conducted. If a HOF researcher is affiliation with a University but doing research at a healthcare entity such as a hospital, ethics approval will be required from both the healthcare center (e.g. hospital) and the academic institution. The usual protocol is to first obtain ethics approval from the hospital and then submit for REB approval to the University where an expedited review of the ethics approval is done. Universities may sign a formal agreement with certain hospitals to better facilitate this two-stage application process. However, this two-stage process can lead to problems. Disputes can occur where ethics approval and conditions of it (e.g. how to recruit patients or conditions of a consent form) were approved by a hospital board but the University board requests changes to the protocol that contradicts what the hospital approval has described. Another factor is the healthcare context where the study is taking place. While hospital ethics boards should be knowledgeable of HOF studies, situations can arise where a healthcare region may have an ethics board that is not familiar with HOF studies and thus may have a hard time understanding the proposal or may be overly critical of the methods or approaches.
To make ethics approval more streamlined some Canadian provinces have a provincial ethics approval process that is a collaboration between universities and research sites such as hospitals or health authorities. An example of such a system is in British Columbia[2]. This system enables one streamlined ethics submission and can prevent conflicting reviews between boards. An overall challenge is that HOF studies are not well known across many Canadian healthcare facilities and universities, which can introduce problems during the review process. To overcome this issue, we need better promotion of HOF methods and approaches.
3.4 Finland (J. Kaipio, M. Tyllinen)
We have experience on two different types of studies involving ethical board review in Finland: health and social care workers’ experiences on their IT systems at a national level and patients’ experiences on their illness, care, and related digital services at a local level. These studies have different ethical review processes.
In Finland, HOF studies fall under the ethical board review process if the requirements set by the Finnish Advisory Board on Research Integrity are met. These include e.g., no informed consent of subjects, subjects under 15-year-old without parental consent, subjects exposed to security risks, or an intervention with impact on physical integrity. These ethical boards exist at the universities and in the Finnish national institute for health and welfare (THL). When hospitals are involved or when the research project comes under the Medical Research Act (national law), hospitals district's ethical boards are responsible for the review.
At Aalto university, the ethical committee is responsible for the ethical evaluation of the university's non-medical research projects in human sciences. Review is required in specific research configurations, but it can also be requested if the study's publication forum, financier, or international partner requests it. The hospital district is the owner and supervisor of any research that studies their patients or in which members of their staff or data systems are utilized, or that is funded by research funds allocated by the district. In all cases, submissions are done via electronic systems and committees meet every month. At least, the following information is required when applying for an ethical review: a research plan including plan for conducting the research, the list of the persons taking part in the research, and the privacy notice of the research for participants.
In different fields of science and their related universities, ethical review boards have varying history and tradition, e.g., medical research has a long-established history compared to technical sciences. On the other hand, the recent changes in legislation, particularly the European General Data Protection Regulation (GDPR), have complicated the study planning in general, especially when best practices are still lacking. It seems that the organizations do not share a common understanding on what is regarded as personal data.
Based on our experience, the advantages of the ethical review process are: organizations have their own review boards, which are easy to reach, and processes are relatively quick. The process supports the planning of the details of the study including the privacy aspects of data storing and sharing. However, there are few concrete examples of how to apply these considerations in HOF studies, which makes the process cumbersome, bureaucratic, and lengthy. It is also difficult for researchers to know in which circumstances an ethical board should be contacted and when it is the case, which one (especially when healthcare organizations are involved in the study as subjects).
In Finland, the law considers medical research and not research ethics in general. Clearer national guidelines would be needed on whether HOF studies should have an approval from the ethical board or not.
3.5 France (S. Pelayo)
The Jarde law is a French law that governs „research organized and carried out on volunteers (either healthy or sick) with the objective of developing biological or medical knowledge“
[11]. This law defines the categories of research and the functioning of the network of national ethical boards called the committees of persons’ protection (CPP). Three categories of research are defined based on the level of risk to the subjects:
-
Interventional studies where the intervention in treatment decisions and treatments differs from usual care;
-
Interventional studies presenting minimal risks and constraints when no pharmaceutical products are involved or only under their usual conditions of use. These studies are listed in a decision[12];
-
Observational studies when there is no intervention in treatment decisions. The researcher observes treatment and results in a systematic manner without changing, influencing, or interfering with diagnosis, treatment, or monitoring.
A decree has been published that defines the kinds of studies covered by the Jarde law[13]. “Experiments in human and social sciences in the ield of health” are excluded. Consequently, it seems HOF studies do not fall within the scope of the Jarde law[14]. Yet, it is generally admitted that HOF research that might put the subjects at physical or psychological risk (e.g. questionnaire about suicidal ideations, stressful simulation) should require an ethical approval because they may be categorized in category 2 or 3. However, criteria to know whether a HOF study has to be submitted to a CPP are still subject to interpretation and discussion among researchers, institutions, and ethics boards. Submission of a protocol to a CPP is done through the platform of the French Healthcare Delivery Authority. This web platform randomly assigns submitted protocols to a regional CPP for appraisal. Review times range from 45 days for research categories 2 and 3 to 60 days for category 1. CPPs base their decision on the conditions under which the researcher ensures the protection of persons, on the merits and relevance of the research project, and on its methodological quality.
For HOF studies not covered by the Jarde law (the large majority), there is no alternative regulation. Yet, if the researcher is affiliated to an institution (e.g., a university) that has its own local ethics board, s/he might need to get its approval. Those local boards are neither standardized in form nor in procedure and have no statutory recognition (only CPPs are recognized); they “only” provide advices to their staff and support them in identifying ethical issues in their research projects and in thinking about their practices.
The Jarde law is a relevant initiative to frame research approval by proposing a common regulation and a one-stop-shop recognized interlocutor. Like any new system, it takes time to operate properly. The Jarde law and related decrees were written for traditional medical research and did not apply to every research case. Now, researchers need to be informed and trained to the Jarde law and related decisions.
The Jarde law also broadened CPP scope: the number of protocols to appraise has grown, increasing the response time and delaying research projects’ implementation. The large majority of CPP members (physicians, methodologists, pharmacists, lawyers, patients’ representatives, psychologists) are not informed about HOF studies and methods. This makes it difficult for them to make standardized and unbiased appraisals. It is also difficult for applicant HOF researchers to write “understandable” protocols. For the time being, practices in the CPP network are still quite heterogeneous with decisions not always harmonized.
3.6 Norway (E. C. Lehnbom, R. Pedersen)
Medical and health research conducted in Norway needs to be approved by an ethics committee if the aim is to trial new experimental treatments, acquire new knowledge about health and disease, or if human biological samples or identifiable personal information (either collected by researchers or obtained from one or several central health registers) is to be used.[3] Testing and evaluation of medical devices, defined as instruments or apparatus, produced to be used on people to diagnose, prevent, monitor, treat, or relieve disease, also require ethics approval. Research projects to evaluate how a new technology is being used (direct observations), to explore opinions (interviews) about new technologies, and quality assurance projects do not require ethics approval. Before submitting a complete ethics application using the online portal, researchers can submit a request for assessment by the ethics committee. The request for assessment should contain information about the study aim, methods, analysis plan, requirement strategy, and data storage, and is also submitted using the online portal. The committee then assesses the project and decides whether a full review is necessary or not.
In addition to ethics assessment or approval, researchers also need to report projects to the regional PVO (Patient Protection Agent), which in turn inform the Data Protection Services (NSD)[4] prior to commencing a research project if any personal information (such as a name on a consent form) is to be collected. There are no fees associated with the application for assessment or full review by the ethics committee, PVO, or NSD.
Submission processes (ethics and data protection) are straightforward. The review process for ethics assessment is relatively quick, and usually done within one month. In our opinion, it is useful that an external committee reviews research projects and plans for data storage to minimize the risk of harm and breaches in privacy. It is important that the process is thorough yet quick, to achieve high compliance and to avoid unnecessary delays in starting research projects.
3.7 Portugal (R. Santos, F. Leite, N.A. Silva)
In Portugal, all clinical research is regulated by Law No. 21/2014. Any systematic study designed to discover or verify the distribution or the effect of health factors, health status or outcomes, health and disease processes, or the performance and / or safety of interventions or services provided, is considered as clinical research. In the context of clinical research there are two main branches of work, clinical trials and clinical studies. Clinical studies may further be subdivided into studies with or without intervention. A study is considered without intervention when clinical practice is not altered by the study.
All clinical research studies must be approved by the Competent Ethics Committee (CEC) which is responsible for ensuring the protection of patients’ rights, safety, and well-being for all patients involved in the research. Depending on the type of study, the CEC may be the nationwide Ethics Committee for Clinical Research (CEIC) for clinical trials and interventional clinical studies, or the local, hospital-specific, Ethics Committee for Health (CES) for clinical studies without intervention.
HOF studies generally fall under the classification of clinical studies without intervention, although the Law makes no special reference to these studies. The private group Luz Saude, owner of the Hospital da Luz network where we operate, has a group-wide CES and a group-wide Clinical Research Commission (CIC), supporting its 30 hospitals and clinics. The CIC assesses the clinical relevance and scientific quality of submitted clinical studies. HOF studies are evaluated by the CIC whenever the objective of the study is within the definition of clinical research. Additionally, studies should always be evaluated by the CES, even if the study is not considered to be clinical research, unless patient data is not used.
If patients are involved in a study, informed consent may also be required, and its clarity and completeness is evaluated by the CES. If personal data of participants based on the GDPR definition are also required, a Data Protection Impact Assessment (DPIA) should be performed. The responsibility for the development of DPIA relies on the study sponsor. Hospital da Luz Learning Health (HLLH), a company within the Luz Saude group responsible for all training and research, and the local Data Protection Officer (DPO) may assist the sponsor on the development of the DPIA, and will ultimately validate (HLLH) and approve (DPO) the DPIA.
The duration of these local assessments may vary. As a reference, in the Luz Saude group, the CIC assessment will last two to four weeks, and the CES assessment will occur within a similar time frame.
If the scope of the study is by no means clinical and does not involve patient data, there is usually no need to involve the CIC or the CES, and what is required is the evaluation by the HLLH and final approval by the Luz Saude Executive Committee.
In Portugal, HOF studies that are part of the improvement of the socio-technical system that supports all care delivery, and specifically HOF studies related to medical technology, are still infrequent and unknown, which may initially hinder the evaluation process. Nevertheless, the increasing submission of HOF studies will improve knowledge and awareness of these types of studies and will help in developing the discipline. It is our belief that the increasing visibility of these studies, also at the level of Ethics Committees, will not only raise the quality requirements for these studies to a level similar to other disciplines, but will also help to create evidence of their usefulness and positive impact.
3.8 Sweden (M. Hdgglund)
In Sweden, there is only one ethical review process that is applied to all research that involves human subjects. The process is guided by the law on ethical review for research involving human subjects (2003:460[5]). This legislation applies to all research that involves physical interventions, methods that aim to affect the research participants, involves biological materials that can be traced back to a person, but also to research that includes handling of sensitive personal information (as defined in the European GDPR §9.1). Does this mean that HOF studies need ethical approval? Well, that would depend from case to case on whether or not they would handle sensitive personal information (e.g., on an individual's health) or whether they may aim to somehow affect the subject physically or psychologically.
In general, this means that a study of healthcare professional's use of IT, organizational studies and usability studies involving healthcare professionals do not require ethical approval according to the current Swedish legislation. If patients are involved, we however need to consider whether any health-related data would need to be gathered; if so - ethical approval would be required. This could be the case even if you only test an application with mock data and you document information about the study participant's current or past health issues, or their contact with healthcare. Therefore, in most health informatics research, ethical approval is sought whenever patients will be involved in the study, as sensitive personal data regarding their health will likely be documented.
All ethical reviews are handled by the Swedish Ethical Review Authority[6], which started on the first of January 2019. The work is still divided between six different regions (Goteborg, Linkoping, Lund, Umea, Uppsala and Stockholm). Each regional office has at least one division for reviewing medical research and one division for other research. Each division has 10 representatives with scientific background and five representatives of the public. The chair of a division should be or has been a judge.
When you apply for ethical approval, you send your application to the Ethical Review Authority, and it will be assigned to the region and division they consider most appropriate. The application is therefore very similar for medical research as it is for other types of research. An application for ethical review costs 5000 SEK (515 USD), or 16000 SEK (1648 USD) if more than one research institution is involved. If at a later point, an ethical approval needs to be changed (e.g. addition of a new study site, or inclusion of more study participants) a new assessment will be made for a cost of 2000 SEK (206 USD). An ethical application will usually be handled within 60 days of submission, and the applicant will have the results within two weeks of the decision.
The ethical review process in Sweden clearly takes its starting point in medical research. It is often unclear what aspects of a study focusing e.g., on the usability of an app or medical device will need ethical approval, and there exists very little guidance for researchers within this field. Therefore, we often end up applying for ethics review just to be on the safe side and are often told that ethical approval is not required for this study. Considering the costs and extensive documentation required to apply for ethical approval, it sometimes creates barriers for important research that needs to be done. An expedited application process for projects that will not expose study participants to great risk would be helpful, or a pre-application process to help determine whether a project will be exempt from applying for ethical approval.
The application process is also poorly designed to meet requirements of more design-oriented research, where formative evaluations, scoping interviews, and focus groups or workshops are needed. Especially as we see an increased interest in patient participation, patients as research partners, and patient-driven research, we need to consider how these types of research formats will affect the ethical approval process. Ethical reviews will continue to be just as important but may require a different format and an understanding of these types of research in the ethical review divisions. Finally, how we guide researchers in interdisciplinary research projects that involve both needs analysis, design, formative evaluations, and clinical testing is still not clear.
3.9 The Netherlands (L. van Velsen, S. Jansen-Kosterink)
In the Netherlands, ethical considerations regarding medical research involving humans and the procedures for requesting permission to conduct these studies, are described in the Dutch Medical Research Involving Human Subjects Act (WMO)[7]. Research is subject to the WMO if the following criteria are met: (1) it concerns medical scientific research and (2) participants are subject to procedures or are required to follow rules of behavior. If a study is subject to the WMO, it must undergo a review by an accredited Medical Research Ethics Committee (MREC). These MRECs are typically hosted by large, academic hospitals.
HOF studies are, normally, not subject to the WMO if people voluntarily participate and if the study does not infringe upon the physical and / or psychological integrity of participants. This is often the case for studies such as usability tests of a new eHealth application or focus groups in which the added value of a technology is discussed. To be sure that our HOF studies do not fall under the scope of the WMO, the authors normally ask an accredited MREC to check this assumption. If so, the MREC provides an official letter, stating that the study is not subject to the WMO. This check requires to submit the study protocol, the information letter for the subjects, and the informed consent form. After submission, this official letter is received within a week. In any case, each study will be conducted according to the principles of the Declaration of Helsinki: all subjects must provide informed consent before participation, and their data will be stored and analyzed anonymously.
If, for some reason, a HOF study does fall within the WMO, a full application for conducting the study must be sent to an MREC. For general (medical) research, that is, no research with a medicinal product, a so-called “reasonable timeline” of eight weeks applies. This means that the MREC has a maximum of eight weeks to reach a decision, unless the MREC gives notice that more time is needed, including an alternative timeline. Conducting a study that falls under the WMO without a positive decision from an accredited MREC is an offence.
The Netherlands has strict rules for research with human subjects and this makes it clear for researchers how to handle ethical review. Furthermore, the review of your medical or HOF study by an accredited MREC is very valuable. It makes one reflect on the benefit of your study against patient burden, so that the study protocol can be improved in order to comply with ethical principles. However, MRECs mainly review general medical research and have less experience with HOF studies. In some cases, this make it difficult for them to review these studies properly. The letter of exception that is often granted for HOF studies, makes it easy for researchers to publish their work, as it serves as an official statement about a study's ethical compliance.
3.10 United Kingdom (V. Lichtner)
I have been doing research on implementation and use of IT in clinical settings in England (UK) for over ten years, such as evaluations of IT implementations in doctors’ clinics in the community and hospitals in the National Health Service (NHS). This HOF research was conducted with qualitative studies aimed at understanding how technology responds to clinicians’ information needs in local sociotechnical contexts.
Any research conducted in the NHS requires approval by an NHS Research Ethics Committees (REC) and approval by local research governance (RG) bodies (e.g. hospital research units). Over the years, applications to NHS REC in England have been streamlined, leading to a centralized online submission through an Integrated Research Application System (IRAS), managed by the NHS Health Research Authority (HRA)[8]. HOF studies are usually classified as ‘low risk’ and a simplified application form is generated by the system, compared to the one suitable for clinical trials where risks of intervention to participants may be considerable. At the time of submission, IRAS suggests ‘the first available slot’ of a REC - anywhere in England - suitable for the type of study submitted. Researchers are not bound to choose ‘the nearest REC’; choosing ‘the first available slot’ may speed up the approval process.
In the context of the studies I conducted, obtaining local RG approval has been much more challenging. The process was often not standardized. Some RG bodies required contracts in place between the University and the NHS organization where the research was taking place. It often involved communication with staff locally to track the status of the application, and what would be needed to move it forward, but identifying the person to speak to was difficult. Receiving RG approval at the level of the hospital, or health authority, was also not sufficient to guarantee access to single clinical wards or GP clinics - each had to be negotiated separately. This made multisite studies incredibly difficult and time consuming to set-up, hindering research. For example, in the case of the evaluation of the electronic transmission of prescriptions in primary care clinics, we had to apply to different local authorities where the clinics were located, each taking months to respond. While the roll-out of the technology across England followed an agile approach, we were unable to be present on the sites at the same speed to observe activities[15].
The HRA has recently centralized the local RG applications, as part of the application to REC[9]. However, the researcher is still responsible for updating local sites of progress. Guidance provided on how to fill in the HRA application to satisfy local approval feels overwhelming[10]. There is also variation in how HRA administrators interpret the GDPR - anecdotally, some are placing unrealistic burdens of patient consent on researchers in clinical areas where patient information is otherwise displayed publicly on whiteboards.
A further stumbling block is that the REC-approved study protocol may place unexpected barriers on the research as it unfolds in practice. HOF qualitative research needs to be ‘open to the unexpected’. For example, understanding technology for medication in a clinical ward may take the researcher to also investigate dispensing activities in the hospital pharmacy. If this is not foreseen in the initial application, the researcher must stop the study and submit an amendment to REC, losing valuable time and perhaps valuable opportunities for gaining insight into current work practices.
3.11 United States (B. Lesselroth, J. Homco)
Our team has nearly 20 years of collective experience working on informatics projects at various US institutions. We have participated on intercollegiate, governmental, and industry projects, affording us the opportunity to work with regulatory oversight bodies at the federal, state, and private levels. Our projects tend to be operational, addressing quality improvement (QI), HIT implementations, or user experience (UX) evaluation.
Christine Grady wrote that medical research and quality improvement occupy a continuum ranging from passive observation to controlled experimentation[16]. The distinctions can be murky when conducting HOF studies - particularly when evaluating users interacting with HIT. This fact notwithstanding, we believe informaticians have an obligation to evaluate HFO impact with objectivity. This requires rigorous standards for data handling and independent review to protect patient and staff safety. Therefore, we self-govern by writing protocols for every project, ensuring a high level of methodologic rigor and a consistent process for data handling. As per institutional expectations, we solicit a review from the IRB (either a full review or an expedited process, as described below). Furthermore, we notify our IRB office of any projects involving human subjects that will likely result in publication of findings that may contribute to a cumulative knowledge base.
The University of Oklahoma rules and regulations governing research operation-alize at a local level federal regulation enforced by the US Department of Health and Human Services, the US Food and Drug Administration, the US Department of Veterans Affairs, and the US Department of Defense. The University also upholds the Federal Health Insurance and Portability and Accountability Act (HIPAA) for the protection of patient health information and the Oklahoma state laws pertaining to protections of vulnerable populations including emancipated minors and cognitively impaired individuals. Criteria used for defining and policing research are consistent with federally published standards and are similar between institutions (US Code of Federal Regulations 45FR.46.102; Bailey, Hastings Center Report, 2006).
We submit all projects - including HOF work - to our institutional IRB well in advance of any data collection. Many IRBs, including at the University of Oklahoma, offer an expedited pathway for quality improvement efforts, educational programs, and non-experimental technologies. Typically, these studies do not include participant consent forms, different exposures, randomization, or a control group. The expedited process permits IRB members to pre-screen projects and allow low-risk ones to bypass the normative IRB review and governance system. The process begins with completion of a “determination of research” checklist and provision of a project description that includes methods, types of data to be collected, and anticipated deliverables. The checklist includes criteria to help reviewers estimate risk and disambiguate quality improvement activities - including HOF work - from traditional research. At a high level, research tests hypotheses to develop generalizable knowledge, whereas quality improvement and HOF apply generalizable knowledge to “quickly” improve health delivery. The IRB at the University of Oklahoma School of Community Medicine typically responds to an application within two to four weeks.
There are several important strengths and limitations of the current IRB processes. Strengths include: (1) a single intake pathway and administrative infrastructure; (2) a clear set of criteria and definitions that can inform protocol development; (3) a streamlined set of monitors and documentation for QI and HOF work; and (4) the presence of a regulatory framework that protects stakeholders and promotes a high level of academic rigor.
Weaknesses include: (1) lack of specific guidance for evaluation of HIT including electronic health record configurations or modules; (2) ambiguity surrounding participation of clinicians or trainees when participation is mandatory as a component of care; (3) lack of administrative stakeholders with experience conducting QI or UX work (leads to difficulty understanding proposal, methods, scope, and impact); (4) the additional time required to determine the single IRB of record when multiple agencies are participating on a project; (5) an often lengthy and bureaucratic process (frequently impacted by personnel shortages); (6) a commonly held, but false dichotomy between “research” and HOF work; (7) organizational opposition to the use of “research language” in HOF and QI applications; and (8) the need for language defining “generalizability” of findings (point of contention when we seek to publish findings or lessons learned about the design, implementation, and evaluation). To this final point, we contend that QI and HOF works make important contributions to our theoretical frameworks and understanding of QI methods and implementation effectiveness.
