Development and Testing Requirements for an Integrated Maternal and Child Health Information System in Iran: A Design Thinking Case Study

Abstract

Background Management of child health care can be negatively affected by incomplete recording, low data quality, and lack of data integration of health management information systems to support decision making and public health program needs. Given the importance of identifying key determinants of child health via capturing and integrating accurate and high-quality information, we aim to address this gap through the development and testing requirements for an integrated child health information system.

Subjects and Methods A five-phase design thinking approach including empathizing, defining, ideation, prototyping, and testing was applied. We employed observations and interviews with the health workers at the primary health care network to identify end-users' challenges and needs using tools in human-centered design and focus group discussion. Then, a potential solution to the identified problems was developed as an integrated maternal and child health information system (IMCHIS) prototype and tested using Software Quality Requirements and Evaluation Model (SQuaRE) ISO/IEC 25000.

Results IMCHIS was developed as a web-based system with 74 data elements and seven maternal and child health care requirements. The requirements of “child disease” with weight (0.26), “child nutrition” with weight (0.20), and “prenatal care” with weight (0.16) acquired the maximum weight coefficient. In the testing phase, the highest score with the weight coefficient of 0.48 and 0.73 was attributed to efficiency and functionality characteristics, focusing on software capability to fulfill the tasks that meet users' needs.

Conclusion Implementing a successful child health care system integrates both maternal and child health care information systems to track the effect of maternal conditions on child health and support managing performance and optimizing service delivery. The highest quality score of IMCHIS in efficiency and functionality characteristics confirms that it owns the capability to identify key determinants of child health.

Keywords

child health - health information systems - primary health care

Introduction

The health management information system (HMIS) has a promising role in monitoring service delivery and provides evidence for health managers, researchers, and stakeholders at the district, state, and national levels.[1] [2] Numerous global initiatives, including the Sustainable Development Goals (SDG) rely on routine HMIS to support policy-making and program implementation.[3] As a result of the SDG's vision, promoting neonatal and child health serves as a global health priority, particularly in low- and middle-income countries (LMICs) with a high burden of morbidity and mortality among newborns and children.[4] The importance of golden opportunity to diagnose child disorders and make appropriate planning for treatment or rehabilitation through timely information highlights the need for reliable HMISs.[5] In 2019 alone, roughly 14,000 children under the age of 5 died every day.[6] The majority of deaths (98%) occur in LMICs with lack of overall or cause-specific deaths.[7] Several studies have drawn attention to the risk factors for death and shown that substantial proportions of all child and perinatal deaths are attributable to the maternal condition.[8] Mothers who suffer from poor nourishment, sickness, pregnancy conditions, or receive inadequate prenatal and delivery care have babies who are at a higher risk of diseases and death.[9] [10] [11] [12] Despite increasing importance being laid on capturing accurate information from various sources into an integrated information system to support decision-making and public health program needs, concerns continue to be expressed over the quality and accuracy data of HMISs.[13] Lack of accurate data of HMISs has led to missed opportunities to use data for quality improvement in health care delivery and develop key interventions affecting maternal and newborn outcomes. Several studies suggested that HMIS suffered from incompleteness and poor quality and a tendency of over- or under-reporting in HMIS data.[14] These included the presence of extreme outliers, lack of consistency of the reported data over time and between indicators (such as vaccination and antenatal care), or irregularities in report generation, data duplication, and data inconsistencies, at all levels of health care delivery for newborn and child health.[15] [16]

Bringing together information from various sources into an integrated information system could facilitate assessment and prompt appropriate services. Making comprehensive information readily available to authorized users would facilitate entry into a community-based services system for all children and their families.[17] Despite these opportunities, the existing child health care information systems suffer from disparate systems and information silos that do not support management decision-making, meaningful data analysis, research, and programmatic efforts.[18] [19] The global magnitude of adverse maternal effect on child health is impossible to determine due to the lack of a comprehensive, integrated mother and child information system.[20] Therefore, identifying key determinants of child health is critical to priority setting in policy-making. We aim to address this gap through the development and testing requirements for an integrated child health information system (CHIS) in Iran through the design thinking approach.

Methods

Study Setting

The presented project was launched by a collaboration of interdisciplinary stakeholders, including health care workers, physicians, health information management (HIM) professionals, and information technology (IT) at an academic medical university in Iran. In Iran, maternal and child health care (MCH) services are provided through (1) the primary health care (PHC) network, including urban health centers (UHCs), rural health centers, and health houses, and (2) hospitals with child and maternity units. MCH services are provided based on the health booklet and the completion of MCH forms by specialized PHC workers, family nurse practitioners, family health practitioners, child care midwives, and physicians.

The existing PHC network is governed at three levels of practice, including district, provincial, and national. Data are routinely collected at health care facilities and submitted to district offices that work under the supervision of academic medical universities. Data are compiled from paper-based documents and electronic format titled “SIB information system.” The SIB information system solely covers outpatient visits for the child (child growth, child diseases, child nutrition) and mothers (antenatal, pregnancy, and postnatal care). The labor and delivery information are collected separately and through the hospital information system.

Study Design

We applied a design thinking methodology, which is a human-centered and iterative approach to provide a solution-based method. By applying the five phases of empathizing, defining, ideation, prototyping, and testing, design thinking is a comprehensive method for information system development.[21] [22] We identified needs (empathize), defined a problem (define), and generated ideas for a solution (ideation). The idea was then implemented as a pilot integrated CHIS (prototype) and evaluated (test) using Software Quality Requirements and Evaluation Model (SQuaRE) ISO/IEC 25000. Details of each phase will be presented below.

Phase 1: Empathize

This phase involves seeking an empathetic understanding of the problem you are trying to solve through user research.[23] [24] The researchers with expertise in HIM and IT conducted interviews based on recommendations from DeJonckheere and Vaughn[25] and also used field observations. In this phase, we started working with all health care workers (seven persons) involved in the monitoring, handling, and analyzing MCH data at the district level of the PHC network at the Deputy of Health in Kashan University of Medical Sciences (DoH- KAUMS). DoH- KAUMS supervises all MCH processes conducted at the UHC, rural health center, and health houses. These seven informants with sufficient “information power,”[26] and all with more than 10 years of work experience, participated in the study as an expert panel group to conduct design thinking phases ([Table 1]). They were invited to determine a suitable time and place for an interview; one researcher interviewed each expert in their determined time and place. Interviews began with an explanation about the study objective; then questions, such as “are you satisfied with the existing MCH processes?” “Are you satisfied with the information communications you have throughout MCH processes at the PHC network?” “Is there any problem with the current MCH processes?” “Based on your point of view, what are the main problems with the current MCH processes?” were asked; each session lasted for almost 45 to 60 minutes. The sessions were audio-recorded, and key points were written down. After analyzing the data obtained through the interviews to discuss the results, two expert panel sessions were held at the DoH-KAUMS, when all members agreed to participate. Then, field observations were performed by the researchers at the DoH-KAUMS through inspecting the existing systems, Excel files, and forms in which data from the PHC network were stored, to record all MCH processes, work duplication and delays, waiting time, data collection, and report generation.

Table 1 Participant characteristics (phase 1)

Gender	Female (n = 5)
	Male (n = 2)
Discipline	General practitioners (n = 1) Childcare midwives (n = 2) Health care workers (n = 3) Health care management (n = 1)

Phase 2: Define

During the define phase, we analyzed the data obtained through observations and interviews and put them to realize the problems in human-centered terms entirely.[24] Two researchers analyzed the qualitative data. The two researchers were HIM and IT specialists because they could recognize the specific needs and requirements of the end-users and turn them into computer language more appropriately. We applied Wolcott's thematic analysis approach,[27] including description, analysis, and interpretation, to analyze the collected data and transform them into insights and themes. The results were clustered into four themes using a mind map tool ([Fig. 1]).

Fig. 1 Determined problems for maternal child health information system using a mind map.

Phase 3: Ideation

Generally, the ideation stage seeks potential solutions to the identified problems using various ideation techniques.[23] In our study, the main solution to address users' needs was to transit to an integrated maternal and child health information system (IMCHIS). Three 5-hour focus group discussions (FGDs) were conducted under the supervision of trained moderators (researchers) to identify all aspects of IMCHIS requirements and data elements. The FGD sessions were digitally recorded and transcribed by two authors to identify common themes using Wolcott's thematic analysis approach.[27]

The preliminary IMCHIS functional requirements and data elements were determined at the end of this phase. Key functional requirements and data elements were changed to a 5-point scale questionnaire from “strongly disagree = 1” to “strongly agree = 5” and distributed among expert panel groups to evaluate consensus about IMCHIS requirements and data elements. Finally, 109 data elements in seven categories of prenatal care, labor/pregnancy, postnatal care, child disease status, child health status, child nutrition status, and child allergy status were proposed for IMCHIS. Functional requirements and meaningful reports, and queries were also determined in this phase.

Phase 4: Prototyping of IMCHIS

A prototype is a realistic representation of the preliminary version that allows end-users to understand the idea's feasibility before testing through the iterative cycle.[23] Then, 109 data elements determined by the experts at the district level of DoH- KAUMS in phase 3 were distributed among all specialized health care workers (12 persons) of the departments of MCH at all urban health levels of the DoH- KAUMS ([Table 2]). We selected only UHCs since specialized health care workers mostly worked at UHCs. They were more accessible than rural health centers and health houses to participate in the different phases of the study. Data were analyzed using the Fuzzy Delphi Technique and MATLAB software (MATLAB Simulink, Math Works Corporation, United States). A table including all the data elements was created to investigate each data element's level of importance, weight coefficient, and triangular fuzzy function scaled between 0 and 1. Data elements with a weight coefficient of 0.05 and higher were approved, and those with 0.049 and lower were omitted.

Table 2 Participant characteristics (phase 4)

Characteristics		Number
Age (y)	≤29	n = 3
	30–39	n = 2
	40–49	n = 6
	50–59	n = 1
Gender	Male	n = 2
	Female	n = 10
Working experiences (y)	<5	n = 3
	5–9	n = 1
	10–14	n = 3
	≥15	n = 5
Job titles	Physicians	n = 3
	Health care workers	n = 9

It should be noted that the data elements with the weight coefficient closer to 1 were more important based on experts' opinions. A prototype model of IMCHIS was developed using the confirmed data elements and the discussed functional requirements in the Structure Query Language (SQL) server environment. Health records of 200 pregnancy cases, including maternal care status (primary care), labor, and delivery (hospital information), were entered into the prototype model of IMCHIS by researchers to verify the functionality of IMCHIS.

Phase 5: Testing of IMCHIS

Testing is the final stage of the five-stage model in which designers or users rigorously test the product that was developed during the prototyping phase. In this study, we employed ISO/IEC 25000 SQuaRE to test IMCHIS. SQuaRE is a series of next-generation software quality evaluation standards by ISO 9000 to identify software quality characteristics, including functionality, reliability, usability, efficiency, maintainability, and portability.[28] Since this project was not a large-scale production and ran without any vendor support, evaluation of maintainability and portability was not feasible in this study.[28] [29] The remaining four attributes rely on users' participation from our evaluation framework ([Table 3]).

Table 3 ISO SQuaRE testing quality model

ISO SQuaRE characteristics	Definition	Sample questions
Reliability	Capability of the software in maintaining stability and its performance under specific conditions.	• Can the system easily restores working and restores lost data after failure? • Does it enable users to handle errors? • Does it clearly and promptly inform the user when it meets an error?
Functionality	Software capability to fulfill the tasks which meet users' needs.	• Can the system perform the tasks required? • Can it produce the expected results? • Can it produce accurate results?
Usability	The quality of a user's experience when interacting with the software in terms of level of effectiveness, efficiency, and satisfaction.	• Does the system support the suitability of learning? • Is it attractive for users? • Is it consistent and complying with characteristics of the user?
Efficiency	The relationship between the level of performance of the software and the amount of the time and resource used.	• How quickly does the system respond? • How good is it at handling large documents? • How easy does the system perform work steps?

To meet ISO/IEC 14598–6 regulation,[28] there should be at least eight participants in each evaluation category. Forty participants were recruited to evaluate IMCHIS ([Table 4]), and their answers were analyzed using MATLAB software based on the Fuzzy scale.[30] [31] [32] Two methods calculated total users' satisfaction: (1) IMCHIS quality characteristic satisfaction rate based on triangular defuzzification value, which was multiplied in the array, and (2) overall satisfaction rate by end-users of the IMCHIS.[30] [31] [32]

Table 4 Participant characteristics (phase 5)

Characteristics		Number
Age (y)	≤29	n = 8
	30–39	n =  19
	40–49	n = 7
	50–59	n = 6
Gender	Male	n = 9
	Female	n = 31
Working experiences (y)	<5	n =  13
	5–9	n = 9
	10–14	n = 10
	≥15	n = 8
Discipline	Physician	n = 8
	Family nurse practitioner	n = 8
	Family health practitioner	n = 8
	Childcare midwives	n =  8
	IT/computer specialists	n = 8

Abbreviation: IT, information technology.

Ethical Considerations

The Ethics Review Board approved this study of the Vice-Chancellorship for Research Affairs of Kashan University of Medical Sciences (IR.KAUMS.REC.1396.7).

Results

In the process of developing the IMCHIS, one workshop, six interviews, two field observations, and three FGDs with different stakeholders were conducted, and the outcomes of each phase are presented as follows.

Phase 1: Empathize

The interviews lasted between 30 and 60 minutes and were transcribed verbatim; the findings were validated in one workshop with users.

Phase 2: Define

We attempted to retell the stakeholders' needs and requirements determined in the empathize phase using human-centered design terms during the define phase. A summary of the problems which users face when using the existing MCH information system can be seen in [Table 5].

Table 5 The outline of the problems which users faced with the existing MCH information system

End users' problems	Description
Use of information generated	Irrelevance of the information gathered: data collection focused more on reporting MCH conditions instead of maternal and child health promotion and only partially addresses management objectives at the health unit level or at the patient/client level.
	Lack of feedback: there is weekly or monthly reports without adequate feedback, Maternal and Child Health Workers (MCHW) rarely receives feedback on the data reported to higher levels.
	Lack of motivation: there is a lack of motivation among health workers about data quality as a result of the absence of feedback.
Data collection	Duplication in data collection: filling endless registers and forms with shared MCH care data elements (e.g., sex, age, maternal and child care conditions) that cause irregularities in report generation in the form of over- or under-reporting.
	Duplication in data entry: although maternal and child care data elements and forms are interrelated, double entry of maternal and child care data happens repeatedly, which means duplication of effort, time and increased costs, and opportunity for error.
	Poor data quality: the data received often suffer from incompleteness and inaccuracy due to standardized instructions on how to collect the data.
Lack of meaningful data analysis	“Data-driven” instead of “action-driven” Information systems: information systems are often not helpful for management decision making and suffer from pattern discovery exploration and analysis.
	Data integration: information silo which resides within a disparate system does not support combining data in different sources and providing users with a unified wide spectrum (data integration).
	Lack of linkage: linkage between maternal care and child care and fragmentation between hospital services and PHC network.
Shortcomings of IT infrastructure for meaningful data analysis	Disparate system: the Information System (IS) is a disparate system without exchanging data between maternal and child information systems that causes a lack of data analysis and meaningful information.
	Lack of IS capabilities: lack of powerful screen generators, report generators, query tools, data visualization, and advanced data analysis (e.g., data mining).
	Lack of interoperability: information system's data cannot be exported for use with other software and shared with other departments.

Abbreviations: IT, information technology; MCH, child health care; PHC, primary health care.

Phase 3: Ideation

We conducted two interactive FGDs with the experts at the district of DoH-KAUMS. The purpose was to develop a solution to meet the following requirements:

• Developing a feasible and approachable information system on a small scale and with the minimum changes on the existing process.

• Use of standardized data elements and dataset for MCH.

• Making a solid integration and linkage between public health services and hospitals.

• Generating meaningful reports and queries for MCH.

At the end of this phase, 109 data elements proposed for IMCHIS were distributed for experts' consensus. Then, 74 of 109 proposed data elements were confirmed through the Fuzzy Delphi approach. [Table 6] reveals that among seven proposed categories for the IMCHIS data elements, “child disease” with weight (0.26), “child nutrition” with weight (0.20), and “prenatal care” with weight (0.16) acquired the maximum weight coefficient.

Table 6 Weight coefficient of main requirements for IMCHIS' data elements based on experts' agreement

Main categories for IMCHIS data elements		Weight coefficient
1	Prenatal care	0.16
2	Postpartum care	0.07
3	Labor/delivery	0.06
4	Child nutritional status	0.20
5	Child health status	0.13
6	Child allergic conditions	0.12
7	Child disease status	0.26

Abbreviation: IMCHIS, integrated maternal and child health information system.

Seventy-four confirmed data elements were applied for determining the logical design phase of IMCHIS, including relationships between data elements, primary keys, and foreign keys. Generating meaningful reports to investigate the effect of mothers' conditions on adverse birth and child outcomes also considered at this phase. Users reported the disparate design as one of the major problems with the existing MCH information system. To address existing gaps, the mothers' national identification numbers were applied to integrate MCH information in the IMCHIS. The flow of events to perform a task, entity name, field, primary key, and tables for data elements' relationship were drawn using scenarios. After confirming the scenarios, we use tools from the visualized ideas and dialog tools such as a flowchart and VISIO software to draw entity relationship diagrams (ERDs). The number of ideas, functional requirements, and reports of IMCHIS were adjusted and refined throughout the ideation process.

Phase 4: Prototyping

The 74 conformed data elements were applied for developing ERDs related to MCH, and then they were transformed into a physical design model using the SQL server environment. At this stage, the IMCHIS user interface was also designed by the computer programmer. Relevant IMCHIS content and functionality were validated by entering health records of 200 pregnancy cases into IMCHIS at the prototyping phase. The proposed IMCHIS creates new insights into maternal and child health and paves the way for future planning, research, and evaluation based on generating simple queries and reports ([Fig. 2]) in the area of MCH. The followings provide some sample reports for the newly proposed IMCHIS.

Fig. 2 IMCHIS-generated simple queries and reports. IMCHIS, integrated maternal and child health information system.

Phase 5: Testing

According to the end-users, the highest level of IMCHIS quality was efficiency and functionality characteristics with a weight coefficient of 0.73 and 0.48, respectively. The “reliability” characteristic was the lowest level of the IMCHIS quality with a weight coefficient of 0.24 ([Table 7]).

Table 7 End-user's evaluation for IMCHIS quality using Fuzzy assessment matrix

ISO SQuaRE quality characteristics	Strongly agree (5)	Agree (4)	No comment (3)	Disagree (2)	Strongly disagree (1)
Efficiency	0.73	0.86	0.21	0	0
Functionality	0.48	1	0.58	0.56	0
Usability	0.35	0.89	0.50	0.06	0
Reliability	0.24	0.49	0.75	0.80	0
Final result of fuzzy evaluation	0.35	0.80	0.59	0.07	0
Overall satisfaction rate	0.82

Abbreviation: IMCHIS, integrated maternal and child health information system.

Discussion

This study aimed to develop and pilot test an integrated CHIS to support child health planning, research, and evaluation. With the aid of a design thinking methodology, the IMCHIS innovation was developed, which is a web-based system with standardized data elements and functional requirements to track the effect of maternal conditions on child health, and develop MCH reports and queries based on a comprehensive information system.

In the first phase of the design thinking approach, our results indicated that users reported duplication in data collection, lack of feedback, lack of linkage, and data integration as the main problems with the existing MCH. Our results are supported by previous studies in routine MCH information systems conducted by Hinman et al[17] and Ouedraogo et al.[33]

According to the literature, duplication of data collection and data entry causes failures in evaluating the effectiveness or efficiency of health programs. It might cause over-reporting of MCH services ranging from 1.4 to 6%.[14] Thirty-eight percent of health workers claimed that their reports and registration books might contain inconsistencies.[34]

Lack of feedback also was presented as one of the problematic areas which users faced when using the existing MCH information system. The importance of implementing peer review and feedback in completeness, timeliness, and accuracy of facility reporting also has been highlighted in previous studies.[15] [35] [36] [37] Bhattacharya et al revealed that introducing data-quality interventions, including feedback, can improve data-quality metrics in terms of availability and timeliness of reporting, completeness of data elements, the accuracy of facility reporting, consistency between related data elements, and frequency of outliers reported.[38] Despite the increasing importance being put on the use of feedback for improvement of data quality, in a study verifying the quality and consistency of immunization monitoring systems in eight countries, Ronveaux et al reported that only 50% (55/108) of districts applied standard feedback formats.[16] Therefore, implementing supervision, monitoring, and feedback to resolve the data entry errors and ensure the accuracy and consistency of the reports need to be considered in the HMIS.

In our study, the expert panel group considered the standardization of the data elements and integration of maternal and child health information systems as a preliminary solution in the ideation phase. The crucial role of integrating MCH information in the continuum of care of a child can save health program resources and enhance data quality and has been recommended in the previous studies.[13] [19] Hinman et al argued that reducing the number of registers and developing an integrated system with standard data elements would pave the way to obtain information readily about a child's status concerning other programs.[39] Standard data elements support generating indicators to monitor the provision of health services and facilitate vertical reporting of this information at all health care delivery systems.[17] [40] In our study, the standardized data elements were first identified and classified into seven main requirements. Considering the increasing importance of integrating the mother and CHIS, end-users mentioned “prenatal care” as one of the primary data elements and functionalities for IMCHIS. Our results are in line with those reported in previous studies. Evidence suggests that the original causes of many childhood diseases and deaths stem from the neonatal and pregnancy period.[9] [41] [42] [43] [44] [45] [46]

Ultimately, design thinking is a solution-based approach to meet users' needs and requirements.[21] To address this possible limitation, the IMCHIS quality was evaluated using the SQuaRE model. The results indicated that the highest IMCHIS quality level was for efficiency and functionality characteristics, focusing on software capability to fulfill the task that meets users' needs. It has been reported that the managers and users of the LMICs tend to adopt information systems that avoid fundamental changes in the existing work structure and processes.[47] Our results are in agreement with previous studies that develop CHIS. Hinman et al also argued the integrated information system, making challenges and changes, is hard to implement. Thus, the individual needs of users should be incorporated into the existing organizational culture and processes.[17] [39] Active participation of stakeholders in the design, implementation, and evaluation of information systems via a design thinking approach would address these challenges. However, this study had two limitations: first, the studied population might not represent the country. Although health care services in Iran are centralized, MCHs are rendered through the same PHC network, which consists of the same organizational structure, work processes, and information systems. Second, since CHIS failures in developing countries are noticeable because of social, cultural, and economic difficulties, we conducted a pilot project. Some parts of the ISO testing quality model, including maintainability and portability, were not considered in the current study. However, IMCHIS was validated by entering health records of 200 pregnancy cases into the prototyping phase. Moreover, the remaining ISO characteristics were conducted through the participation of multidisciplinary end-users from technical and clinical domains.

Conclusion

According to the end-users, the most suitable CHIS integrates mother and child health care requirements to minimize data duplication and inconsistencies and supports management decision-making, meaningful data analysis, research, and programmatic efforts. The highest quality score in efficiency and functionality of the IMCHIS in the testing phase acknowledges that it can fulfill end-users' needs for identifying key determinants of child health. The design thinking approach serves as a user-centered and solution-based approach for developing an integrated CHIS based on users' perspectives, workflows, and tasks.

Conflict of Interest

None declared.

Authors' Contributions

Z.M., A.M., and S.G., have made substantial contributions to the conception and design. S.G., H.G., and F.V. have made substantial contributions to data collection. Z.M., S.G., G.M., A.N., F.H., and S.Z. analyzed the data. Z.M., S.Z., and F.H. drafted the manuscript and all the authors have read it to revise it critically for important intellectual content.

• References

• 1 Arsenault C, Yakob B, Kassa M, Dinsa G, Verguet S. Using health management information system data: case study and verification of institutional deliveries in Ethiopia. BMJ Glob Health 2021; 6 (08) e006216
• 2 Nshimyiryo A, Kirk CM, Sauer SM. et al. Health management information system (HMIS) data verification: a case study in four districts in Rwanda. PLoS One 2020; 15 (07) e0235823
• 3 Bhattacharya AA, Umar N, Audu A. et al. Quality of routine facility data for monitoring priority maternal and newborn indicators in DHIS2: a case study from Gombe State, Nigeria. PLoS One 2019; 14 (01) e0211265
• 4 Ismail S, Alshmari M, Latif K, Ahmad HF. A granular ontology model for maternal and child health information system. J Healthc Eng 2017; 2017: 9519321
• 5 Gozali E, Safdari R, Ghazisaeedi M, Rahimi B, Eslamlou HF, Mehraeen E. Identification and validation of requirements for a registry system of children's developmental motor disorders in Iran. Methods Inf Med 2019; 58 (4-05): 124-130
• 6 UNICEF. Under five mortality. Accessed March 15, 2018 at: https://data.unicef.org/topic/child-survival/under-five-mortality/
• 7 Arach AAO, Tumwine JK, Nakasujja N. et al. Perinatal death in Northern Uganda: incidence and risk factors in a community-based prospective cohort study. Glob Health Action 2021; 14 (01) 1859823
• 8 Debelew GT. Magnitude and determinants of perinatal mortality in Southwest Ethiopia. J Pregnancy 2020; 2020: 6859157
• 9 Worke MD, Mekonnen AT, Limenh SK. Incidence and determinants of neonatal mortality in the first three days of delivery in northwestern Ethiopia: a prospective cohort study. BMC Pregnancy Childbirth 2021; 21 (01) 647
• 10 Kendrick J, Sharma S, Holmen J, Palit S, Nuccio E, Chonchol M. Kidney disease and maternal and fetal outcomes in pregnancy. Am J Kidney Dis 2015; 66 (01) 55-59
• 11 Mahande MJ, Obure J. Effect of interpregnancy interval on adverse pregnancy outcomes in northern Tanzania: a registry-based retrospective cohort study. BMC Pregnancy Childbirth 2016; 16 (01) 140
• 12 Blomberg M, Birch Tyrberg R, Kjølhede P. Impact of maternal age on obstetric and neonatal outcome with emphasis on primiparous adolescents and older women: a Swedish Medical Birth Register Study. BMJ Open 2014; 4 (11) e005840
• 13 Mate KS, Bennett B, Mphatswe W, Barker P, Rollins N. Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One 2009; 4 (05) e5483
• 14 Sharma A, Rana SK, Prinja S, Kumar R. Quality of health management information system for maternal & child health care in Haryana State, India. PLoS One 2016; 11 (02) e0148449
• 15 Maïga A, Jiwani SS, Mutua MK. et al. Generating statistics from health facility data: the state of routine health information systems in Eastern and Southern Africa. BMJ Glob Health 2019; 4 (05) e001849
• 16 Ronveaux O, Rickert D, Hadler S. et al. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull World Health Organ 2005; 83 (07) 503-510
• 17 Hinman AR, Eichwald J, Linzer D, Saarlas KN. Integrating child health information systems. Am J Public Health 2005; 95 (11) 1923-1927
• 18 Damayanti NA, Setijanto D, Hargono A. et al. Integrated information system for early detection of maternal risk factors based on continuum of care approach of mother and toddler cohorts. Healthc Inform Res 2019; 25 (03) 153-160
• 19 Frøen JF, Myhre SL, Frost MJ. et al. eRegistries: electronic registries for maternal and child health. BMC Pregnancy Childbirth 2016; 16: 11
• 20 Graham W, Cairns J, Bhattacharya S. et al. Maternal and perinatal conditions. In: Jamison DT, Breman JG, Measham AR. et al, eds. Disease Control Priorities in Developing Countries. 2nd ed. Washington, DC: The International Bank for Reconstruction and Development/The World Bank; New York: Oxford University Press; 2006
• 21 Petersen M, Hempler NF. Development and testing of a mobile application to support diabetes self-management for people with newly diagnosed type 2 diabetes: a design thinking case study. BMC Med Inform Decis Mak 2017; 17 (01) 91
• 22 Ector GI, Westerweel PE, Hermens RP. et al. The development of a web-based, patient-centered intervention for patients with chronic myeloid leukemia (CMyLife): design thinking development approach. J Med Internet Res 2020; 22 (05) e15895
• 23 McLaughlin JE, Wolcott MD, Hubbard D, Umstead K, Rider TR. A qualitative review of the design thinking framework in health professions education. BMC Med Educ 2019; 19 (01) 98
• 24 Gestwicki P, McNely B. A case study of a five-step design thinking process in educational museum game design. Proceeding Meaningful Play. 2012. Accessed June 1, 2022 at: https://meaningfulplay.msu.edu/proceedings2012/mp2012_submission_37.pdf
• 25 DeJonckheere M, Vaughn LM. Semistructured interviewing in primary care research: a balance of relationship and rigour. Fam Med Community Health 2019; 7 (02) e000057
• 26 Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res 2016; 26 (13) 1753-1760
• 27 Wolcott HF. Writing up qualitative research...better. Qual Health Res 2002; 12 (01) 91-103
• 28 Behkamal B, Kahani M, Akbari MK. Customizing ISO 9126 quality model for evaluation of B2B applications. Inf Softw Technol 2009; 51 (03) 599-609
• 29 Jung H-J, Hong S-J. The quality control of software reliability based on functionality, reliability and usability. Paper presented at: International Conference on Future Generation Information Technology; Gangneug, Korea, December 16–19, 2012
• 30 Kanellopoulos Y, Panos A, Dimitris A. et al. Code quality evaluation methodology using the ISO/IEC 9126 standard. Int J Softw Eng Appl 2010; 1 (03) 17-36
• 31 Chou P. Study of the measurement of software product quality model. Accessed February 16, 2020 at: https://www.airitilibrary.com/Publication/alDetailedMesh1?DocID=U0081-0607200917233220
• 32 Amiri M, Darestani-Farahani A, Mahbob-Ghodsi M. Multi-criteria Decision Making [in Persian]. 1st ed. Tehran: Kian Academic Press; 2016
• 33 Ouedraogo M, Kurji J, Abebe L. et al. A quality assessment of Health Management Information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia. PLoS One 2019; 14 (03) e0213600
• 34 Abajebel S, Jira C, Beyene W. Utilization of health information system at district level in jimma zone oromia regional state, South west ethiopia. Ethiop J Health Sci 2011; 21 (Suppl. 01) 65-76
• 35 Wagenaar BH, Gimbel S, Hoek R. et al. Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009-2012. Popul Health Metr 2015; 13: 9
• 36 Mphatswe W, Mate KS, Bennett B. et al. Improving public health information: a data quality intervention in KwaZulu-Natal, South Africa. Bull World Health Organ 2012; 90 (03) 176-182
• 37 Mutale W, Chintu N, Amoroso C. et al; Population Health Implementation and Training – Africa Health Initiative Data Collaborative. Improving health information systems for decision making across five sub-Saharan African countries: implementation strategies from the African Health Initiative. BMC Health Serv Res 2013; 13 (Suppl. 02) S9
• 38 Bhattacharya AA, Allen E, Umar N. et al. Improving the quality of routine maternal and newborn data captured in primary health facilities in Gombe State, Northeastern Nigeria: a before-and-after study. BMJ Open 2020; 10 (12) e038174
• 39 Hinman AR, Atkinson D, Diehn TN. et al. Principles and core functions of integrated child health information systems. J Public Health Manag Pract 2004; ;(Suppl): S52-S56
• 40 Shaw V. Health information system reform in South Africa: developing an essential data set. Bull World Health Organ 2005; 83 (08) 632-636
• 41 Chevallier M, Debillon T, Pierrat V. et al; Neurodevelopment EPIPAGE 2 Writing Group. Leading causes of preterm delivery as risk factors for intraventricular hemorrhage in very preterm infants: results of the EPIPAGE 2 cohort study. Am J Obstet Gynecol 2017; 216 (05) 518.e1-518.e12
• 42 Beck S, Wojdyla D, Say L. et al. The worldwide incidence of preterm birth: a systematic review of maternal mortality and morbidity. Bull World Health Organ 2010; 88 (01) 31-38
• 43 Lumley J, Oliver SS, Chamberlain C, Oakley L. Interventions for promoting smoking cessation during pregnancy. Cochrane Database Syst Rev 2004; (04) CD001055
• 44 Chan A, Keane RJ, Robinson JS. The contribution of maternal smoking to preterm birth, small for gestational age and low birthweight among Aboriginal and non-Aboriginal births in South Australia. Med J Aust 2001; 174 (08) 389-393
• 45 Robert MF, Neff RK, Hubbell JP, Taeusch HW, Avery ME. Association between maternal diabetes and the respiratory-distress syndrome in the newborn. N Engl J Med 1976; 294 (07) 357-360
• 46 Halvorson KL, Vogt HB, Kightlinger L, Stevens D. The impact of maternal diabetes, obesity and race on infant birth weights in South Dakota. S D Med 2017; 70 (02) 61-66
• 47 Kheybari S, Rezaie FM, Naji SA, Javdanmehr M, Rezaei J. Evaluation of factors contributing to the failure of information systems in public universities: the case of Iran. Inf Syst 2020; 92: 101534

Publication History

Received: 21 June 2021

Accepted: 24 May 2022

Accepted Manuscript online:
24 May 2022

Article published online:
19 September 2022