User-Centered Development of an Online Platform for Drug Dosing Recommendations in Pediatrics

Abstract

Background Drug therapy in pediatric patients is a complex process. Children are subject to continuous growth and variation in drug-metabolizing enzyme activity, requiring continuous adaption of dosages. In Germany, currently no publicly available database exists that provides evidence-based information on drug dosages in pediatrics. For local drug dosing support, a prototype database has been developed within the Children's Hospital, Erlangen. A user-centered development process was initiated to establish an online platform for evidence-based dosing recommendations, as well as pharmacological and pharmaceutical drug information in pediatrics.

Objectives The objectives of the study were to survey the demand for such a platform and to assess the usability of the different versions of the developed system.

Methods The developed prototype was evaluated in a pluralistic walkthrough with prospective end users. After a redesign, the second prototype of the online platform underwent an online usability testing based on a tailored questionnaire and the System Usability Scale (SUS) (n = 12).

Results Eleven of 12 participants expressed a demand for an online platform for pediatric dosing recommendations. The majority of the participants requested the integration of extended features, such as drug–drug interaction alerts, or information on adverse effects, pharmacokinetics, and pharmacodynamics. Particularly noteworthy is the demand for an online calculator; 5 of a total of 15 participants explicitly requested a calculator for dosages (based on age, weight, body surface) and glomerular filtration rate. The usability of the second prototype was rated “good to excellent” with a median SUS of 81.25.

Conclusion Local domain experts demand an online platform for pediatric dosing recommendations. The application of the user-centered design approach enabled the development of a prototype suitable for practical use. Multiple additional required functionalities have been identified, whereby the importance of an online calculator for patient–individual dosing recommendations was particularly emphasized.

Keywords

decision support techniques - drug therapy - computer-assisted - online systems - knowledge bases - pediatric dosages

Background and Significance

Drug therapy in pediatric patients is a complex process. Children are not simply small adults, but are subject to continuous growth and variation in drug-metabolizing enzyme activity, which requires continuous adaption of dosages. Extrapolation of adult data to guide drug dosing in children requires great diligence and cannot replace clinical trials on efficacy, safety, and pharmacokinetics in pediatric patients.[1] [2] For most drugs, the costs of specific clinical trials and the development of an age-appropriate formulation (e.g., a liquid with good palatability, dosing accuracy for low doses, and without inappropriate preservatives) exceed the possible revenue, especially for conditions that seldom occur in minors. Both the U.S. and the European Union have introduced legislation for the development of pediatric medicines. Pharmaceutical companies are granted an additional 6 months' patent extension as an incentive for pediatric trials on newly developed drugs. According to the pediatric regulation, since 2007 for all new drugs early in the developmental stage, a pediatric investigation plan has to be delivered, that will be assessed and agreed by the pediatric committee of the European Medicine Agency.[3] This is mandatory for drug licensing.

Nonetheless, there is still a substantial number of medicines on the market that are not licensed for pediatric populations. Thus, off-label use is common in both ambulatory and hospital settings.[4] [5] [6] The Study of Health of Childs and Youth in Germany (KiGGS study) found that nearly one-third of medicines used in pediatric outpatients were not licensed for their age.[4] The younger the patient, the higher the prevalence of off-label uses.[4] [7] This prevalence is higher in the inpatient setting—and especially in intensive care units.[8] In Germany, there is currently no database that provides publicly accessible evidence-based information on drug dosages in children and adolescents. However, worldwide there are very few formularies providing evidence-based dosing recommendations and transparency on sources used.[9] Therefore, the information provided may not correspond to the best available evidence. Especially for the widespread off-label use, these details are essential to find a rational, best evidence-based therapy for a patient and at the same time minimize the risks during drug application. In other countries, such as the United Kingdom, the Netherlands, and Switzerland, a national database for pediatric medicines has brought significant progress in pediatric pharmacotherapy.[9]

With the implementation of an electronic prescribing system at the Department of Pediatrics and Adolescent Medicine at the University Hospital Erlangen, Germany, in 2012, dosing recommendations based on a systematic research of clinical pharmacological evidence were established for internal use.[10] To expand access to this comprehensive information to a wider audience of pediatricians in Germany, initial funding was granted from the German Ministry of Health for the development of a Web-based online platform, termed Pediatric Dosing Recommendations (PaedDos). PaedDos is intended to serve as a German reference database for evidence-based dosing recommendations as well as pharmacological and pharmaceutical drug information in pediatrics. A separate journal article focusing on the pharmaceutical content of the platform is currently being finalized.

To achieve a high level of acceptance among medical experts, the development of PaedDos was based on a user-centered design (UCD) approach with its iterative steps.[11] In a UCD approach, the end user—in this case the resident doctors, clinicians, and pharmacists—is highly involved in all steps of the development process and the user interface design. Functional scope and operation are tailored to the feedback of the test users. Through cyclic evaluations using various methods (e.g., pluralistic walkthroughs, user tests, thinking-aloud tests[12] [13] [14] [15]), the system is constantly evaluated in terms of its usability and acceptability by the end users. Standardized scoring systems, for example, System Usability Scale (SUS),[16] [17] [18] can be used for objective usability assessment. Research shows that a UCD process increases the usability of applications in health care and thus can improve the productivity of systems, their accessibility, and reduces the risk of harm.[19] [20] [21]

This article describes for the first time the development of Web-based online platform for pediatric dosing recommendation in Germany. The specific purpose of this article is: (1) to survey the demand of physicians for an evidence-based knowledge base, to check to what extend the requirements are fulfilled, and which new requirements arise, and (2) to assess a prototype of an online platform with end users in terms of usability and functionality.

Methods

Initial Prototype Development

Pediatricians from the Erlangen Children's Hospital and the project management created a list of initial requirements for the online platform. One of these requirements was to tailor the online platform to usage by three main user groups according to the content, the presentation of the prepared information, as well as the functionality and the design. The first group consists of pediatricians in private practice, including general practitioners with a large proportion of pediatric patients, which later would be the largest and most important user group; the second group comprises clinicians from pediatric hospitals; and the third group are pharmacists, who also need information on dosages of drugs for children and adolescents in their daily work. The initial requirements led to paper mock-ups having a visual basis for discussions around the platform, its design, and its functionalities. After an internal feedback round, the platform and its content were divided into six modules:

1. Information (general information about the active substances).

2. Dosage (dosing recommendations).

3. Special dosage information (information about liver and kidney insufficiency).

4. Side effects and warnings (including overdosing and contraindications).

5. Drug–drug interactions.

6. Preparations (information about the available formulations like flavor).

Subsequently, a first interactive prototype was developed, based on common Web technologies (HTML, CSS, JavaScript, PHP, and SQL) and frameworks (W3.CSS, Bootstrap, JQuery, AngularJS). This prototype was evaluated in two rounds with external users previously not involved in the project to analyze to what extent user requirements have already been considered and to derive further requirements.

First Evaluation Round—Pluralistic Walkthrough

During a pluralistic walkthrough in July 2017, experts and future end users of the system were invited to evaluate the initial prototype to draw attention to errors and provide information on its acceptance at an early stage of development. A pluralistic walkthrough is a collaborative usability evaluation method that involves both users and members of the product team. In this method, the test moderator presents the participants the system and invites discussion related to usability issues. At this development state, the system contained information in all modules for seven active ingredients. The active substances were selected from different groups (on-demand medication, antibiotics, antiepileptics, etc.). About 100 active substances were listed with dosing recommendations but not yet fully developed with all information in all modules.

Participants

In total, three participants, one representative of each of the main user groups, and the development team took part in the pluralistic walkthrough.

Procedure

The pluralistic walkthrough was divided into two parts. For the first part, three exemplary drug ordering scenarios with clindamycin, ibuprofen, and acetylsalicylic acid were defined. Based on these scenarios, the online platform was presented to the walkthrough participants. Next, the scope of search functions (search for active substances, trade name, indication, and Anatomical Therapeutic Chemical [ATC] code) and the live search functionality (search while typing) were demonstrated to the participants. Finally, all participants had the opportunity to test the system themselves with open scenarios and ask questions about the platform's user interface.

In the second part of the pluralistic walkthrough, the participants were asked to write down both positive and negative aspects of the platform on cards. All cards were collected by the moderator on the flipchart, categorized, and prioritized in an open discussion with the participants. Participants were further encouraged to evaluate the user interface in terms of color, terminology, redundancy, content, and features. In addition, they were motivated to express wishes concerning the functionalities which the current prototype version did not yet meet.

The complete walkthrough was recorded with the software Camtasia Studio (https://www.techsmith.de/camtasia.html) and later transcribed verbatim. The main statements, as well as the results of the categorization and prioritization of recommendations and wishes, were summarized to create an enhanced version of the requirements specification. The joint categorization process led to three requirement priority levels, from “high need for action,” via “medium need for action,” to “low need for action.” Feedback on issues that required only minor adjustments was implemented into the platform immediately after the walkthrough.

Second Evaluation Round—Online User Test

Prior to the second evaluation round, some requirements defined and analyzed during the pluralistic walkthrough were integrated into the development process to tailor the online platform more to the end users' needs. The requirements defined under high and medium need for action could not be integrated due to various problems explained in the “Discussion” section. All other requirements were implemented. For the second evaluation round, the number and content of active substances in the database was expanded. For the online test, approximately 50 active substances with information in all modules were available to the users, as well as a total of approximately 150 listed active substances with dosage recommendations.

Participants

For the user test, pediatricians, pediatric clinicians, and pharmacists from Bavaria, Germany, were contacted by the authors. A total of 12 persons could be recruited who were willing to participate in the online test: 6 resident doctors, 3 clinicians, and 3 pharmacists. The chosen distribution reflects the intended distribution of end users after the release of the online platform. None of the participants had previously participated in the pluralistic walkthrough. The participants had no economic interest and were not part of the project team.

Scenarios

For the online user test, two different drug ordering scenarios were defined by the development team for each of the three end-user groups ([Supplementary Appendix A], available in the online version). During the online user test, the participants got access to the revised prototype and were asked to evaluate the prototype based on their respective two scenarios. All scenarios were developed in collaboration with a pediatrician and pharmacists.

Online Questionnaire

After testing the prototype, the participants were instructed to answer questions about the content, functionality, and user-friendliness. The questions were based on the SUS[16] [17] [18] and a set of self-developed questions ([Supplementary Appendix B] [available in the online version]: questions about the design and functionality), focusing on the particular drug dosing information to capture an overall level of usability.

All self-developed questions were designed by a usability expert at the Department of Medical Informatics in Erlangen and pretested in terms of understandability. A 5-point Likert scale (ranging from “does not apply at all” [1] to “completely applies” [5]) was used for numerical evaluation. If a question was answered in negative ([1], [2], or [3]), an additional question was displayed to ask the participant for details in free text.

In total, the online questionnaire comprised 90 questions, categorized as follow:

• Demographic questions (n = 11).

• SUS questionnaire (n = 10).

• Questions about the technical devices used (n = 5).

• Questions about the scenarios (n = 12).

• Questions about the design and functionality of the online platform (n = 49).

• Positive and negative aspects of the online platform (n = 2).

• Free-text field for notes and wishes of any kind (n = 1).

The following demographic information was collected (with different answer options): user group (physician in private practice, clinician, pharmacist), professional experience in years (little: ≤ 10, medium: > 10 and ≤ 20, much: > 20 years), number of prescriptions per day (less: ≤ 10, medium: > 10 and ≤ 25, many: > 25 medications per day), and computer knowledge (subjective/own assessment: low, medium, high) to examine the dependencies of the results according to these aspects.

System Usability Score

To ensure global comparability of the system's usability with similar international developments and systems, the SUS was calculated. The SUS is a standardized questionnaire with a 5-point scale (ranging from “does not apply at all” to “completely applies”), whereby questions are alternately formulated positively and negatively. The statements relate to a range of aspects of system use, such as complexity, ease-of-use, and learnability. Overall SUS results range from 0 (poor) to 100 (very good).[17] [18] As described by Brooke,[16] the threshold values are worst imaginable (25), poor (38.5), ok (52), good (73), excellent (85), and best imaginable (100). To summarize SUS responses from participants, mean, median, minimum, and maximum were calculated.

Procedure

The online questionnaire was created with the online tool SoSci Survey (https://www.soscisurvey.de/). The participants had 2 weeks in December 2017 to complete the questionnaire for the two scenarios. In addition, each test user had the opportunity to explore the knowledge base independently, to search for dosing recommendations and information on active substances and to familiarize oneself with the functions, even during the processing of the questionnaire. The test users had full access to the online platform and its contents.

Data Analysis

Through the structure of the questions in 5-point Likert scales, a numerical evaluation of the questions was possible; frequencies were analyzed and means were calculated. With the exception of the SUS questionnaire, a rating of 5 was interpreted as positive and a rating of 1 as negative. Mean differences were compared between the three user groups and concerning a possible influence of information technology knowledge, years of work experience, and the number of daily prescriptions. The text-based answers were examined and scanned for significant statements, clustered, and also checked for dependencies.

Results

Results of the Pluralistic Walkthrough—First Evaluation Round

Positive Aspects

Especially the clarity of the user interface and the structure were appreciated, so that the users were able to quickly comprehend the user interface layout and system navigation. The different search functions (by list of active substances, dropdown list of live search, search options) turned out to be very user-friendly and were well received by the participants.

Negative Aspects

Negative feedback (wishes, missing functions for a good workflow, and optimization potential of the system regarding design and structure) was categorized as high need for action, medium need for action, and low need for action.

High need for action:

• Interaction check of the active substance with other active substances (e.g., interactions with isoniazid).

• Smart prioritization of active substances for the treatment of children (e.g., display ibuprofen before acetylsalicylic acid in search results for indication fever) based on the experts' definitions.

• A calculator for dosages (age, weight based on active substance), glomerular filtration rate according to Schwartz et al,[22] [23] and body surface.

Medium need for action:

• Compressing and standardization of the structure of the dosing recommendation table (previously 6 subdivisions, recommended only 2, maximum 4) ([Fig. 1]).

Low need for action:

• Add application time (e.g., before/after a meal) to some active substances.

• Applying a uniform font style and font size.

• Addition of search function by trade name.

The following optimization suggestions were implemented immediately after the walkthrough:

• Links to the referenced sources in the individual dosing recommendation for quick and easy verification.

• Prioritization of the drug monographs and dosing recommendations when the page is opened (not showing disclaimer).

• Disclaimer as menu item, but not as main page.

• Link to official form for spontaneous reporting of adverse drug reactions.

Fig. 1 Excerpt from the table of dosing recommendation using ibuprofen as an example.

Results of the Online User Test—Second Evaluation Round

The results of the first evaluation round were implemented and a next prototype version was created, which was evaluated by an online test with end users. In the following, the results according to the different question sections are presented.

Technical Devices Used

Most participants (n = 10) responded that they would prefer to use PaedDos on a desktop personal computer. Nevertheless, they also expressed demand to use the system on mobile devices (smartphones: n = 8, tablets: n = 5; multiple answer selections were allowed). Therefore, a responsive design of the user interface for future PaedDos versions is recommended. Problems with the use of different browser types were not recorded.

Design and Functionality of the Online Platform

Almost all questions about the functions and design of the online platform received positive or very positive feedback. Except for one question (GP07_04), all averages and medians were above 3 and thus on the positive side of the answer scale ([Fig. 2]). All answers to questions about design and functionality can be seen in [Table 1].

Fig. 2 Mean values of questions concerning the design and functionality of PaedDos.

Table 1 Excerpt of questionnaire and answers with mean, median, maximum, and minimum about the design and functions of the online platform

ID	Question	Mean	Median	Maximum	Minimum
GP02	If I had an automatic dosage calculator on the platform, I would use it	4.58	5	5	2
GP05	The possibility of searching according to the criteria “active substance,” “trade name,” “indication,” and “ATC code” is sufficient for my work	4.64	5	5	3
GP15	The user interface of the online platform is visually attractive (colors, fonts, contrast)	3.83	4	5	2
GP16	The font size was easy for me to read	4.50	5	5	2
GP19	I believe that the online platform has the potential to improve the quality of the medication prescription for children and adolescents	4.67	5	5	3
GP23	If I had access to this online platform, I would most likely use it in the next months	4.58	5	5	4
GP24	Please specify for each module how often you would use it in your clinical routine
GP24_01	Module 1: Information	3.33	3	5	2
GP24_02	Module 2: Dosage	4.50	4.5	5	4
GP24_03	Module 3: Special dosage information	3.92	4	5	3
GP24_04	Module 4: Side effects and warnings	4	4	5	3
GP24_05	Module 5: Interactions	3.83	4	5	2
GP24_06	Module 6: Preparations	4.08	4	5	3

Abbreviation: ATC, Anatomical Therapeutic Chemical.

Note: Full questionnaire available in [Supplementary Appendix B] (available in the online version).

The function for searching via ATC codes was perceived as unimportant.

As already expressed in the pluralistic walkthrough, there is a great demand for an automatic online dosage calculator. The free-text responses confirmed this pattern, insofar as four users explicitly mentioned a dosage calculator.

Strengths and Weaknesses of the Online Platform

The free-text answers to the pros and cons of the online platform were clustered. The classified answers to the strengths of the online platform are shown in [Table 2]. Multiple answers of a participant that were categorized in the same category were summed. The user-friendliness was perceived as positive by 8 out of 12 participants (e.g., “Fast, clear, easy”).

Table 2 Clustered free-text answers (with examples) to the strengths of the online platform

Category	Frequency
Usability “Easy, fast”, “Quick overview […]”, “[…] simple availability […]”	14
Dosing recommendations “Therapy duration visible”, “Dosage available for various indications”	6
Indication “Disease specific listing”, “Linking drugs with indications”	5
Preparations	4
Special dosage and application information	3
Search function	3
Active substance information	2
Approval status	1
Reference (transparency)	1

The presentation, the information content, and the quick availability of the dosing recommendations have been explicitly named a total of 6 times as one of the great strengths of the system (e.g., “Fast dosage finding”).

Particular emphasis was placed on the combination of indications and diseases for active substances. This was also named as one of the strengths by the participants (e.g., “Dosage can be viewed for different indications”).

Participants' free-text responses to existing weaknesses, suggestions for improvement, and wishes were classified into four categories ([Table 3]). Multiple answers of one participant that were categorized in the same category were summed.

Table 3 Clustered free-text answers (with examples) to the weaknesses of the online platform

Answer	Frequency
Content and further information “Further drugs”, “[…] side effects according to clinical relevance”	17
Design “Graphical representation”, “standardized dose display”	10
Functions “List therapy alternatives”, “Alternative drugs references […]”	9
(Dosage-) Online-Calculator “Automatic calculation […] after entering the weight”	5

SUS Results

Of the total of 12 participants, 2 had to be dropped from the SUS calculation because of partly missing answers, thus invalidating their results for the SUS.

With an average value of 79.5 and a median of 81.25, the prototype of the online platform is classified between “good” and “excellent” and is thus regarded as an “acceptable system.”[16] Every participant rated the system as “accepted” (minimum: 52.5; maximum: 97.5).

Physicians in private practice tend to rate the system slightly lower, while pharmacists rate it slightly higher. Further, SUS results did not show any significant deviations according to the respective parameters ([Table 4]).

Table 4 SUS mean and median according to the dependencies of the analyzed parameters

Dependency on…	Resident doctors	Clinicians	Pharmacists
User group	Average: 74	Average: 81.25	Average: 87.5
	Median: 75	Median: 81.25	Median: 85
	Less	Medium	Much/Many
Experience in the profession	Average: 80	Average: 78.5	Average: 81.25
	Median: 82.5	Median: 80	Median: 81.25
Number of prescriptions per day	Average: 78.75	Average: 85.625	Average: 80.83
	Median: 78.75	Median: 86.25	Median: 80
	Low	Medium	High
Computer knowledge	Average: -	Average: 76.68	Average: 83.75
	Median: -	Median: 77.5	Median: 83.75

Abbreviation: SUS, System Usability Scale.

In summary, the possible end users have expressed themselves very positively about the online platform and are already describing it as practical.

Discussion

Principal Findings

The results of the study clearly show the added value that such a platform would provide for pediatric and adolescent medicine. Nine participants agreed completely that the platform has the potential to increase the quality of the medication prescription for children and adolescents. All participants stated that they would most likely use the online platform if they had access to this page. This confirms that the health care professionals in pediatrics and adolescent medicine want and need such an online platform and consider it to be useful.

A serious problem in pediatrics is the extensive off-label use, where the summary of product characteristics rarely provides sufficient information for dosing of drugs in children and adolescents. Health care professionals, such as physicians and pharmacists, need to retrieve information from various sources, which often do not provide the evidence required to the intended uses. This is time-consuming, incompatible with the stressful daily routine, and entail an increased risk for the patients. PaedDos aims to provide children and adolescents with comprehensive, evidence-based dosing information and thus significantly supports pediatricians in the prescribing and pharmacists in the dispensing and verification process.

During the pluralistic walkthrough, the use of varying fonts was criticized, leading to redesign and font standardization prior to the online user test. With the exception of 3 participants, everyone was completely satisfied with the font size of the system. This illustrates that the close user involvement and early system adaption were positively accepted. The positive feedback from the participants of the user test indicates that the UCD approach provides an effective way of structuring and simplifying the development process. Moreover, it might make the development comprehensible to end users. UCD facilitates the identification of requirements for design, function, and content, both on a general level and with respect to specific details. The implemented functionality related to those requirements that resulted from the pluralistic walkthrough in the first round (e.g., to link the source information) received a positive feedback when they were evaluated in the second round. This might serve as another example for the advantages of a UCD approach.

The results of the user test indicate that the implemented search functions are sufficient for users to obtain information. The division of the contents into 6 modules made it possible to individually assess the added value of each module. The participants stated that the extent of use of dosing information (module 2) in clinical routine is expected to be “often to always.” On average, users would often use information on specific dosage information (module 3), side effects and interactions (module 4 and module 5), and information on existing formulations in everyday clinical practice (module 6). Such a knowledge base should therefore not only contain information on the dosing recommendations and make them available to users, but should also provide further knowledge and evidence. General information on active substances, as offered in module 1 on the online platform, would be used occasionally by participants. Even if this demand is lower than for the other modules, it is nonetheless present, and it is therefore reasonable to integrate this information into the online platform.

Two main points of criticism were expressed by the participants with the scope of the evaluations, which need to be discussed in detail. First, the presentation of the dosing recommendations was criticized, in particular the table width. Second, the demand for an online calculator was stated several times.

The variety of information (single dose, daily dose, maximum daily dose, frequency, and standard dose) increases the width of the table of dosing recommendations, thereby forcing the user to scroll sideways, especially on screens of normal or small size. Nevertheless, it is necessary to present each of the different contents, since for some drugs the description of a single dose is important, while for others the description of the standard dose and frequency is important. As a result, some table cells are empty ([Fig. 1]). It would be possible to hide them, but at the cost of fast information extraction, since the user would be forced to check the information twice. Therefore, it is important to always provide the information in the same place, or alternatively to apply an unambiguous label. Labeling options could be colors or symbols that would need to be checked in a further evaluation round.

The pluralistic walkthrough and the online test have shown a strong demand for an online dosage calculator to facilitate daily work with pediatric medication and to improve the quality of care. The implementation of a dosage calculator would be especially helpful if PaedDos would be integrated in a providers' electronic health record (EHR) system and input parameters for such formula (e.g., patient age, weight, and body surface) would be directly drawn from the patients' medical record. Actually such a feature would be a core functionality of a pediatric EHR[24] and would be a future extension of our platform. However, such an integrated decision support functionality could fall under the regulations of the Medical Device Act (German: Medizinproduktegesetz[25]) as well as Directive 95/46/EC,[26] and was thus not yet in the focus of our current research.

To the question “What would be your reasons for nonuse” of an integrated calculator, a physician in private practice answered “False safety,” which is an important point to consider. Doctors and others are using devices and systems that are not transparent from the user's point of view, which means that they have to rely on the system if they intend to use it. In a proven system, doctors could blindly accept dosing recommendations, which in the worst case could have serious consequences for the patient. It should therefore be noted that neither a dosage calculator nor an online platform for dosing recommendations can replace doctors' decisions and knowledge, but can only support them. The doctor is still responsible for the medication, which makes the transparency of dosage calculations so important.

The results showed a demand for advanced functions related to drug safety, such as an interaction check, or information about potential therapeutic alternatives. Such functions require that the necessary knowledge is made available to the platform. However, such extended functions make the validation procedure of the system and its contents more complex. The current PaedDos prototype can be seen as a starting point and might be extended in the future.

Comparison with Other Online Platforms

In the search for comparable systems, two systems were examined in more detail. The “kinderformularium” from the Netherlands[27] is a freely accessible platform for dosing recommendations in pediatrics published in 2008. Like PaedDos, this platform contains information on the level of evidence of dosage recommendations, systematic primary literature research, license status, national trade names, side effects and warning for use in children, contraindications, interactions, and adjustments in renal insufficiency. The information on dosages is not displayed in tabular form, but as a bullet list of text elements. The “kinderformularium” provides an implemented and CE certified calculator, which supports to combine the dosing recommendations with patient–individual variables, thus enabling the calculation of a patient–individual recommendation.[9] The University Children's Hospital Zurich has a Web site “kinderdosierung.ch” with information on dosing recommendations for pediatrics[28] only, but without warnings, side effects, interactions, etc. This platform focuses on inpatient drugs and is in German language like PaedDos. The system provides a calculator, which highlights the appropriate age group in the information list, copies it to the top of the site for easier reading, and calculates the dosing to the manually inserted patient parameters. The page is only accessible to qualified personnel and requires a login.

All three platforms are Web-based, evidence-based, independent of the pharmaceutical industry, and structured according to drug, indication, route of application, age, and weight. However, no scientific publications were found describing technical details and evaluating the development process for “kinderformularium” and “kinderdosierung.ch.”

No studies could be found which address the usability aspects with the SUS of online platforms for dosing recommendations in pediatrics.

Limitations

Three limitations of the study have to be considered. First, the pluralistic walkthrough was conducted with only one representative from every end-user group. Actually, according to Bias, 6 to 10 participants are recommended,[29] but due to immense recruiting difficulties in this project phase (nonavailability of the physicians for the pluralistic walkthrough in addition to their professional activity) and time constraints of the project, this was not possible. For this reason, a sample size of three users representatives of the target audience was chosen, which according to Riihiaho[30] is also sufficient for this method. However, this limitation was compensated by subsequent system iterations and evaluations.

Second, a total of 12 participants of the online test does not represent the entire PaedDos end-user group. Nevertheless, it is assumed that approximately 5 participants are sufficient to identify at least 80% of the major usability problems.[31] [32] [33] [34] In Brooke's retrospective study on SUS,[16] Tullis and Stetson[35] are referenced as having shown that it is possible to get reliable results with samples of 8 to 12 users. In addition, further iterations and evaluations are planned within the UCD, which will later include a larger sample.

Third, the user test has been performed in a test setting and not in a real-world environment. Thus, the extent to which the actual end users are willing or able to use the system in their daily work cannot be anticipated with certainty. The need for an Internet connection or problems with the interoperability of the respective information system of the doctor's practice, pharmacy, or clinic could lead to nonuse of the online platform. This has to be examined in future field tests.

Conclusion

There is a strong demand for an evidence-based pediatric dosage information system among resident doctors, clinicians, and pharmacists, which motivated the development of the PaedDos prototype. Applying an UCD approach during system development resulted in a rating between “good” and “excellent” (mean: 79.5, median: 81.25). Thus, the PaedDos prototype is regarded as an “acceptable system.” The implemented search variants, such as live search, to get quickly to the desired information in different ways, received excellent feedback. The UCD approach proved to be a beneficial tool for optimizing usability. There is a strong demand for patient–individual support functions, in particular a dosage calculator. It has to be investigated in a field trial, similar to the studies described in Ateya et al[36] and Kim et al,[37] whether and to what extent the developed platform prevents dosing errors and in which ways it can support physicians.

Future work will focus on the integration of additional drug information for pediatrics, the development of the responsive design, the creation of an app version for the user interface, and the integration of an online calculator.

Clinical Relevance Statement

As of now, no system providing evidence-based dosing information for children in Germany is publicly available for pediatricians in private practice, clinicians from pediatric hospitals, and pharmacists, who need information on dosages of drugs for children and adolescents. The aim of the PaedDos project was to establish a national Web platform for pediatric drug information. The platform presented in this study is a first prototype that was iteratively evaluated by real end users to provide such information for doctors and pharmacists in their daily work.

Multiple Choice Questions

1. Which is the most explicitly mentioned additional feature for an online platform about dosing recommendations in pediatrics?

   • Interaction check.

   • Dosage calculator.

   • List of alternative therapies.

   • Interoperability to EPRs.

   Correct Answer: The correct answer is option b. In the scope of this study, interaction check and listing of alternative therapies was less named than a dosage calculator. A possible connection between the platform and an internal system or the electronic patient records (EPRs) was mentioned in a conversation with only one participant after the survey.

2. What is the problem with such a state-certified recommendation system?

   • Parents of patients could complain in case of complications.

   • If the system crashes, prescribing is not possible.

   • Doctors could trust the implemented system too much.

   • The doctor will be replaced by an electronic system.

   Correct Answer: The correct answer is option c. Information on this platform is aligned for health care professionals. To avoid misunderstandings of parents with regard to the dosing, the database should be restricted to health care professionals only. In case of a system crash, the doctor can still prescribe medication. A technical tool will never be able to replace a doctor, but is always there to support him. Like a stethoscope, an online reference book could also become a tool in the daily work of a doctor. Doctors still need to keep themselves informed.

Conflict of Interest

None declared.

Acknowledgment

The present work was performed in (partial) fulfillment of the requirements for obtaining the degree “Dr. rer. biol. hum.” from the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) (WR).

Protection of Human and Animal Subjects

Ethical approval was not required.

• References

• 1 de Wildt SN, Tibboel D, Leeder JS. Drug metabolism for the paediatrician. Arch Dis Child 2014; 99 (12) 1137-1142
• 2 Kearns GL, Abdel-Rahman SM, Alander SW, Blowey DL, Leeder JS, Kauffman RE. Developmental pharmacology--drug disposition, action, and therapy in infants and children. N Engl J Med 2003; 349 (12) 1157-1167
• 3 Wimmer S, Rascher W, McCarthy S, Neubert A. The EU paediatric regulation: still a large discrepancy between therapeutic needs and approved paediatric investigation plans. Paediatr Drugs 2014; 16 (05) 397-406
• 4 Knopf H, Wolf I-K, Sarganas G, Zhuang W, Rascher W, Neubert A. Off-label medicine use in children and adolescents: results of a population-based study in Germany. BMC Public Health 2013; 13: 631
• 5 Conroy S, Choonara I, Impicciatore P. , et al; European Network for Drug Investigation in Children. Survey of unlicensed and off label drug use in paediatric wards in European countries. BMJ 2000; 320 (7227): 79-82
• 6 Bücheler R, Meisner C, Kalchthaler B. , et al. “Off-label” Verschreibung von Arzneimitteln in der ambulanten Versorgung von Kindern und Jugendlichen1. Dtsch Med Wochenschr 2002; 127 (48) 2551-2557
• 7 Mühlbauer B, Janhsen K, Pichler J, Schoettler P. Off-label use of prescription drugs in childhood and adolescence: an analysis of prescription patterns in Germany. Dtsch Arztebl Int 2009; 106 (03) 25-31
• 8 Kimland E, Odlind V. Off-label drug use in pediatric patients. Clin Pharmacol Ther 2012; 91 (05) 796-801
• 9 van der Zanden TM, de Wildt SN, Liem Y, Offringa M, de Hoog M. ; Dutch Paediatric Pharmacotherapy Expertise Network NKFK (Nederlands Kenniscentrum voor Farmacotherapie bij Kinderen). Developing a paediatric drug formulary for the Netherlands. Arch Dis Child 2017; 102 (04) 357-361
• 10 Toni I, Wimmer S, Rascher W, Neubert A. AVOID – Kann eine elektronische Patientenkurve mit Dosierungsrechner die Arzneimitteltherapiesicherheit in der Pädiatrie erhöhen?. Arzneiverordnung in der Praxis. 2018; 45: 46
• 11 International Standards Organization. Ergonomics of Human-System Interaction—Part 210: Human Centred Design for Interactive Systems, ISO 9241–210; 2010
• 12 Bhutkar G, Konkani A, Katre D, Ray GG. A review: healthcare usability evaluation methods. Biomed Instrum Technol 2013; ;(Suppl): 45-53
• 13 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
• 14 Shneiderman B. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 3rd ed. Reading, MA: Addison-Wesley; 1998
• 15 Nielsen J. Usability Engineering. Amsterdam: Kaufmann; 2010
• 16 Brooke J. SUS: a retrospective. J Usability Stud 2013; 8: 29-40
• 17 Finstad K. The System Usability Scale and non-native English speakers. J Usability Stud 2006; 1: 185-188
• 18 Brooke J. SUS – a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B. , eds. Usability Evaluation in Industry. Boca Raton, FL: Chapman and Hall/CRC; 1996: 189-194
• 19 Brunner J, Chuang E, Goldzweig C, Cain CL, Sugar C, Yano EM. User-centered design to improve clinical decision support in primary care. Int J Med Inform 2017; 104: 56-64
• 20 Tippey KG, Weinger MB. User-centered design means better patient care. Biomed Instrum Technol 2017; 51 (03) 220-222
• 21 Luna DR, Rizzato Lede DA, Otero CM, Risk MR, González Bernaldo de Quirós F. User-centered design improves the usability of drug-drug interaction alerts: Experimental comparison of interfaces. J Biomed Inform 2017; 66: 204-213
• 22 Schwartz GJ, Feld LG, Langford DJ. A simple estimate of glomerular filtration rate in full-term infants during the first year of life. J Pediatr 1984; 104 (06) 849-854
• 23 Schwartz GJ, Haycock GB, Edelmann Jr CM, Spitzer A. A simple estimate of glomerular filtration rate in children derived from body length and plasma creatinine. Pediatrics 1976; 58 (02) 259-263
• 24 Dufendach KR, Eichenberger JA, McPheeters ML. , et al. Core Functionality in Pediatric Electronic Health Records. Rockville, MD: Agency for Healthcare Research and Quality; 2015
• 25 Medizinproduktegesetz in der Fassung der Bekanntmachung vom 7. August 2002 (BGBl. I S. 3146), das zuletzt durch Artikel 7 des Gesetzes vom 18. Juli 2017 (BGBl. I S. 2757) geändert worden ist
• 26 DIRECTIVE 95/46/EC of the European Parliament and of the Council. Available at: https://eur-lex.europa.eu/legal-content/DE/TXT/?uri=celex%3A31995L0046 . Accessed August 21, 2018
• 27 Nederlands Kenniscentrum voor Farmacotherapie bij Kinderen (NKFK). Available at: https://www.kinderformularium.nl/ . Accessed August 21, 2018
• 28 Universitäts-Kinderspital Zürich. Kinderdosierungen.ch. Available at: https://kispiportal.uzh.ch/kinderdosierungen/D/HOMEpage_Kinderdosierungen_HOME.html . Accessed August 21, 2018
• 29 Bias RG. The pluralistic usability walkthrough: coordinated empathies. In: Nielsen J, Mack RL. , eds. Usability Inspection Methods. New York: John Wiley & Sons; 1994: 63-76
• 30 Riihiaho S. The pluralistic usability walk-through method. Ergon Des 2002; 10: 23-27
• 31 Virzi R. Streamlining the design process: running fewer subjects. In: Proceedings of the Human Factors Society Annual Meeting: SAGE Publications; 1990: 291-294
• 32 Virzi R. Refining the test phase of usability evaluation: how many subjects is enough?. Hum Factors 1992; 34: 457-468
• 33 Nielsen J, Landauer T. A mathematical model of the finding of usability problems. In: Proceedings of the INTERACT'93 and CHI'93 Conference on Human Factors in Computing Systems: ACM; 1993: 206-213
• 34 Nielsen J, Molich R. Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: ACM; 1990: 249-256
• 35 Tullis TS, Stetson JN. A Comparison of Questionnaires for Assessing Website Usability. In: Usability Professional Association Conference. Minnesota, Minneapolis; 2004
• 36 Ateya MB, Aiyagari R, Moran C, Singer K. Insulin bolus calculator in a pediatric hospital. Safety and user perceptions. Appl Clin Inform 2017; 8 (02) 529-540
• 37 Kim GR, Chen AR, Arceci RJ. , et al. Error reduction in pediatric chemotherapy: computerized order entry and failure modes and effects analysis. Arch Pediatr Adolesc Med 2006; 160 (05) 495-498