Usability Evaluation of a Distributed User Interface Application for Visuomotor Organization Assessment

Abstract

Background This article describes the development and evaluation of a distributed user interface (DUI) application to assess visuomotor organization ability. This application enables therapists to evaluate the acquired brain injury (ABI) on patients, and patients, to perform the assessment on a touch screen while therapists can observe the assessment process in real time on a separated monitor without interfering patients during the process as in traditional methodologies employing physical elements.

Objectives The main goal of this research is the evaluation of the quality in use of DUIs in the Pegboard Construction assessment with patients with ABI from the therapist perspective in the area of occupational therapy.

Methods To evaluate our system, we have performed a usability evaluation following the ISO/IEC 25010 and ISO/IEC 25062 standards to evaluate software usability and quality and it was conducted in collaboration with therapists and psychologists that have previously worked with people with ABI in diagnostic and assessment tasks.

Results We show the results of the evaluation collected in a table that shows the completeness rate for each user for both, assisted (i.e., the percentage of tasks where participants performed with test director assistance) and unassisted tasks (i.e., the percentage of tasks where participants completed tasks autonomously), the total time participants required to complete proposed tasks, the number of mistakes participants performed during the session, and the number of assists they required to finish proposed tasks. In addition, we also evaluated the user satisfaction regarding our application using the system usability scale.

Conclusion The use of information technologies in this field enables therapists to perform these evaluations in a simpler, efficient, and automated way. This proposal enables patients to perform the assessment as it is performed traditionally using paper providing them with a touch screen in which they can easily insert a set of pins into the holes. The usability evaluation of the proposal meets the appropriate design standards for applications of this type, and this is demonstrated by the high degree of satisfaction of the participants.

Keywords

cognition - assessment - occupational therapy - acquired brain injury - distributed user interface

Introduction

The purpose of this work is the development and evaluation of the D-Pegboard Construction application. This application employs a distributed user interface (DUI) to support the Pegboard Construction assessment test, which enables therapists to evaluate the degree of acquired brain injury (ABI) of patients. This test is part of the set of the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery tests. In our opinion, information technology must collaborate with medicine, bringing advances and solutions for the diagnosis, evaluation and treatment of patients with ABI while facilitating the work of therapists in this area. ABI involves brain structures in people who, having been born without any type of brain injury, suffer at a later time in their lives, brain injuries that cause an impairment of cognitive, emotional, behavioral, and/or physical functioning. As the statistics reflect, this type of injury represents a serious health problem, mainly due to the large number of people affected (more than 400,000 in Spain), the duration of this type of injury, which is usually chronic, and the severity and the variety of sequels. In addition, it should be noted that ABI represents the main cause of disability in adults in developed countries.[1]

The causes of ABI are diverse.[1] They include cranioencephalic trauma, stroke, anoxia or hypoxia, brain tumors, and encephalitis of various etiologies, among many others. [Fig. 1] shows a series of charts with the incidence and prevalence of ABI in Spain.

Fig. 1 Incidence and prevalence of ABI in Spain. ABI, acquired brain injury.

The ABI can affect all areas of human functioning. The affected area and the deficits presented by the affected person, depend on the type of injury, the initial location, and severity of the injury as well as the characteristics of affected people such as age, personality, or abilities prior to the injury. The sequels are grouped into four dimensions that may, or may not, be present in the same person.[2] These groups of sequels are: physical-motor deficits, cognitive deficits, alterations in communication, and behavioral and/or emotional alterations.

Our solution focuses on the first stage of the brain injury rehabilitation process where the assessment of ABI sequels and the difficulties or deficits it produces are assessed. Thus, we focused on the LOTCA battery as a starting point to develop a digital platform capable of assisting therapists in the diagnosis and assessment of ABI.

The development of this proposal was performed in conjunction with the University of Granada, specifically with the Faculty of Health and followed the model presented in the ISO/IEC 25062 standard for the evaluation of the quality in use of software applications.

The LOTCA[3] battery was developed as a technique to measure basic cognitive abilities and visual perception in adults with neurological disabilities. It provides an in-depth assessment of basic cognitive skills which can be used for treatment planning as well as for treatment progress reviews.[4] The LOTCA battery measures the basic cognitive abilities required for daily functions including: orientation, visual perception and psychomotor skills, problem solving abilities, and thought operations. The development of this battery is based on information obtained from clinical and neuropsychological experience and development theories. Usually, LOTCA is used in the initial stage of the assessment of patients; however, it can be used to establish therapeutic goals and review the cognitive status over time.[5] [6]

Regarding the suitability of individuals, LOTCA can be used with: patients who have had a stroke, cardiovascular accident, elderly individuals with dementia, patients with aphasia,[7] individuals with traumatic brain injuries, and individuals with intellectual disabilities[7] and mental illness[8] can also use it. In addiction an adapted version was developed for children with learning difficulties.[9]

The original version of LOTCA[3] was developed to be used for individuals under 70 with neurological dysfunction and was made up of 20 items grouped into four areas: Orientation, (two items), perception (six items), visual-motor organization (seven items), and thought operations (five items). It was standardized for the Israeli population[5] [10] and it was suitable for use in the populations of the United States.[11] Regarding the assessment time, LOTCA takes approximately 45 minutes, with a range between 30 and 90 minutes.[5] [6]

This work focuses on digitizing a test belonging to the area of visual-motor organization, consisting of reproducing a puzzle. In the test, 15 plastic nails are delivered to the patient jointly with a board having 100 holes. The therapist provides a pattern (a triangle) that the patient should imitate introducing the set of nails into the correct holes in the board. The patient can obtain a score between 1 and 4, depending on whether she/he can imitate the entire pattern without errors, or not.

[Table 1] lists possible scoring for the test.

Table 1 Pegboard test scoring

Cannot reproduce	1
Reproduce vertical and horizontal lines. Cannot reproduce oblique line	2
Reproduces all lines. But location is wrong	3
Good performance. No trial and error	4

Objectives

The main goal of this research is the evaluation of the quality of use of DUIs in the Pegboard Construction assessment with patients having ABI from the therapist's perspective in the area of occupational therapy. To achieve this goal, we set the following objectives:

• Evaluate the impact of DUI paradigm in the diagnosis and assessment of patients with ABI.

• Evaluate the impact of DUI paradigm in the development of D-Pegboard Construction assessment.

• Evaluate the interaction of therapists in the assessment of ABI using DUIs.

Artifact

The software artifact used to perform the Pegboard Construction assessment exploits the DUI environment whose foundations and architecture are explained in this section.

D-Pegboard Assessment Session

This section shows how the D-Pegboard application works. [Fig. 2] shows a sequence diagram of the interactions that occur between the therapist and the patient through the application during an assessment session.

Fig. 2 Sequence diagram of our application for a real use case without error.

The application UI is divided into two parts according to user's roles (i.e., patient and therapist). On the one hand, the patient UI enables patients to perform assessment actions. On the other hand, the therapist UI enables therapist to control assessment procedure as well as patient's assessment development.

Therapist UI

From the therapist point of view, they access the application and selects their role (i.e., “I am the therapist”). Once selected, therapist's UI is displayed. This UI enables therapists to a new session or enables him to access previous session information. When creating a new session, they can type, or use the automatically generated identification for the session, which is used to restore session information afterward. Finally, they introduce therapist and patient names.

Once a session is created, the UI displayed in [Fig. 3] is presented to therapists. This UI is divided into two parts: the workspace UI (on the left) and the session control UI (on the right).

Fig. 3 The therapist UI. UI, user interface.

Workspace The workspace UI is empty while patients are not logged into the system. As soon they log into the system, the UI shows two pegboards. While the figure to be reproduced by the patient is shown on the left; the figure that is under construction by the patient is show on the right. Therefore, as patients tap on the touch screen, the pins that are introduced and removed from the pegboard are displayed in the therapist's display in real time too.

Session control Session control UI enables therapists to start/end sessions, introduce session notes, and control assessment session times using a digital chronometer.

Once the session is finished, statistical information such as time of completion of the session, punctuation, observations, and so on, is stored into our system to be retrieved by the therapist at any point of time.

Patient UI

Thus, UI supports patient activities during D-Pegboard assessment sessions. When therapists create sessions, they set up the patient UI for the target patient. To perform this task, they select the “I am the patient” option from the application main menu. Then, they introduced the session identifier they generated while creating the session. Once this information is introduced, patients are able to interact with the system using the UI shown in [Fig. 4].

Fig. 4 The patient interface.

This UI shows two boards. While the target pattern to be reproduced by the patient is shown on the right, an “empty” pegboard is shown on the left. To interact with the digital pegboard, patients should tap on a black circle to “insert” a pin into a hole. Consequently, the circle becomes yellow to represent that a pin was “inserted” into a hole. They can also “remove” a pin from the hole tapping on the yellow hole. Consequently, it changes to black again to show that the hole is empty.

Distributed User Interface

The software solution proposed in this article uses DUI paradigm to enable patients and therapist interact with each other. A DUI is defined as user interface (UI) whose components are distributed through one or more dimensions of input, output, platform, space, and time,[12] where:

• Input refers to administration of input in a single computational device or distributed through several different devices.

• Output is the graphic output linked to a single device (screen) or distributed through several devices (called screens or content redirectors).

• Platform is the interface run on a single computational platform or distributed across several platforms (architectures, systems operatives, networks, etc.).

• Space is the interface restricted to the same physical space (and geographic), or it may be geographically distributed.

• Time is the set of elements of the interface executed simultaneously (synchronous) or distributed over time (asynchronous).

In Penãlver et al,[13] authors specify a set of essential properties of DUIs: portability, simultaneity, continuity, and decomposability.

The DUI concept is associated with the following set of properties: portability, decomposability, simultaneity, continuity, multiplatform, multimonitor, multiuser, consistency, flexibility, and efficiency.

Some authors have recently revisited the concept of DUI, explaining that the adjective “distributed” denotes only a subset of applications that distributes system UIs. Therefore, they redefine the DUI concept as distributable user interface (DeUI) to include both, DUI systems covering UI that are distributed but cannot be distributed on run-time, and DeUI systems which enable the UI distribution at run-time.[14] [15]

As we have seen in the previous section, our application uses the paradigm of DUIs to facilitate communication between the patient and the therapist. When the test is performed in a traditional way, following the documentation and using the devices proposed in the LOTCA battery, the therapist must use a chronometer to measure the time, pencil, and paper to write down the observations and to write the test results, and he must provide the patient with a board and a series of nails to reproduce the given pattern.

Thanks to the use of DUIs, our application allows the patient and therapist to work simultaneously, each one on a different screen, saving us from using the rest of the devices necessary for the test. The application is responsible for storing all the information related to the session and allows the therapist to focus on observing how the patient performs the test to facilitate and improve their diagnosis. In addition, it allows this observation to be less invasive since it is not necessary even for them to be in the same physical space.

Related Work

As Elmqvist,[16] Vandervelpen,[17] and other researchers state, we can find numerous literary references related to UIs, such as in Grolaux et al and Bandelloni,[18] [19] among others. Additionally, we can find recent literary references to the concept of DUI.[17] [20] [21]

Penitchet et al gathered all their data and studies[22] [23] [24] and numerous literary references to different types of configurations, ecosystems, and different definitions that exist around DUI.

Within the different types of interaction that technology allows us today, as radiofrequency identification, which enables storing and remotely retrieving data, which allow the identification of an object from the distance or near field communication, and standardizes the way in which smartphones and other mobile devices establish radio communication,[22] the one that interests us and we have used in this work is WebSocket. WebSocket is a type of web communication which establish a series of protocols and standards to exchange data between applications. This type of protocol provides a full-duplex communication channels through a single Transmission Control Protocol connection. As indicated by Penichet et al, it can be used by any client or server application providing real-time interaction through the web.[22] Our work in this article does not consist of collecting more literary references or expanding the existing literature, but in implementing a novel environment of real DUI applied to the occupational therapy sector, based on a test that is traditionally performed using a paper and pen.

D-Pegboard Application Architecture

This section describes the architecture developed to support the D-Pegboard application. The architecture is based on a client-server approach in which therapists create a personalized session for patients including personal data of both patients and therapists.

The server part of the application acts as a broker which synchronizes patient's and therapist's UIs. Besides, it is in charge of managing session information as well as controlling the assessment session. It is implemented using Node.js[1] and operated through a REST[2] service Application Programming Interface which is connected to a NoSQL (no structured query language) database supported by MongoDB[3] where session information is stored. The server is not only responsible for collecting session information introduced by the therapist; it also collects all patient actions by the means of WebSocket to reproduce patient actions on therapist UI in real time. In addition, the server is also responsible for maintaining all the information into the database to provide access to therapists afterward ([Fig. 5]).

Fig. 5 View of the architecture of the application.

The main reason for choosing JavaScript, and specifically, Node.js, to implement the application server is the high performance provided by this technology in service-based applications. Node.js uses an event loop (event loop), which manages all asynchronous operations. In case of needing a blocking operation, an asynchronous request is sent to that Event Loop together with a callback, and the server proceeds with the request when no operations are pending. Therefore, Node.js creates a new event inside the main loop of Node.js for every new connection to the server. If we compare this approach to other technologies such as PHP or Java, where every connection to the server generates a new thread associated with its corresponding memory consumption, the resource consumption, in this case, is significantly lower. This translates into great memory savings and therefore, supports for a larger number of simultaneous requests using fewer resources.

The justification for choosing an NoSQL database, like MongoDB, is fundamentally due to its advantages like better performance, high scalability with respect to the SQL database. The deployed version of the application does not work with sensitive data about patients or their medical files, which according to the European legislation are considered specially protected data. Therefore, the solution adopted does not require extremely restrictive measures regarding the storage and processing of data.

Life Cycle Activities

To develop D-Pegboard, two complementary development methodologies have been chosen: SCRUM and the Usability Engineering Lifecycle. SCRUM has guided the development process, following the phases and workflows proposed by the methodology, and in turn, in these phases and flows we include some activities proposed by the life cycle. In this article, we are going to focus on activities related to the life cycle, since they have a greater relationship with human computer interaction. Usability engineering, according to Deborah Mayhew in her book “The Usability Engineering Lifecycle,”[25] is a discipline that provides a set of structured methods for achieving usability in UI design during software product development.

When we talk about usability, we refer to a measurable characteristic of a product that is present to a greater or lesser degree. Specifically, this feature measures how easy it is to learn and use the product or how easy it is to operate it. On the other hand, when we refer to a UI, we refer to the language through which the user and the product communicates with each other.

This faculty is responsible for contacting various associations of the city that works with patients associated with ABI. Specifically, we worked with a small group of users who were in a mild phase of their illness, in addition to the professionals who worked in the center; among them are psychologists, therapists, nursing assistants, etc.

In the early stages of the development of our solution, we focus on gathering requirements as well as identifying potential users of D-Pegboard. Activities such as the contextual analysis of tasks or the profile of users were performed. The contextual analysis of the tasks is done to find out what tasks are performed, what tasks are performed to carry out the ABI evaluation in a traditional way, identifying the steps they take and help they may need to do it. The purpose of the methodology followed to obtain this information was to meet with the professionals in an interview, who provided us with a series of lists in which they specified the tests that make up the battery of LOTCA and how they are performed.

In the later stages of the development of our application, we focus on the analysis, design, and implementation of the different use cases identified as a result of the requirements previously obtained. The life cycle activities that were performed included prototyping at each iteration ([Fig. 6]), which was tested by the actual users. These users were asked to perform a series of tasks with the prototypes provided, and once it was verified that they would be able to use them, the UI was definitively implemented.

Fig. 6 Example of prototype of the interface made in paper.

If a problem arose during the test, it was reflected in the results and the test proceeded to its solution for the next iteration. In this way, in each iteration the functionalities that had been added along with those that had been tested and corrected (if applicable) previously were tested in each iteration. In the final phase of the development of D-Pegboard, life cycle activities focused on evaluating the β version of the application, as detailed in the evaluation section.

Another important aspect to consider in the design of the interfaces is that it should be in accordance with each cultural environment and be designed in such a way that it adapts to the characteristics of each region, unlike nowadays where people are getting affected by the computing environment.[26] As Masip RG[27] indicates, on one hand there are the so-called cultures of high context (HC) like Japan and China, in which the largest part of the information is in context and is not necessary to transmit it. On the other hand, exist cultures of low context, like Germany, that need all the explicit information to avoid possible distortions in its meaning.

In our case, we consider that our tool should not present too many difficulties based on the type of culture, because it is not a tool that is responsible for transmitting information or intends to sell a product to a specific group of public. Our tool has a simple interface with just four to five buttons, which shows a board with holes that is a replica of the board that is shown in the traditional LOTCA test. This traditional test is standardized and adequate to meet the requirements of diverse cultures, such as the EEUU population[11] or the Israel population,[5] [10] as we said before. We believe that our tool faithfully represents the traditional test and therefore, if the traditional test has been evaluated and validated in diverse cultures, it should also be able to do so with our tool.

Methods

This section presents the usability and quality evaluation of the D-Pegboard application that was conducted once the implementation of the functional β version of the system was complete. The evaluation follows the ISO/IEC 25010 and 25062 standards[28] [29] to evaluate software usability and quality and it was conducted in collaboration with therapists and psychologists who have previously worked with people with ABI in diagnostic and assessment tasks.

Objective

The goal of the evaluation is the assessment of the quality in use of the therapist UI of the D-Pegboard application. It is particularly focused on the following aspects:

• Comprehensibility The therapists consider that the software is adequate and they understand how they can use it to carry out the evaluation of patients with ABI, adapting the application to the particular conditions of each patient.

• Ease of learning Therapists are able to learn how to use the application quickly and easily.

• Operability The application shows a simple and intuitive UI that enables users to carry out the proposed tasks. These users encompass both, experienced users as well as users without experience.

• Attractiveness The therapists consider that the UI is aesthetically attractive.

Participants

To carry out the test, 10 participants were selected, and none of them were involved in the development of the application. These participants are therapists and computer science professionals who have worked with the battery of the commercial version of the LOTCA exercises and other similar techniques used in ABI evaluation. [Table 2] shows a summary of selected participant profiles.

Table 2 Profile of evaluation participants

Participant	IT skill	D-Pegboard	ABI apps	Age
1	2	1	28
2	3	1	32
3	1	23
4	3	1	23
5	3	24
6	3	1	2	55
7	2	1	2	46
8	2	1	3	25
9	1	2	3	42
10	1	2	3	52

Experience: 1, Low; 2, Middle; 3, High.

Context of Use

Information about the tasks that users should perform is collected in the sheet that was provided to them when carrying out the test. All the participants had to perform four basic tasks:

1. Create a session with the ID provided.

2. Carry out a complete assessment with a nonreal patient using the D-Pegboard application.

3. Evaluate patient results and end the session.

4. Visualize the results of the assessment.

All proposed tasks are the same as users should carry out with the application on a daily basis. We can associate these evaluation tasks with the different interactions that the therapists perform with our application in a real use scenario, as represented in the sequence diagram of the [Fig. 2]

• The first task corresponds to the “create session” action of the sequence diagram.

• The second task corresponds to the “start timer” and “stop timer” actions in the diagram, with the corresponding response from the patient.

• The third task corresponds to the actions “evaluate session” and “end session” of the diagram.

• The fourth task corresponds to the actions “visualize session summary” of the diagram.

The information given to evaluation participants indicate the set of actions they should perform; however, it does not include how they should perform these tasks. Therefore, participants are provided with a visual preview of different interactions that should help them to use the application.

As per the compliance criteria, it was considered that users had finished the test when they had completed the assessment session and accessed the session results.

Metrics

The following set of metrics are used to perform the application evaluation:

Effectiveness is related to the level of accuracy and integrity achieved with respect to product objectives. This metric is expressed in terms of:

• Completion rate Percentage of tasks completed correctly with respect to the total number of tasks proposed in the test.

• Errors Number of times the participant did not complete the task successfully, or she/he had to repeat the parts of the task more than once.

• Assistance Number of times participants could not perform the task, or the administrator of the test had to intervene directly by giving information to the participant to complete the task.

• Efficiency is related to the level of effectiveness achieved with respect to the amount of resources required to perform a task. It is measured as the average time participants took to complete a task.

• Satisfaction It describes the subjective response of the user when using the product. This satisfaction usually affects user's motivation to use the product and sometimes users' performance. To measure user's satisfaction, we provided participants with questionnaires that use Likert scales. It is a system usability scale (SUS) questionnaire[30] consisting in ten questions to generate a unique number that represents a composite measure of the usability of the global system under study.

Ethical Considerations

The current study was approved by the Research Ethics Committee of the Granada University with code number 661/CEIH/2018 and was in accordance with the Declaration of Helsinki. Before the beginning of every assessment, informed consent was obtained from all the participants or their legal guardians, and they were informed that their cognitive abilities would be tested.

Results

Test results are presented in [Table 3]. This Table depicts the rate of completeness of the task by each user for both, assisted (i.e., the percentage of tasks performed using director's assistance) and unassisted(i.e., the percentage of tasks where participants completed the task autonomously) groups, the total time participants required to complete the proposed task, number of mistakes the participants committed during the session, and the number of assists they required to finish the proposed task.

Table 3 Global results

Participant	Completeness rate (%)		Task time (mm:ss)	Errors	Assists
	Assisted	Unassisted
1	25	75	03:26	1
2	0	100	03:30	0
3	0	100	03:07	0
4	25	75	01:49	0	1
5	0	100	01:56	0
6	25	75	03:42	0
7	0	100	04:21	1	0
8	0	100	03:34	0	1
9	0	100	02:12	0
10	0	100	02:55	0
Mean	7.5	92.5	02:46	0.2	0.3
Min.	0	75	01:49	0
Max.	25	100	04:21	1

The result analysis is performed according to the following criteria:

• Data classification: User behavior is classified in terms of the number of actions performed: successfully, with errors, and with verbal assists. It is considered an error when the patient is not able to perform the task alone and performs an action that does not lead to the completeness of the task. Likewise, each intervention, to indicate how to continue the task by the evaluator, is considered as verbal assistance.

• Data reduction: Each task is analyzed independently.

• Statistical analysis: The average, minimum, and maximum values are used as statistical measures.

As we have indicated previously, to evaluate the satisfaction of the users we performed a satisfaction questionnaire (SUS) whose results are shown in [Table 4].

Table 4 SUS results

Participant	SUS score
1	90
2	82.5
3	97.5
4	95
5	97.5
6	85
7	90
8	92.5
9	100
10	92.5
Mean	92.25

Abbreviation: SUS, system usability scale.

Discussion

As we have seen in the [Table 3], the majority of participants in the test were able to complete the tasks autonomously. Only three participants required verbal assistance. In all the three cases, this assistance corresponded to the same design flaw, which has already been resolved. The ruling was that on the screen in which the therapist was supposed to enter the session data, an ID for the session was already provided automatically, and seeing that field filled in, the participants did not know where to enter the ID they had been provided in the test form. To avoid this, the title of each field was added next to the form, instead of appearing inside the input field.

When we look at the completeness rate in [Table 3], we observe that no participant performed less than 75% of the tasks autonomously. In other words, participants who needed assistance only required it for 25% of the tasks and none of them had any previous experience with this application.

Regarding the number of errors, there was only one participant who committed an error in the test. The error occurred while saving the evaluation data. The participant did not click on the save button in the form. She/he accidentally clicked outside the form. As it is a modal window, it hides when clicking on the outside.

Consequently, the participant did not realize that it had not been saved correctly. This behavior was corrected, and feedback was provided when action taken.

We can also observe that the results of the test are similar in both, participants with skills computer science and without experience in this field. It means that our application is easy to learn and use due to the development of a UI based on quality standards that make the interface friendly and intuitive.

[Table 4] reveals that satisfaction questionnaire results are exceptionally good. The average score obtained is 92.25 out of 100. Main reason behind this result is the fact that participants found the application “not very complex, well integrated, quite comfortable to use, and easy to learn and use.”

Conclusions

This work focuses on the lack of technology support for ABI assessments that are currently performed on paper. Under this situation, data collection and analysis is quite limited. The used of information technologies in this field enables therapists to perform these evaluations in a simpler, efficient, and automated way.

This proposal implements a set of tests proposed in the LOTCA battery test for the diagnosis and assessment of people with ABI. It enables patients to perform the assessment as it is performed traditionally using paper, providing them with a touch screen in which they can easily insert a set of pins into the holes.

The application UI implements a DUI paradigm that enables patients and therapists to work together and in real time avoiding the intrusion of the therapist in the assessment during the session. However, the therapist can observe patient's assessment development from a DUI without even sharing the same physical space. This solution not only enables therapists to visualize patient actions directly in real time, but it also enables therapists to share this session information with colleges in real time too. In addition, this technology facilitates the development of the test, reducing the number of artifacts needed to perform the assessment (e.g., chronometer, paper, pen, board, nails, etc.) to only two devices. As all assessment session information is digital, it can be retrieved from the application by the therapists afterward.

The realization of the activities proposed by the life cycle of usability engineering, together with the collaboration of the professionals and patients of the University of Granada, has allowed us to develop an application that satisfies the real needs of a certain collective users using UIs, designed especially for them, thinking about their physical and mental limitations.

The usability evaluation of the proposal meets the appropriate design standards for applications of this type, and this is demonstrated by the high degree of satisfaction of the participants (92.25 out of 100) and the low number of errors and assists that occurred during evaluation tests.

As future work, we believe that this proposal can be extended to cover most of the tasks included in the LOTCA battery test. Thus, laborious traditional tests could replace digitized tests taking advantage of analogous benefits obtained by the digitalization of the Pegboard construction test. Finally, we are currently working on the validation of the patient UI with respect to the traditional version of the Pegboard construction test. Thus, the correlation between both approaches can be obtained.

Therefore, it would be very interesting to use the new digital system proposed in different brain injury diagnostic centres, increasing the sample of patients, thus being able to detract possible errors and solve them. For this, it is seriously necessary that you begin to suffer injuries in hospitals and diagnostic centers of this type of brain lesions, thus increasing the database of our system and allowing to solve the errors that appear.

We believe that the use of technology applied to the medical sector is a subject of great interest today, and that it will have an even greater boom in the future, as evidenced by the main lines of research in Europe. Therefore, we believe that this tool can be extremely useful in the future, if it is complemented with the rest of the tests that make up the LOTCA battery.

Conflict of Interest

C. C. reports and the application D-Pegboard have been developed with the consent of the original authors of the LOTCA test, where the Pegboard subtest is included.

¹ https://nodejs.org/es/.

² Representational State Transfer.

³ https://www.mongodb.com/.

• References

• 1 Navarra ACIA. Available at: https://www.adacen.org/blog/index.php/que-esel-dano-cerebral-y-cuales-son-sus-secuelas/ . Accessed November 9, 2018
• 2 Cerebral Injury HHRM. Available at: http://xn–daocerebral-2db.es/papelde-la-terapia-ocupacional-en-la-rehabilitacion-del-dano-cerebral/ . Accessed November 12, 2018
• 3 Katz N, Itzkovich M, Averbuch S, Elazar B. Loewenstein Occupational Therapy Cognitive Assessment (LOTCA) battery for brain-injured patients: reliability and validity. Am J Occup Ther 1989; 43 (03) 184-192
• 4 McDermott A. Purpose of measure. In: Nicol Korner-Bitensky, ed. Loewenstein Occupational Theraphy Cognitive Assessment. Ottawa: Heart and Stroke Foundation; 2011
• 5 Annes GCS. Comparison of younger and older healthy American adults on the Loewenstein Occupational Therapy Cognitive Assessment. Occup Ther Int 1996; 3: 157-173
• 6 Zwecker M, Levenkrohn S, Fleisig Y, Zeilig G, Ohry A, Adunsky A. Mini-mental state examination, cognitive FIM instrument, and the Loewenstein Occupational Therapy Cognitive Assessment: relation to functional outcome of stroke patients. Arch Phys Med Rehabil 2002; 83 (03) 342-345
• 7 Jang Y, Chern JS, Lin KC. Validity of the Loewenstein occupational therapy cognitive assessment in people with intellectual disabilities. Am J Occup Ther 2009; 63 (04) 414-422
• 8 Josman N, Katz N. Relationships of categorization on tests and daily tasks in patients with schizophrenia, post-stroke patients and healthy controls. Psychiatry Res 2006; 141 (01) 15-28
• 9 Josman N, Abdallah TM, Engel-Yeger B. Cultural factors affecting the differential performance of Israeli and Palestinian children on the Loewenstein Occupational Therapy Cognitive Assessment. Res Dev Disabil 2010; 31 (03) 656-663
• 10 Cermak SA, Katz N, McGuire E, Greenbaum S, Peralta C, Maser-Flanagan V. Performance of Americans and Israelis with cerebrovascular accident on the Loewenstein Occupational Therapy Cognitive Assessment (LOTCA). Am J Occup Ther 1995; 49 (06) 500-506
• 11 Katz N, Champagne D, Cermak S. Comparison of the performance of younger and older adults on three versions of a puzzle reproduction task. Am J Occup Ther 1997; 51 (07) 562-568
• 12 Melchior J, Vanderdonckt J, Van Roy P. A model-based approach for distributed user interfaces. Paper presented at: Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems. Pisa, Italy; 2011: 11-20
• 13 Penãlver A, L'opez-Esp'ın JJ, Gallud JA, Lazcorreta E, Botella F. Distributed User Interfaces: Specification of Essential Properties in Distributed User Interfaces. Designing Interfaces for the Distributed Ecosystem. London: Springer; 2011: 13-21
• 14 Tesoriero R, Altalhi AH. Model-based development of distributable user interfaces. Univ Access Inf Soc 2019; 18: 719-746
• 15 Villanueva PG, Tesoriero R, Gallud JA. Revisiting the concept of distributed user interfaces. 2013: 1-15 . DOI: 10.1007/978-1-4471-5499-0_1
• 16 Elmqvist N. Distributed user interfaces: state of the art. Paper presented at: Workshop on Distributed User Interfaces 2011 (DUI) at the 29th ACM CHI Conference on Human Factors in Computing Systems; May 7, 2011; Vancouver
• 17 Vandervelpen C, Vanderhulst K, Coninx K. Light-weight distributed web interfaces: preparing the web for heterogeneous environments. Paper presented at: Proceeding of 5th Int. Conf. on Web Engineering ICWE'2005. Sydney, Australia; 2005
• 18 Grolaux D, Roy PV, Vanderdonckt J. Migratable user interfaces: beyond migratory interfaces. Paper presented at: Proceedings of the IEEE/ACM Conference on Mobile and Ubiquitous Systems; August, 2004
• 19 Bandelloni F. Flexible interface migration. Paper presented at: the ACM Conference on Intelligent User Interfaces; January, 2004. Boston, Massachusetts, United States; Funchal
• 20 Demeure J, Sottet S, Calvary G, Coutaz V, Vanderdonckt J. The 4C reference model for distributed user interfaces. Paper presented at: Proceedings of the International Conference on Autonomic and Autonomous Systems; 2008; NW Washington, DC
• 21 Melchior J, Grolaux D, Vanderdonckt J, Roy PV. A toolkit for peer-to-peer distributed user interfaces: concepts, implementation, and applications. Paper presented at: Proceedings of the ACM Symposium on Engineering Interactive Computing System; July, 2009; Pittsburgh, PA
• 22 Penichet VMR, Lozano MD, Gallud JA, Tesoriero R. 12+1 Questions in the Design of Distributed User Interfaces. Paper presented at: Proceedings of the 2014 Workshop on Distributed User Interfaces and Multimodal Interaction; July 1, 2014; Toulouse
• 23 Gallud JA, Penichet VMR. Distributed user interfaces: distributing interactions to facilitate universal access. Univ Access Inf Soc 2019; 18 (04) 717-718
• 24 Marco FA, Gallud JA, Penichet VMR. A study on how users perceive distributed interactions on web applications. IEEE Access 2018; 6: 57389-57399
• 25 Mayhew DJ. The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design. San Francisco; CA: Morgan Kaufmann Publishers; 1999
• 26 Collazos CA, Gil R. Using cross-cultural features in web design patterns. Paper presented at: Proceedings of the 2011 Eighth International Conference on Information Technology: New Generations (USA) IEEE Computer Society. Las Vegas, Nevada, United States; 2011
• 27 Llu'cia Masip RG, Granollers T, Collazos CA. Multiculturality and internationalization in Web interfaces. Rev Adv Sistemas Inform 2009; 6: 192-196
• 28 ISO/IEC 25010. 2011 Systems and software engineering—Systems and software Quality Requirements and Evaluation (SQuaRE) —systems and software quality models; 2013
• 29 Software Engineering—Software Product-Quality Requirements and Evaluation (SQuarE)—Common Industry Format (CIF) for Usability Test Reports. International Organization for Standardization; 2006
• 30 Brooke J. SUS: a “quick and dirty” usability scale. In: Usability Evaluation in Industry. CRC Press; 1996: 189-194

Publication History

Received: 05 November 2019

Accepted: 02 March 2020

Article published online:
07 September 2020