Keywords
nursing - information security - knowledge - attitude and practice - validation - reliability
Background and Significance
Background and Significance
Information security comprises three elements: confidentiality, integrity, and availability.[1] The lack of adequate regulations in Chinese medical institutions regarding the protection of clinical health data poses challenges to nursing information security. One case occurred in 2020, where two obstetrical nurses in a Chinese hospital illegally leaked thousands of maternal staff information, resulting in frequent harassment of maternal staff via text messages and phone calls.[2] In addition, with the huge scale of health big data of “Internet + nursing service,” the privacy and medical information contained in the data about nurses and patients may be illegally obtained if a cyberattack occurs.[3] These breaches inflict the confidentiality, and availability of information is compromised. According to surveys, 96.64% of nurses believe that emergency drills are very helpful for dealing with network and information security emergencies or major software and hardware failures. Such drills can improve the emergency response speed and decision-making ability of clinical nurses for nursing information, to maintain the integrity of data.[4]
Threats to the safety of health care data need much more focused attention than they have received in the past.[5] As such, ensuring the security of health information has emerged as a significant challenge.[6]
[7]
[8] The awareness of information security among nurses in China is generally inadequate. Due to the shortage of nurses and inadequate nursing information systems, Chinese nurses often resort to using unauthorized social software such as WeChat, DingDing, and email for disseminating patient information and shift updates. However, they may not be aware of the potential impact on information security. And there is a lack of standardized evaluation tools for nurse information security in nursing practice. The nursing sector necessitates the establishment of a robust culture surrounding information security. Dr. David Blumenthal highlighted that the security of health care information requires joint efforts of medical institutions and health care staff.[5]
[9] Similarly, for care information security, in addition to actions at the level of a health care institution or organization regarding the hospital information system, as the primary provider of care service, nurses' adherence to information security protocols assumes a pivotal role in upholding overall information security.[10]
[11] Therefore, maintaining the security of nursing information also requires the efforts of the nurse community. The security of nursing information is pivotal in preventing incidents such as data deletion, tampering, and unauthorized access within information systems, which directly affect the quality and effectiveness of nursing work.[12] However, there is no uniform definition of nursing information security. Kang and Seomun[13] conducted a conceptual analysis of nursing information security in 2021, and defined nursing information security as “attitudes of nurses to deepen their awareness of the importance of medical information and to prevent information disclosure by considering technical, physical, and administrative aspects.” It only emphasizes the prevention of information leakage but this is not comprehensive. The utilization of specific instruments can be employed to assess the extent of information security among nurses. Kang and Seomun[10] compiled a nurses' information security attitude questionnaire. This particular questionnaire can be utilized for the purpose of investigating the attitudes held by nurses. This questionnaire was not developed by following a thorough definition. Magdalinou et al[14] surveyed 165 nurses in Greek hospitals using an adapted version of the Human Aspects of Information Security Questionnaire (HAIS-Q).[15] The research did not have a well-developed theoretical framework, and the questionnaire was not specific to nurses. The field of nursing information security in China is still in its early stages, primarily concerning nurses' attitudes towards information security and their level of knowledge, with a lack of dedicated assessment tools for nurses.
The security of nursing information plays a crucial role in mitigating nurse–patient disputes and enhancing nurses' operational efficiency and the quality of care they provide. Currently, however, there is no mature concept and suitable theoretical framework to guide the construction of instruments. Moreover, there is currently an absence of a reliable and effective tool for assessing the level of information security in nursing care. Therefore, this study is purposed to construct and evaluate a self-reporting tool for nursing information security from the perspective of nurses and enhance nursing information security behavior.
Methods
This study covered four phases : conceptualization, item pool development, development of preliminary questionnaire, and development of the nursing information security questionnaire (NIS-Q). The paper “Essential elements of questionnaire design and development” provides a logical, systematic, and structured approach to questionnaire design and development, and explores the process by which a reliable and valid questionnaire can be developed.[16] We followed these approaches with a view to develop the present instrument.[8] To assure content validity, the research team followed the research guidelines for the Delphi survey technique and carried out two rounds of Delphi expert consultation.[17]
[Fig. 1] shows the flowchart of this study.
Fig. 1 Flowchart of this study.
Study Design and Participants
According to the feedback received from the experts in the Delphi phase,[18] the research team formulated a preliminary questionnaire of the Nursing Information Security. Then, the reliability and validity of the preliminary questionnaire were evaluated based on a cross-sectional survey.
The sample size had to be at least five times the number of items, which, in this case, amounted to 200.[19] Since the preliminary questionnaire consisted of 40 items, and considering an estimated 10% attrition rate, the aim was to recruit a minimum of 220 participants. Moreover, an additional minimum of 200 cases was necessary for conducting confirmatory factor analysis (CFA). Therefore, the participants were randomly divided into two halves, with Sample 1 being used for item analysis, exploratory factor analysis (EFA), and analysis of internal consistency, and Sample 2 being used for CFA, convergent and discriminative validity. Participants recruited via convenience sampling had to meet the following inclusion criteria: (1) nurses with a professional qualification certificate in nursing; (2) over 12 months of experience in a tertiary general hospital; and (3) informed consent and voluntary participation. In addition, off-duty nurses were excluded.
Data Collection
The formal recruitment began in June 2023 and included online and offline recruitment. The research team provided access to the electronic questionnaire as well as a paper version for participants to complete. In addition to the investigator's hospital, the participants were sampled from five tertiary general hospitals in China, and hospitals were selected based on convenience sampling. The directors of nursing departments or head nurses from these five hospitals were contacted beforehand via WeChat or email. This communication aimed to elucidate the study's objectives and intentions. The directors would send the link to the questionnaire in the relevant work groups, and interested nurses would volunteer for the test. To ensure accuracy, each internet protocol (IP) address was limited to submitting responses only once. As for the offline questionnaires, the research team visited various departments of their own hospital to distribute paper questionnaires and explain the study objective, and nurses participated voluntarily. Both the online and offline questionnaires included 9 general information questions and 40 items of preliminary NIS-Q.
Conceptualization
The initial step of developing a new questionnaire is to define the concept to be measured.[20] Nursing information refers to all the information related to nursing,[21] for example, patients' health information, patients' privacy, nursing records, and other work information. Privacy is patients' right to have full control of their data.[22] Confidentiality and integrity of patient information are crucial to ensuring that patients' privacy is respected and high-quality care is provided.[21] As for sensitive clinical data, professionals who have access to patient records have an ethical/legal obligation to hold that information in confidence.[22]
[23] The three pillars that uphold the security of protected health information as outlined by the Health Insurance Portability and Accountability Act (HIPAA) are access, administrative, and physical safeguards.[24] On this basis, Kang and Seomun[13] conducted a concept analysis, the definition of information security in nursing was derived as follows: “Attitudes of nurses to deepen their awareness of the importance of medical information and to prevent information disclosure by considering technical, physical, and administrative aspects.” A total of seven attributes of information security in nursing were derived. The physical security aspect included two types: “facility stability” and “environmental control.” Technical security has two types: “information accessibility” and “taking advantage of features.” The administrative security aspect consisted of three types: “systematicity of work,” “execution of education,” and “professional responsibility.” Ge et al[25] took the ability to protect privacy and information security as one of the training goals in the master of nursing specialist in the field of health informatics. The specific requirements were to train students to understand and comply with relevant laws, regulations, and ethical guidelines, and to protect patients' privacy and information security. Because this study is purposed to develop a self-reporting tool for nurses; based on the above, the nursing information security in this study is defined as nurses understanding relevant laws and regulations, abiding by professional ethics, keeping work information confidential, protecting patients' privacy, and preventing information disclosure by considering access, technical, physical, and administrative aspects.
Item Pool Development
The item pool is constructed mainly through a literature review and the research team's discussion. The research team discussed and identified search terms, such as “nurse,” “nursing,” “information security,” “informatization,” and “information system,” and the relevant literature was reviewed and analyzed.[13]
[14]
[26]
[27]
[28]
[29] As found in the literature review, a positive correlation exists between nurses' practice behavior in information security and their knowledge and attitude towards information security.[14] Therefore, we want to comprehensively evaluate nurses' knowledge, attitude, and practice. In the setting of the questionnaire's dimensions, it was framed by the knowledge–attitude–practice theory, including three dimensions: knowledge, attitude, and practice. For specific items in each dimension, based on a conceptual analysis of nursing information security, we created the items. In other words, the content of each item maps certain connotations of the conceptual analysis. Furthermore, we referred to the information security attitude questionnaire (ISA-Q), as well as other instruments that are used to investigate nursing information security attitudes.[10]
[14] This approach produced an initial pool of 51 items (17 items in each dimension) rated using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).
Expert Consultation
Expert consultation questionnaires were prepared based on the item pool. The Delphi method was used for two rounds of expert letter consultation. The expert selection criteria were as follows: (1) experts in the fields of nursing management, nursing or nursing informatics; (2) associate senior title or above; (3) over 10 years of professional experience; (4) bachelor's degree or above; and (5) voluntary participation. The scale items were assigned using a 5-point Likert scale, with a score from 1 to 5 indicating “very unimportant” to “very important.” The items were selected according to the criteria of importance mean ≥0.4 and coefficient of variation <0.25.[30] The experts were also asked to point out and comment on poor wording or redundancies in the proposed questions and suggest missing questions. Subsequently, the items were revised according to the collected experts' suggestions and scores.
A preliminary questionnaire was determined, and each item could be scored by means of the 5-point Likert method. A higher score indicated a higher level of nursing information security knowledge attitude and practice.
Pilot Test
Before the formal recruitment, a preliminary survey was conducted using convenience sampling, involving 30 registered nurses employed at a tertiary general hospital for a minimum of 12 months. The aim of the presurvey was to gather their responses and feedback before the formal assessment. This confirmed the accuracy, readability, and ambiguity of the preliminary questionnaire.[8]
Item Analysis
Items were screened using the classical test theory,[31] specifically employing the critical ratio method, correlation coefficient method, and Cronbach's α coefficient method. The cumulative scores from the scale were sorted in descending order. The association between the high-scoring group (top 27%) and the low-scoring group (bottom 27%) was examined. Subsequently, each individual item's mean was compared to determine the extent of discrimination within the questionnaire. If the correlation coefficient between items and total scores was less than 0.4, the item needed to be deleted.[32] In the event that removing an item resulted in an increase in the overall Cronbach's α coefficient, it indicated that said item had an Impact on the internal consistency of the questionnaire. Consequently, the item would be deleted.[33]
Validity Verification
The validity tests of the NIS-Q content validity, concerned structure validity, convergent validity, and discriminative validity.
In the present study, EFA and CFA were used to evaluate the structural validity of the questionnaire. Before conducting EFA, Bartlett's sphericity test was executed, and the Kaiser–Meyer–Olkin (KMO) index was assessed. EFA was then conducted utilizing the principal component analysis method with varimax rotation. The aim was to unveil the underlying structure of the questionnaire.[30]
The average variance extraction (AVE) and combined reliability (CR) of each factor were calculated to evaluate the convergent and discriminative validity. An AVE value exceeding 0.5 and a CR value surpassing 0.7 would indicate satisfactory inherent validity.[34]
Five experts were invited to participate in the content validity assessment, including two experts in nursing informatics and three experts in nursing. A 4-point Likert scale was used to score the relevance of each item, where 1 to 4 indicated “no correlation” to “perfect correlation,” to calculate the item-level content validity index (I-CVI) and the scale-level content validity index (S-CVI). Each I-CVI was calculated based on the number of experts scoring either 3 or 4, and the S-CVI was defined as “the proportion of items on an instrument that achieved a rating of 3 or 4 by all the content experts.”[33] I-CVI at or above 0.78 and S-CVI at or above 0.80 are generally considered acceptable.[35]
Reliability Verification
The reliability verification of the present study included internal consistency and test–retest reliability, and all results were expressed using Cronbach's coefficient. Two weeks after the formal test, 20 of the nurses who had participated in the test were selected to refill the questionnaire to calculate the test–retest reliability. In order to ensure the representativeness of the sample, we adopted a random sampling method.
Ethical Considerations
This study was approved by the ethical review of the Zibo Central Hospital Ethics Committee (approval no.: 2023049). Informed consent was obtained from the participating nurses. Those recruited online were unable to proceed until they had read the informed consent form and clicked the “informed consent” button. For offline recruits, signing a physical informed consent form was mandatory. Nurses were told that participation was voluntary and that they could withdraw at any time. The nurses' information was kept anonymous and completely confidential, and all data were used for the present study only.
Results
Fifteen experts participated in two rounds of Delphi expert consultations. The specialists were aged 37 to 57 (45.60 ± 5.34) years and had 10 to 39 (22.80 ± 9.75) years of work experience. Among the 15 experts, there were 10 nursing management experts (five of them are nurse managers on the hospital's information management committee), 2 nursing education experts, 2 clinical nursing experts, and 1 nursing information expert. Their educational level was bachelor's degree (n = 4), master's degree (n = 10), and doctor (n = 1). After adding, modifying, merging, and deleting items according to two rounds of expert opinions, 40 items remained. The primary questionnaire was developed based on these 40 items, which contained 13 items about knowledge (K1–K13), 11 items about attitude (A1–A11), and 16 items about practice (P1–P16).
A pilot test was administered to 20 nurses at the investigator's hospital. There were 2 (10%) males and 18 (90%) females, and the age of the nurses ranged from 26 to 44 years. Two participants (10%) were engaged in nursing management and the other 18 (90%) participants were nurses. Based on their feedback after answering the questionnaire, the details that were difficult to understand were revised and brief explanations were added.
A total of 549 nurses participated in the formal survey, and the data were carefully reviewed and entered by two investigators, leading to the removal of invalid questionnaires. Ultimately, 501 valid questionnaires were recovered, with an effective recovery rate of 91.26%. Of the 501 valid study participants, 452 (90.2%) were female, and most of them (n = 204, 40.7%) were senior nurses. About half (n = 255, 50.9%) of the nurses had experienced training involving information security. The average age of the respondents was 29.54 ± 5.88 years, and the vast majority (n = 398, 79.44%) had undergraduate degrees ([Table 1]).
Table 1
Characteristics of participants (n = 501)
|
Variables
|
Mean ± SD (range) or n (%)
|
|
Sex
|
Female
|
49 (9.8%)
|
|
Male
|
452 (90.2%)
|
|
Age, y
|
29.54 ± 5.88
|
|
20–29
|
301 (60.1%)
|
|
30–39
|
163 (32.5%)
|
|
≥40
|
37 (7.4%)
|
|
Years of nursing experience
|
1–5
|
273 (54.5%)
|
|
6–10
|
116 (23.2%)
|
|
11–20
|
93 (18.6%)
|
|
≥21
|
19 (3.8%)
|
|
Positional title
|
Staff nurse
|
138 (27.5%)
|
|
Senior nurse
|
204 (40.7%)
|
|
Supervisor nurse
|
150 (29.9%)
|
|
Associate chief nurse or above
|
9 (1.8%)
|
|
Position
|
Nothing
|
457 (91.2%)
|
|
Assistant to the charge nurse
|
25 (5.0%)
|
|
charge nurse or above
|
19 (3.8%)
|
|
Level of education
|
Associate degree
|
79 (15.8%)
|
|
Bachelor's degree
|
398 (79.4%)
|
|
Master's degree or above
|
24 (4.8%)
|
|
Experience of clinical teachers
|
Yes
|
203 (40.5%)
|
|
No
|
298 (59.5%)
|
|
Experience in nursing information security training was included
|
Yes
|
255 (50.9%)
|
|
No
|
246 (49.1%)
|
Abbreviation: SD, standard deviation.
The results of the critical ratio method, correlation coefficient method, and Cronbach's α coefficient method are as follows: The critical ratio method showed that the results of the independent-sample t-test for all the items in the high and low groups were between 6.30 and 22.90, which was statistically significant (p < 0.05). The correlation coefficient of each item and the total score was distributed between 0.41 and 0.78, which was confirmed to be above 0.30. The Cronbach's coefficient did not increase significantly after deleting any one item separately. Thus, the 40 items had good homogeneity and discrimination and were retained to test construct validity.
The content validity of the questionnaire was good. For I-CVI, it ranged from 0.80 to 1. For S-CVI, it was 0.94. Both of them met the criteria.
The KMO value of sample 1 was high (0.94), and Bartlett's test of sphericity showed a significant p-value (χ2 = 6,768.39, p < 0.0001). Both indicate that the data were appropriate for EFA. EFA was mainly used to reevaluate and filter items, and it was conducted on the preliminary questionnaire of 40 items. A procedure was followed to ensure that the factors were stable across the extraction and rotation methods. The result of EFA shows that two items (P2 and P4) were excluded due to having factors comprising less than three items. In other words, these two items have created a dimension on their own. All loadings of the remaining items were above the traditional cutoff value of 0.40. Factor 1 was named Practice because it included amounting to a total of 14 items about practice P1, P3, and P5–P16. Similarly, Factor 2 was named Attitude because it included 11 items about attitudes. Factor 3 contained the knowledge-related items K1 to K7, and Factor 4 contained the knowledge-related items K8 to K13. Both factors were related to knowledge and were thus named Knowledge a and Knowledge b, respectively. The results show that the variance contribution rates of the four factors were 13.25, 4.23, 3.68, and 1.68%, respectively, and the cumulative variance contribution rate was 60.10%. [Table 2] provides further details.
Table 2
Exploratory factor analysis (n = 250)
|
Factor
|
Items
|
Factor loading
|
|
1
|
2
|
3
|
4
|
|
Practice
|
P3. I will promptly switch to my own account upon logging into the nursing work system.
|
0.799
|
0.188
|
0.128
|
0.059
|
|
P14. I will not copy, transmit or share the hospital's medical or nursing information without permission.
|
0.773
|
0.143
|
0.136
|
0.191
|
|
P6. I will prevent other unauthorized persons from accessing medical and nursing records.
|
0.773
|
0.166
|
0.032
|
0.015
|
|
P5. I will not utilize the hospital's network resources or access information without authorization. If necessary for work or research, I will apply for patient information in accordance with regulations.
|
0.766
|
0.179
|
0.017
|
0.075
|
|
P10. I will help rotating nurses, continuing education nurses and nursing students avoid actions that pose information security risks.
|
0.766
|
0.152
|
0.169
|
0.052
|
|
P15. I will not dismantle, install or debug software or hardware on any office computer by myself.
|
0.763
|
0.202
|
0.159
|
0.187
|
|
P16. I will actively participate in various emergency response drills organized by managers, so as to deal with equipment failures and network collapses.
|
0.752
|
0.197
|
0.163
|
0.052
|
|
P9. I will not disclose the hospital's information (such as the specific name, specifications, and models of drugs or consumables) to manufacturers or medical representatives.
|
0.752
|
0.092
|
0.062
|
0.101
|
|
P8. When conducting activities such as nursing rounds, medical record reporting, I will mosaic or conceal any private information that is not relevant to the patient's condition.
|
0.749
|
0.189
|
0.169
|
0.090
|
|
P13. I will not disclose patient information to anyone not involved in medical activities without the patient's consent.
|
0.746
|
0.216
|
0.116
|
0.082
|
|
P12. I actively participate in information security training activities organized by managers.
|
0.741
|
0.203
|
0.188
|
0.049
|
|
P7. I will prevent patients, family members, and other non-staff members from accessing the hospital network.
|
0.739
|
0.209
|
−0.006
|
0.193
|
|
P11. Upon discovering issues with information equipment, I will promptly consult with professionals (Information Department, Equipment Department, Logistics Supply Department) to prevent any potential information leakage.
|
0.724
|
0.229
|
0.135
|
0.022
|
|
P1. I will abide by the relevant laws and regulations of information security and the relevant rules and regulations of the hospital in my work.
|
0.689
|
0.259
|
0.006
|
0.210
|
|
Attitude
|
A9. I think it's very important to conduct various emergency response drills to deal with equipment failures and network paralysis.
|
0.191
|
0.765
|
0.020
|
0.248
|
|
A11. I think it is not advisable to access patient information when not engaged in work or study.
|
0.151
|
0.765
|
0.096
|
0.145
|
|
A8. I think it's important to conduct a clinical information security risk assessment.
|
0.187
|
0.757
|
0.111
|
0.102
|
|
A10. I think technology support is important in promoting nursing information security.
|
0.149
|
0.753
|
0.024
|
0.225
|
|
A1. I think that nursing information security is extremely important and constitutes a critical component of hospital information security
|
0.077
|
0.742
|
−0.060
|
0.174
|
|
A2. I believe that protecting the nursing information security is conducive to the orderly development of nursing work and safeguarding the legitimate rights and interests of nursing staff.
|
0.219
|
0.724
|
0.077
|
0.033
|
|
A7. I think it's important to take preventive measures to prevent information loss and leakage in clinical nursing work.
|
0.189
|
0.685
|
0.161
|
0.067
|
|
A6. I believe it's very necessary to abide by the laws and regulations of information security and the hospital's information security management system in the work process.
|
0.232
|
0.685
|
0.199
|
0.021
|
|
A3. I believe that clinical nursing information security can influence patient outcomes.
|
0.233
|
0.663
|
−0.022
|
0.103
|
|
A4. I am willing to proactively learn about nursing information security.
|
0.285
|
0.659
|
0.088
|
−0.009
|
|
A5. I am willing to proactively undergo training on nursing information security.
|
0.271
|
0.629
|
0.043
|
0.155
|
|
Knowledge
|
K3. I am aware that nursing information security has independent theory and content.
|
0.140
|
0.095
|
0.819
|
0.060
|
|
K5. I understand that technical security measures protect information data, control access to information system, and utilize software to enhance security.
|
0.095
|
0.025
|
0.766
|
0.154
|
|
K1. I understand that China has laws and regulations pertinent to information security (Data Security Law of the People's Republic of China, Personal Information Protection Law of the People's Republic of China, etc.).
|
0.212
|
0.079
|
0.745
|
0.126
|
|
K6. I understand that the management security level represents a macro security measure, encompassing laws, education, systems, and security plans.
|
0.066
|
0.032
|
0.696
|
0.315
|
|
K4. I understand that physical security measures should be taken to ensure the integrity of information equipment and protect information systems against unauthorized access and malicious damage.
|
0.114
|
0.146
|
0.675
|
0.299
|
|
K2. I am aware of the hospital's current information security management system.
|
0.202
|
0.077
|
0.655
|
0.156
|
|
K7. I understand that to protect data security, virus prevention and software maintenance should be done regularly in software management.
|
0.055
|
0.062
|
0.636
|
0.328
|
|
K10. I understand that important data should be backed up and properly managed to prevent loss.
|
0.058
|
0.237
|
0.208
|
0.718
|
|
K12. I understand that nursing information security capabilities can be improved through education and training.
|
0.206
|
0.211
|
0.274
|
0.696
|
|
K13. I understand that nursing information security is a responsibility that nurses must master and implement during their professional practice.
|
0.128
|
0.200
|
0.183
|
0.692
|
|
K11. I understand that nursing information security requires strict management of access rights to information to prevent misuse of data.
|
0.166
|
0.126
|
0.126
|
0.689
|
|
K8. I understand that management personnel may assign specialized staff for inspection and maintenance of hardware facilities and equipment, ensuring they are in good condition.
|
0.089
|
0.061
|
0.322
|
0.689
|
|
K9. I understand the emergency response strategies for emergencies that result in equipment failure or network paralysis.
|
0.135
|
0.126
|
0.270
|
0.632
|
Based on the EFA screening items, CFA evaluates the entire model and filters the problematic items again. The goodness of fit of the model structure was validated using 251 samples from Sample 2 ([Fig. 2]). The maximum likelihood estimation method suggested that the fitting indexes of the four-factor model reached the reference value and the model fit well, and the CMIN/DF (Discrepancy Divided by Degree of Freedom) = 1.254 < 3, AGFI (Adjusted Goodness of Fit Index) = 0.841 > 0.8, RMSEA (Root Mean Square Error of Approximation) = 0.032 < 0.08, CFI (Compare Fit Index) = 0.975 > 0.90, GFI (Goodness of Fit Index) = 0.975 > 0.90, TLI (Tucker Lewis Index) = 0.974 > 0.90.
Fig. 2 Model for confirmatory factor analysis of the KAP-NIS-Q (n = 251). NIS-Q, nursing information security questionnaire.
The CR of the questionnaire was 0.91 to 0.95, which met the criterion of 0.7 or higher. The AVE ranged from 0.59 to 0.61. The correlation coefficient between factors revealed by the discriminant validity test is shown in [Table 3]. After calculation, the correlation coefficients of any two factors were less than the AVE values of both of these two factors, indicating the discriminant validity was good.
Table 3
Convergent validity and discriminative validity analysis
|
F1
|
F2
|
F3
|
F4
|
AVE
|
|
F1
|
1
|
|
|
|
0.59
|
|
F2
|
0.65[a]
|
1
|
|
|
0.59
|
|
F3
|
0.71[a]
|
0.66[a]
|
1
|
|
0.61
|
|
F4
|
0.69[a]
|
0.62[a]
|
0.63[a]
|
1
|
0.61
|
Abbreviation: AVE, average variance extraction.
a
p < 0.05.
The Cronbach's α reliability coefficients for each dimension of the questionnaire were found to be acceptable. The coefficients obtained for the dimensions of knowledge, attitude, and practice were 0.899, 0.924, and 0.953, respectively. The overall Cronbach's α coefficient was 0.948, which demonstrated that the questionnaire had good internal consistency reliability. Two weeks later, the overall reliability of the retest with 20 nurses was 0.837, and the test–retest reliability of each dimension ranged from 0.880 to 0.929. After analysis, the reliability of the questionnaire was good.
The results of the analyses revealed that the validity and reliability levels of the final version of the questionnaire with 38 items were high.
Discussion
There is still a lack of systematic special training and education on information security in China. In many cases, nurses begin to learn information security only after they do their jobs. In contrast, foreign countries list information security as a special course during the school or training period, so as to improve the safety awareness of medical staff in advance.[36] Information security is affected by environmental factors, information equipment, information technology, and personnel cognition, among others. Personnel may undermine information security due to insufficient knowledge on information security. For example, health care workers may use unauthorized tools to communicate information because hospital equipment is inadequate. They may use WeChat, SMS, and other tools to communicate and disseminate patients' medical messages, thus inadvertently and unknowingly leading to the leakage of patients' information. They may even hold the following misunderstanding: what they are adopting is a more convenient way.[29] Moreover, the awareness of information security in nursing practice among nurses is generally insufficient, and the organization lacks standardized behavior regarding nursing information security. The present study is the first in which an instrument was constructed for assessing the level of nursing information security in China. Partial items in the questionnaire were prepared in a specific Chinese context, so they have Chinese characteristics. For example, K1 involves China's laws and regulations; in P11, the setting of relevant departments of hospitals might vary from country to country. When using this scale in different cultural contexts, adjustments are needed. At present, there is a lack of a mature definition of nursing information security. Therefore, this study conceptualizes the nursing information security from the perspective of nurses. This is an innovation of this study. Moreover, KAP (Knowledge-attitude-practice) theory is introduced in the setting of dimensions, the NIS-Q can evaluate the level of nursing security-related knowledge, attitude, and practice comprehensively. In essence, effective information security practices require nurses to possess a sound understanding and favorable attitude towards nursing information security.[14]
[37] The NIS-Q provides a basis for conducting targeted educational improvement projects.
Some similar studies used questionnaires to survey nurses' level of nursing information security. Magdalinou et al[14] developed an instrument that was related to the policies and procedures of information security, which is different from the content of the questionnaire in the present study. Its theoretical foundation lies in the self-reporting perception of danger, training opportunities, the enforcement of IT security policies, and the culture in the organization that can affect employees' IT security practices by examining the nurses' non-secure IT practices. This is different from this study, which is based on the research team's conceptual analysis of nursing information security. In addition, in terms of research methodology, it assesses the internal consistency of the tool only. This study makes complete item analysis, validity verification, and reliability verification. Based on their own research team's conceptual analysis of nursing information security,[13] Kang and Seomun[10] developed the nurses' information security attitude questionnaire. The concept analysis contained seven attributes: within the physical aspect were environmental control and facility stability; the technical aspect comprised information accessibility and leveraging features while the administrative aspect included work systematicity, educational execution, and professional responsibility. Kang and Seomun[10] extracted six dimensions directly from these three dimensions and seven attributes. In addition to the differences in conceptual analysis, this study refers to the KAP theory for the setting of dimensions based on the results of the literature review. The three dimensions of the questionnaire, that is, knowledge, attitude, and practice, can comprehensively investigate nurses' levels of knowledge, attitude, and practice about nursing information security. In contrast, the questionnaire prepared by Kang and Seomun[10] is used only to investigate nurses' attitudes. However, there are some similarities between this study and the other research. In the preparation of each specific item, the study makes reference to the content of the items of the other research, while making a conceptual analysis.
To maintain the scientific rigor of the questionnaire, the initial draft underwent a process involving literature analysis, deliberation within the research team, and two rounds of consultations with 15 nursing experts. The questionnaire was subsequently revised based on the insights provided by these experts. The questionnaire development process was followed and pilot testing and item analysis were conducted. The reliability and validity of the questionnaire were verified. The sample was randomly divided into two groups, ensuring relatively independent sets. These samples were then used for the CFA to ensure the statistical validity of the sample and to meet the required size criteria.[19]
[38] During the EFA, four factors were examined. Within the knowledge dimension, two factors emerged, referred to as “Knowledge a” and “Knowledge b.” The reason may be that the knowledge section of Factor 3 is mainly extracted from institutional regulations and literature, so it is focused more on theoretical knowledge; in contrast, the knowledge section of Factor 4 is mainly related to clinical practice. Different focuses of knowledge content lead to this phenomenon. Following team discussions, the two factors were consolidated into a single dimension termed “Knowledge.” The content validity of the questionnaire was above 0.8, which showed good content validity.[39]
[40] Moreover, the convergent validity and discriminative validity were confirmed to ensure the objectivity of the instrument.[41] After analysis, the reliability of the questionnaire was high. The overall Cronbach's α coefficient reached 0.948 and it was high in all dimensions (>0.8).[42]
We encountered challenges during the process of collecting questionnaires. We observed significant variations in the level of nursing IT across hospitals of different grades, as well as disparities between the practices of medical professionals in community and small-scale hospitals compared to general hospitals. In community and small-scale hospitals, insufficient attention is given to nursing information security, with many questionnaire items not being addressed in their daily tasks. For instance, management rules on nursing information security are often lacking, training and education related to information security are inadequate, and essential nursing activities such as rounds and medical record discussions are rarely conducted. Furthermore, their nursing information systems are frequently unusable. Consequently, we found that this questionnaire is more suitable for nurses working in large general hospitals.
Limitations
There are certain limitations in the present study. Considering that the degree of nursing informatization in small medical institutions is insufficient, the 501 participants were all from tertiary general hospitals, and nurses from other level hospitals were not considered, which may have led to some bias. Additionally, with a convenience sample of just five hospitals across the country, the representation of hospitals may not be typical. In addition, there is a lack of cross-sectional research and influencing factor analysis in large samples about nurses' knowledge, attitude, and practice related to information security, so no proper bases are available for identifying the defects in nurses' information security management. The results of this study only provide a tool for self-reporting on nursing information security from the perspective of nurses. In the future, in-depth research can be done by health care institutions or organizations regarding the hospital information system.
Conclusion
In this study, the NIS-Q was developed and its reliability and validity were verified. The NIS-Q demonstrates high levels of reliability and validity, making it a valuable tool for assessing the proficiency level of nursing information security in China. Our model can assist in identifying the knowledge, attitudes, and behaviors related to information security in nursing practice, enhancing the standardization of clinical nurses' information safety behavior, and ensuring a secure medical information environment. Further testing of this newly developed instrument with a larger number of nurses from various backgrounds and different settings is recommended.
Clinical Relevance Statement
Clinical Relevance Statement
This study used a newly developed instrument with the aim of comprehensive evaluation of clinical nurses' knowledge, attitude, and practice of nursing information security. This study can provide valuable guidance for clinical nurses to improve their information security practice behavior.
Multiple-Choice Questions
Multiple-Choice Questions
-
Which dimensions are included in the new development tools?
-
Knowledge
-
Attitude
-
Practice
-
All of the above.
Correct Answer: The correct answer is option d. The questionnaire on information security knowledge, attitude, and practice in nursing provides an instrument for surveying the proficiency level of nursing information security.
-
What was mainly examined of the NIS-Q in this study?
-
Reliability
-
Validity
-
Both a. and b.
-
Specificity
Correct Answer: The correct answer is option c. The NIS-Q was developed and its reliability and validity were verified.
