Download PDF
Methods Inf Med 2018; 57(04): 208-219
DOI: 10.3414/ME17-02-0012
Focus Theme – Original Articles
Georg Thieme Verlag KG Stuttgart · New York

Creation of a Robust and Generalizable Machine Learning Classifier for Patient Ventilator Asynchrony[*]

Gregory B. Rehm
1   Department of Computer Science, University of California Davis, Davis, CA, USA
,
Jinyoung Han
2   College of Computing, Hanyang University, Seoul, Korea
,
Brooks T. Kuhn
3   Division of Pulmonary, Critical Care, and Sleep Medicine, University of California Davis, Sacramento, CA, USA
,
Jean-Pierre Delplanque
4   Department of Mechanical and Aerospace Engineering, University of California Davis, Davis, CA, USA
,
Nicholas R. Anderson
5   Department of Public Health Sciences, University of California Davis, Sacramento, CA, USA
,
Jason Y. Adams   **
3   Division of Pulmonary, Critical Care, and Sleep Medicine, University of California Davis, Sacramento, CA, USA
,
Chen-Nee Chuah   **
6   Department of Electrical and Computer Engineering, University of California Davis, Davis, CA, USA
› Author Affiliations Funding This work was generously supported by the National Heart, Lung, and Blood Institute Emergency Medicine K12 Clinical Research Training Program (K12 HL108964), the Center for Information Technology Research in the Interest of Society (2014–227 and 2015–325), by the UC Davis Clinical and Translational Science Center, and by the National Center for Advancing Translational Sciences, National Institutes of Health (UL1 TR001860). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Further Information

Publication History

received: 29 July 2017

accepted: 26 February 2018

Publication Date:
24 September 2018 (online)

    Permissions and Reprints

Summary

Background: As healthcare increasingly digitizes, streaming waveform data is being made available from an variety of sources, but there still remains a paucity of performant clinical decision support systems. For example, in the intensive care unit (ICU) existing automated alarm systems typically rely on simple thresholding that result in frequent false positives. Recurrent false positive alerts create distrust of alarm mechanisms that can be directly detrimental to patient health. To improve patient care in the ICU, we need alert systems that are both pervasive, and accurate so as to be informative and trusted by providers.

Objective: We aimed to develop a machine learning-based classifier to detect abnormal waveform events using the use case of mechanical ventilation waveform analysis, and the detection of harmful forms of ventilation delivery to patients. We specifically focused on detecting injurious subtypes of patient-ventilator asynchrony (PVA).

Methods: Using a dataset of breaths recorded from 35 different patients, we used machine learning to create computational models to automatically detect, and classify two types of injurious PVA, double trigger asynchrony (DTA), breath stacking asynchrony (BSA). We examined the use of synthetic minority over-sampling technique (SMOTE) to overcome class imbalance problems, varied methods for feature selection, and use of ensemble methods to optimize the performance of our model.

Results: We created an ensemble classifier that is able to accurately detect DTA at a sensitivity/specificity of 0.960/0.975, BSA at sensitivity/specificity of 0.944/0.987, and non-PVA events at sensitivity/specificity of .967/.980.

Conclusions: Our results suggest that it is possible to create a high-performing machine learning-based model for detecting PVA in mechanical ventilator waveform data in spite of both intra-patient, and inter-patient variability in waveform patterns, and the presence of clinical artifacts like cough and suction procedures. Our work highlights the importance of addressing class imbalance in clinical data sets, and the combined use of statistical methods and expert knowledge in feature selection.

Keywords

Patient ventilator asynchrony - artificial respiration - machine learning - intensive care units - clinical decision support systems

* Supplementary material published on our website https://doi.org/10.3414/ME17-02-0012.


** These authors contributed equally to this work.


 

  • References

  • 1 Phillips J, Barnsteiner JH. Clinical Alarms: Improving Efficiency and Effectiveness.. Critical Care Nursing Quarterly 2005; 28 (04) 317.
    CrossrefPubMedGoogle Scholar
  • 2 Graham KC, Cvach M. Monitor Alarm Fatigue: Standardizing Use of Physiological Monitors and Decreasing Nuisance Alarms.. Am J Crit Care 2010; 19 (01) 28-34.
    CrossrefPubMedGoogle Scholar
  • 3 Blum JM, Kruger GH, Sanders KL, Gutierrez J, Rosenberg AL. Specificity Improvement for Network Distributed Physiologic Alarms Based on a Simple Deterministic Reactive Intelligent Agent in the Critical Care Environment.. J Clin Monit Comput 2009; 23 (01) 21-30.
    CrossrefPubMedGoogle Scholar
  • 4 Imhoff M, Kuhls S. Alarm Algorithms in Critical Care Monitoring.. Anesthesia & Analgesia 2006; 102 (05) 1525-1537.
    CrossrefPubMedGoogle Scholar
  • 5 Koski EMJ, Mäkivirta A, Sukuvaara T, Kari A. Clinicians’ opinions on alarm limits and urgency of therapeutic responses.. J Clin Monit Comput 1995; 12 (02) 85-88.
    CrossrefPubMedGoogle Scholar
  • 6 Nouira K, Trabelsi A. Intelligent Monitoring System for Intensive Care Units.. J Med Syst 2012; 36 (04) 2309-2318.
    CrossrefPubMedGoogle Scholar
  • 7 Drew BJ, Harris P, Zègre-Hemsey JK. et al. Insights into the Problem of Alarm Fatigue with Physiologic Monitor Devices: A Comprehensive Observational Study of Consecutive Intensive Care Unit Patients.. PLOS ONE 2014; 9 (10) e110274.
    CrossrefPubMedGoogle Scholar
  • 8 Mac Namee B, Cunningham P, Byrne S, Corrigan OI. The problem of bias in training data in regression problems in medical decision support.. Artificial Intelligence in Medicine 2002; 24 (01) 51-70.
    CrossrefPubMedGoogle Scholar
  • 9 Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential.. Health Information Science and Systems 2014; 2 (01) 3.
    CrossrefPubMedGoogle Scholar
  • 10 Carrington JM, Effken JA. Strengths and Limitations of the Electronic Health Record for Documenting Clinical Events.. CIN: Computers, Informatics, Nursing 2011; 29 (06) 360-367.
    CrossrefPubMedGoogle Scholar
  • 11 Collins SA, Fred M, Wilcox L, Vawdrey DK. Workarounds used by nurses to overcome design constraints of electronic health records.. NI 2012 (2012) 2012; 2012: 93.
    PubMedGoogle Scholar
  • 12 Kim J, Shin HS, Shin K, Lee M. Robust algorithm for arrhythmia classification in ECG using extreme learning machine.. Biomed Eng Online 2009; 8: 31.
    CrossrefPubMedGoogle Scholar
  • 13 Kam HJ, Kim HY. Learning representations for the early detection of sepsis with deep neural networks.. Computers in Biology and Medicine 2017; 89: 248-255.
    CrossrefPubMedGoogle Scholar
  • 14 Calvert JS, Price DA, Chettipally UK. et al. A computational approach to early sepsis detection.. Computers in Biology and Medicine 2016; 74: 69-73.
    CrossrefPubMedGoogle Scholar
  • 15 Herasevich V, Yilmaz M, Khan H, Hubmayr RD, Gajic O. Validation of an electronic surveillance system for acute lung injury.. Intensive Care Med 2009; 35 (06) 1018-1023.
    CrossrefPubMedGoogle Scholar
  • 16 Koenig HC, Finkel BB, Khalsa SS. et al. Performance of an automated electronic acute lung injury screening system in intensive care unit patients*.. Critical Care Medicine 2011; 39 (01) 98-104.
    CrossrefPubMedGoogle Scholar
  • 17 Ziad Obermeyer EJE. Predicting the Future — Big Data, Machine Learning, and Clinical Medicine.. N Engl J Med 2016; 375 (13) 1216-1219.
    CrossrefPubMedGoogle Scholar
  • 18 Blanch L, Villagra A, Sales B. et al. Asynchronies during mechanical ventilation are associated with mortality.. Intensive Care Med 2015; 41 (04) 633-641.
    CrossrefPubMedGoogle Scholar
  • 19 Gilstrap D, MacIntyre N. Patient–Ventilator Interactions.. Implications for Clinical Management. American Journal of Respiratory and Critical Care Medicine 2013; 188 (09) 1058-1068.
    PubMedGoogle Scholar
  • 20 Colombo D, Cammarota G, Alemani M. et al. Efficacy of ventilator waveforms observation in detecting patient-ventilator asynchrony.. Critical Care Medicine 2011; 39 (11) 2452-2457.
    CrossrefPubMedGoogle Scholar
  • 21 Mulqueeny Q, Ceriana P, Carlucci A, Fanfulla F, Delmastro M, Nava S. Automatic detection of ineffective triggering and double triggering during mechanical ventilation.. Intensive Care Med 2007; 33 (11) 2014-2018.
    CrossrefPubMedGoogle Scholar
  • 22 Beitler JR, Sands SA, Loring SH. et al. Quantifying unintended exposure to high tidal volumes from breath stacking dyssynchrony in ARDS: the BREATHE criteria.. Intensive Care Med 2016; 42 (09) 1427-1436.
    CrossrefPubMedGoogle Scholar
  • 23 Chanques G, Kress JP, Pohlman A. et al. Impact of Ventilator Adjustment and Sedation–Analgesia Practices on Severe Asynchrony in Patients Ventilated in Assist-Control Mode*.. Critical Care Medicine 2013; 41 (09) 2177-2187.
    CrossrefPubMedGoogle Scholar
  • 24 Blanch L, Sales B, Montanya J. et al. Validation of the Better Care® system to detect ineffective efforts during expiration in mechanically ventilated patients: a pilot study.. Intensive Care Med 2012; 38 (05) 772-780.
    CrossrefPubMedGoogle Scholar
  • 25 Sinderby C, Liu S, Colombo D. et al. An automated and standardized neural index to quantify patientventilator interaction.. Critical Care 2013; 17 (05) R239.
    CrossrefPubMedGoogle Scholar
  • 26 Yeong Shiong Chiew, Pretty CG, Beatson A, et al. Automated logging of inspiratory and expiratory non-synchronized breathing (ALIEN) for mechanical ventilation. IEEE 2015: 5315–5318.
    PubMed
  • 27 Thille AW, Cabello B, Galia F, Lyazidi A, Brochard L. Reduction of patient-ventilator asynchrony by reducing tidal volume during pressure-support ventilation.. Intensive Care Med 2008; 34 (08) 1477-1486.
    CrossrefPubMedGoogle Scholar
  • 28 Adams JY, Lieng MK, Kuhn BT. et al. Development and Validation of a Multi-Algorithm Analytic Platform to Detect Off-Target Mechanical Ventilation.. Sci Rep 2017; 7 (01) 14980.
    CrossrefPubMedGoogle Scholar
  • 29 Blum AL, Langley P. Selection of relevant features and examples in machine learning.. Artificial Intelligence 1997; 97 1–2 245-271.
    CrossrefPubMedGoogle Scholar
  • 30 Guyon I, Elisseeff A. An Introduction to Variable and Feature Selection.. Journal of Machine Learning Research 2003; 3 Mar 1157-1182.
    PubMedGoogle Scholar
  • 31 Thille AW, Rodriguez P, Cabello B, Lellouche F, Brochard L. Patient-ventilator asynchrony during assisted mechanical ventilation.. Intensive Care Med 2006; 32 (10) 1515-1522.
    CrossrefPubMedGoogle Scholar
  • 32 Kubat M, Holte RC, Matwin S. Machine Learning for the Detection of Oil Spills in Satellite Radar Images.. Machine Learning 1998; 30 2–3 195-215.
    CrossrefPubMedGoogle Scholar
  • 33 Wang S, Minku LL, Yao X, A Systematic Study of Online Class Imbalance Learning with Concept Drift. IEEE Transactions on Neural Networks and Learning Systems. Forthcoming 2017.
    PubMed
  • 34 Tomczak JM. Zi[eogonek]ba M. Probabilistic combination of classification rules and its application to medical diagnosis.. Machine Learning 2015; 101 1–3 105-135.
    CrossrefPubMedGoogle Scholar
  • 35 Japkowicz N. The class imbalance problem: Significance and strategies.. Proc of the Intl Conf on Artificial Intelligence.; 2000. p. 111-117.
    Google Scholar
  • 36 Molina JFG, Zheng L, Sertdemir M, Dinter DJ, Schönberg S, Rädle M. Incremental Learning with SVM for Multimodal Classification of Prostatic Adenocarcinoma.. PLOS ONE 2014; 9 (04) e93600.
    CrossrefPubMedGoogle Scholar
  • 37 Li Q, Clifford GD. Dynamic time warping and machine learning for signal quality assessment of pulsatile signals.. Physiol Meas 2012; 33 (09) 1491-1501.
    CrossrefPubMedGoogle Scholar
  • 38 Rehm GB, Kuhn BT, Delplanque J-P. et al. Development of a research-oriented system for collecting mechanical ventilator waveform data.. J Am Med Inform Assoc.. Forthcoming 2017
    PubMedGoogle Scholar
  • 39 Bishop CM. Pattern Recognition and Machine Learning.. New York: Springer; 2006
    Google Scholar
  • 40 Martín-González F, González-Robledo J, Sánchez Hernández F, Moreno-García MN. Success/Failure Prediction of Noninvasive Mechanical Ventilation in Intensive Care Units.. Methods Inf Med 2016; 55 (03) 234-241.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 41 Dash M, Liu H. Feature selection for classification.. Intelligent Data Analysis 1997; 1 1–4 131-156.
    CrossrefPubMedGoogle Scholar
  • 42 Kotsiantis SB. Supervised Machine Learning: a Review of Classification Techniques. Proceedings of the conference on Emerging Artificial Intelligence Applications in Computer Engineering Real Word AI Systems with Applications in eHealth, HCI. Information Retrieval and Pervasive Technologies; 2007: 3-24.
    Google Scholar
  • 43 Sun Y, Chan KL, Krishnan SM. ECG signal conditioning by morphological filtering.. Computers in Biology and Medicine 2002; 32 (06) 465-479.
    CrossrefPubMedGoogle Scholar
  • 44 Shoeb AH, Guttag JV. Application of machine learning to epileptic seizure detection.. In: International Conference on Machine Learning; 2010
    PubMedGoogle Scholar
  • 45 Jin X, Xu A, Bie R, Guo P. Machine Learning Techniques and Chi-Square Feature Selection for Cancer Classification Using SAGE Gene Expression Profiles.. In: Li J, Yang Q, Tan AH. editors. Data Mining for Biomedical Applications.. Vol 3916. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2006. p. 106-115.
    Google Scholar
  • 46 Hastie T, Rosset S, Tibshirani R, Zhu J. The Entire Regularization Path for the Support Vector Machine.. Journal of Machine Learning Research 2004; 5 Oct 1391-1415.
    PubMedGoogle Scholar
  • 47 Guyon I, Weston J, Barnhill S, Vapnik V. Gene Selection for Cancer Classification using Support Vector Machines.. Machine Learning 2002; 46 1–3 389-422.
    CrossrefPubMedGoogle Scholar
  • 48 Wold S, Esbensen K, Geladi P. Principal component analysis.. Chemometrics and Intelligent Laboratory Systems 1987; 2 1–3 37-52.
    CrossrefPubMedGoogle Scholar
  • 49 Mika S, Ratsch G, Weston J, Scholkopf B, Mullers KR. Fisher discriminant analysis with kernels.. IEEE. 1999: 41-48.
    PubMedGoogle Scholar
  • 50 Lee T-W. Independent Component Analysis.. In: Lee T-W. Independent Component Analysis.. Boston, MA: Springer; 1998. p. 27-66.
    CrossrefGoogle Scholar
  • 51 Nilsestuen JO, Hargett KD. Using Ventilator Graphics to Identify Patient-Ventilator Asynchrony.. Respiratory Care 2005; 50 (02) 202-234.
    PubMedGoogle Scholar
  • 52 Leatherman J. Mechanical Ventilation for Severe Asthma.. Chest 2015; 147 (06) 1671-1680.
    CrossrefPubMedGoogle Scholar
  • 53 Chao DC, Scheinhorn DJ, Stearn-Hassenpflug M. Patient-Ventilator Trigger Asynchrony in Prolonged Mechanical Ventilation.. Chest 1997; 112 (06) 1592-1599.
    CrossrefPubMedGoogle Scholar
  • 54 Mulqueeny Q, Ceriana P, Carlucci A, Fanfulla F, Delmastro M, Nava S. Automatic detection of ineffective triggering and double triggering during mechanical ventilation.. Intensive Care Med 2007; 33 (11) 2014-2018.
    CrossrefPubMedGoogle Scholar
  • 55 Covidien. Operator’s and Technical Reference Manual. Puritan Bennett 800 Series Ventilator System. Covidien; 2011.
    PubMed
  • 56 Prusa J, Khoshgoftaar TM, Dittman DJ, Napolitano A. Using Random Undersampling to Alleviate Class Imbalance on Tweet Sentiment Data.. IEEE. 2015: 197-202.
    PubMedGoogle Scholar
  • 57 Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: Synthetic Minority Over-sampling Technique.. Journal of Artificial Intelligence Research 2002; 16: 321-357.
    CrossrefPubMedGoogle Scholar
  • 58 Friedman JH. On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality.. Data Mining and Knowledge Discovery 1997; 1 (01) 55-77.
    CrossrefPubMedGoogle Scholar
  • 59 Michie D, Spiegelhalter DJ, Taylor CC. Machine Learning, Neural and Statistical Classification.. Ellis Horwood. 1994
    PubMedGoogle Scholar
  • 60 Friedman JH. Greedy Function Approximation: A Gradient Boosting Machine on JSTOR.. Annals of Statistics 2001; 29 (05) 1189-1232.
    PubMedGoogle Scholar
  • 61 Pedregosa F, Varoquaux G, Gramfort A. et al. Scikit-learn: Machine Learning in Python.. Journal of Machine Learning Research 2011; 12 Oct 2825-2830.
    PubMedGoogle Scholar
  • 62 Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees.. Monterey: Wadsworth & Brooks; 1984
    Google Scholar
  • 63 Ross Quinlan J, Rivest RL. Inferring decision trees using the minimum description lenght principle.. Information and Computation 1989; 80 (03) 227-248.
    CrossrefPubMedGoogle Scholar
  • 64 LeCun YA, Bottou L, Orr GB, Müller K-R. Efficient BackProp.. In: Montavon G, Orr GB, Müller KR. editors. Neural Networks: Tricks of the Trade.. Berlin, Heidelberg: Springer; 1998. p. 9-48.
    Google Scholar
  • 65 Rubinstein R. The Cross-Entropy Method for Combinatorial and Continuous Optimization.. Methodology and Computing in Applied Probability 1999; 1 (02) 127-190.
    CrossrefPubMedGoogle Scholar
  • 66 Geurts P, Ernst D, Wehenkel L. Extremely randomized trees.. Machine Learning 2006; 63 (01) 3-42.
    CrossrefPubMedGoogle Scholar
  • 67 Keijzer M, Babovic V. Genetic Programming, Ensemble Methods and the Bias/Variance Tradeoff – Introductory Investigations.. In: Poli R, Banzhaf W, Langdon WB, Miller J, Nordin P, Fogarty TC. editors. Genetic Programming.. Berlin, Heidelberg: Springer; 2000. p. 76-90.
    Google Scholar
  • 68 Kriegeskorte N, Simmons WK, Bellgowan PSF, Baker CI. Circular analysis in systems neuroscience: the dangers of double dipping.. Nature Neuroscience 2009; 12 (05) 535-540.
    CrossrefPubMedGoogle Scholar
  • 69 Zheng ZS, Reggente N, Lutkenhoff E, Owen AM, Monti MM. Disentangling disorders of consciousness: Insights from diffusion tensor imaging and machine learning.. Human Brain Mapping 2017; 38 (01) 431-443.
    CrossrefPubMedGoogle Scholar
  • 70 Lalkhen AG, McCluskey A. Clinical tests: sensitivity and specificity.. Contin Educ Anaesth Crit Care Pain 2008; 8 (06) 221-223.
    CrossrefPubMedGoogle Scholar
  • 71 Beitler JR, Thompson BT, Matthay MA. et al. Estimating Dead-Space Fraction for Secondary Analyses of Acute Respiratory Distress Syndrome Clinical Trials.. Critical Care Medicine 2015; 43 (05) 1026-1035.
    CrossrefPubMedGoogle Scholar
  • 72 Raghuramu A, Pathak PH, Zang H, Han J, Liu C, Chuah C-N. Uncovering the footprints of malicious traffic in wireless/mobile networks.. Computer Communications 2016; 95: 95-107.
    CrossrefPubMedGoogle Scholar
  • 73 Crussell J, Stevens R, Chen H. MAdFraud: Investigating Ad Fraud in Android Applications.. New York, New York, USA: ACM; 2014. p. 123-134.
    Google Scholar
  • 74 Verbaeten S, Van Assche A. Ensemble Methods for Noise Elimination in Classification Problems.. In: Windeatt T, Roli F. editors. Multiple Classifier Systems.. Vol 2709. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2003. p. 317-325.
    Google Scholar
  • 75 Polikar R. Ensemble based systems in decision making.. IEEE Circuits Syst Mag 2006; 6 (03) 21-45.
    CrossrefPubMedGoogle Scholar
  • 76 Dietterich TG. Ensemble methods in machine learning.. In: Multiple classifier systems.. Berlin, Heidelberg: Springer; 2000. p. 1-15.
    Google Scholar