Summary
Objectives: Artificial neural networks have proved to be accurate predictive instruments in several
medical domains, but have been criticized for failing to specify the information upon
which their predictions are based. We used methods of relevance analysis and sensitivity
analysis to determine the most important predictor variables for a validated neural
network for community-acquired pneumonia.
Methods: We studied a feed-forward, back-propagation neural network trained to predict pneumonia
among patients presenting to an emergency department with fever or respiratory complaints.
We used the methods of full retraining, weight elimination, constant substitution,
linear substitution, and data permutation to identify a consensus set of important
demographic, symptom, sign, and comorbidity predictors that influenced network output
for pneumonia. We compared predictors identified by these methods to those identified
by a weight propagation analysis based on the matrices of the network, and by logistic
regression.
Results: Predictors identified by these methods were clinically plausible, and were concordant
with those identified by weight analysis, and by logistic regression using the same
data. The methods were highly correlated in network error, and led to variable sets
with errors below bootstrap 95% confidence intervals for networks with similar numbers
of inputs. Scores for variable relevance tended to be higher with methods that precluded
network retraining (weight elimination) or that permuted variable values (data permutation),
compared with methods that permitted retraining (full retraining) or that approximated
its effects (constant and linear substitution).
Conclusion: Methods of relevance analysis and sensitivity analysis are useful for identifying
important predictor variables used by artificial neural networks.
Keywords
Neural networks (computer) - diagnosis (computer-assisted - sensitivity - analysis
- pneumonia
