Considerations and prospects for validating a machine learning-based choledocholithiasis prediction model

We are writing to express our great interest in the recent study by Steinway et al. [1], which presents an impressive machine learning-based choledocholithiasis prediction model. The study demonstrated the feasibility of machine learning in gastroenterology, showing its strong clinical utility. However, we have some considerations.

First, the availability of the second set of laboratory tests poses a challenge, as many patients may have undergone endoscopic retrograde cholangiopancreatography (ERCP) when the first set of test results strongly indicated choledocholithiasis. As stated in the Methods section, “If available, a second set…” suggests that not all patients had the second tests, potentially impacting the second and third models. We are interested in the authors’ approach to addressing the issue of missing data.

Second, after the first set of liver function tests showed elevated levels, it is uncertain whether any hepatoprotective measures were implemented, which could have impacted the second set of tests. The authors suggest that stones may pass spontaneously, leading to decreased liver function indicators. Table 1 shows a higher proportion of elevated bilirubin and alkaline phosphatase (ALP) in the second test. Follow-up bilirubin and ALP levels were strong predictors. We speculate that choledocholithiasis led to bile duct obstruction, increasing bilirubin and ALP over time. A more rigorous experimental design is necessary to confirm whether dynamic liver tests can predict choledocholithiasis.

Third, calibration curves, Hosmer–Lemeshow goodness-of-fit tests, and decision curve analysis curves, which are recommended by the TRIPOD statement [2] for assessing calibration and clinical utility, should be considered.

Fourth, the computer-based calculator addresses the limitation of the gradient boosting model in visualizing weights and generating a nomogram. However, the lack of public access restricts its widespread clinical application and external validation. As a large ERCP center, we are keen to prospectively validate this model and hope for its public release to facilitate broader application.

Publication History

Article published online:
27 June 2024

• References

• 1 Steinway SN, Tang B, Telezing J. et al. A machine learning-based choledocholithiasis prediction tool to improve ERCP decision making: a proof-of-concept study. Endoscopy 2023; DOI: 10.1055/a-2174-0534.
• 2 Collins GS, Reitsma JB, Altman DG. et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015; 350: g7594 DOI: 10.1136/bmj.g7594. (PMID: 25569120)