A recent study showed that state-of-the-art machine-learning algorithms had a higher diagnostic accuracy of pigmented skin lesions compared with human readers and experts.
The researchers conducted a web-based diagnostic study that compared the accuracy of human readers with 139 algorithms created in 77 machine-learning labs that participated in the International Skin Imaging Collaboration 2019 challenge. A total of 511 human readers form 63 countries attempted at least one reader study (238 were dermatologists, 118 were dermatology residents, and 83 were general practitioners), which consisted of randomly selected 30-image batches.
You may also like...
Which Profession Has the Highest Risk for AK and NMSC?
Artificial Intelligence and Dermatology: Partnership or Takeover?
The ground truth of each lesion was classified as one of seven predefined disease categories: intraepithelial carcinoma (including actinic keratoses and Bowen disease), basal cell carcinoma, benign keratinocytic lesions (solar lentigo, seborrheic keratosis, and lichen planus-like keratosis), dermatofibroma, melanoma, melanocytic nevus, and vascular lesions. Main outcomes included the differences in the number of correct specific diagnoses per batch between all human readers and algorithms and between experts and the top three algorithms.
Compared with all human readers, all machine-learning algorithms achieved a mean 2.01 more correct diagnoses of pigmented lesions (17.91 vs 19.92, respectively), the researchers observed. Additionally, 27 experts with more than 10 years of experience had a mean 18.78 correct answers compared with 25.43 correct answers among the top 3 machine algorithms, with a mean difference of 6.65.
In a test set that was collected from sources not included in the training set, the difference between experts and the top three algorithms was significantly lower (human underperformance of 11·4%, 95% CI 9·9–12·9 vs 3·6%, 0·8–6·3; p<0·0001), the researchers said.
“State-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice,” they concluded. “However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research.”
Tschandl P, Codella N, Bengü Nisa Akay BN, et al. Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study [published June 11, 2019]. Lancet Oncol. doi:10.1016/S1470-2045(19)30333-X