AI Diagnostic Support for Generalized Pustular Psoriasis in One Study

Key Takeaways

• Reader accuracy for generalized pustular psoriasis was higher after device-assisted review than during unaided review.
• Improved diagnostic performance was reported in both primary care practitioners and dermatologists.
• The authors noted limits on generalizability, including a teledermatology-style design and underrepresentation of darker skin phototypes.

Overall reader accuracy for generalized pustular psoriasis rose from 23.7% to 46.67% after access to Legit.Health in a multireader multicase evaluation. The AI-enabled dermatology medical device was assessed for generalized pustular psoriasis diagnosis in a virtual clinical setting that paired images with clinical context. Fifteen healthcare practitioners reviewed the same cases first without support and then after seeing the device’s top-5 predictions and confidence scores.

This multireader multicase clinical evaluation followed CLEAR Derm guidance and involved 15 practitioners, including 11 primary care practitioners and 4 dermatologists. All participants were based in Spain, received training before the exercise, first accessed the platform on June 24, 2024, and submitted responses between June 25 and July 6, 2024. The case set contained 100 high-resolution images spanning 15 skin conditions, with each image paired with patient anamnesis in a virtual clinical setting. Readers first recorded an unaided diagnosis, then reviewed the device’s five highest-ranked predictions with confidence values before entering a final diagnosis. For fine-tuning, 4,397 new generalized pustular psoriasis images were added; 3,608 remained after excluding images without visible signs, and these were split into training, validation, and test sets.

After fine-tuning, the class IIb CE-marked device showed top-1 sensitivity of 0.8026 with specificity of 0.9983 for generalized pustular psoriasis. Top-3 sensitivity was 0.8633 with specificity 0.9957, and top-5 sensitivity was 0.9002 with specificity 0.9611. Reader accuracy for generalized pustular psoriasis increased by 22.97 percentage points across all participants, and McNemar testing showed a statistically significant overall effect on diagnostic capabilities with P<.001. Primary care practitioners improved from 20.20% to 44.44%, while dermatologists improved from 33.33% to 52.78% after viewing device output. Table-level analyses also showed changes in final diagnoses across both reader groups after assisted review.

When the top prediction missed generalized pustular psoriasis, the most frequent alternatives were generalized eczematous dermatitis, pustular psoriasis, nonspecific lesions, plaque psoriasis, and morphea. All included images met Dermatology Image Quality Assessment requirements, although the investigators noted that image quality could still affect performance, especially in subtle presentations. The authors also cited single-device scope, underrepresentation of darker skin phototypes, limited user-dynamics data, and a teledermatology-style design that may not reflect in-person assessment.