A new analysis of dermatology mobile applications (apps) revealed transparency, validation, and accuracy concerns.
"With advancements in mobile technology and artificial intelligence (AI) methods, there has been a substantial surge in the availability of direct-to-consumer mobile applications (apps) claiming to aid in the assessment and management of diverse skin conditions," the study authors wrote in JAMA Dermatology. "Despite widespread patient downloads, these apps exhibit limited evidence supporting their efficacy."
The analysis aimed to assess the landscape of dermatology mobile apps featuring artificial intelligence (AI) available for download, focusing on their transparency, validation, and potential risks. Between November to December 2023, researchers searched the Apple and Android app stores using relevant terms like 'dermatology' and 'skin.' After initially 909 apps were identified, with 41 meeting inclusion criteria after eliminating duplicates and non-relevant apps. Of the 41 apps analyzed, the majority targeted lay public or patients (78.0%), with some aimed at clinicians (9.76%) or both (12.2%). Some of the most common purposes of the apps included skin cancer detection (34.1%), diagnosis/identification of skin/hair conditions (31.7%), and mole tracking (17.1%).
The overall analysis results revealed a lack of transparency regarding AI model effectiveness, dataset usage, and user image handling. Concerns about accuracy and safety were also raised, especially when it came to the limited clinician involvement with many of the apps, particularly from dermatologists. Notably, 10 apps (24.4%) claimed diagnostic capability, but lacked scientific evidence. Only five of the apps had supporting peer-reviewed journal publications, with only one of those being a multicenter prospective diagnostic clinical trial. Only six of the apps provided data availability, with the majority of apps providing vague information about datasets (none of the apps provided datasets for the researchers).
"In this scoping review involving 41 apps, several concerning and unsafe aspects were identified," the authors wrote. "These include a lack of supporting evidence, insufficient clinician/dermatologist input, opacity in algorithm development, questionable data usage practices, and inadequate user privacy protection. Presently, artificial intelligence dermatology apps may cause harm associated with bias, inconsistent validation, and misleading user communication; thus, further work is necessary to standardize the information provided by these apps, ensuring both patients and dermatologists are supported with accurate and reliable data while minimizing associated risks."
Study limitations such as regional app variations and dynamic changes in app store content were reported.
Source: Wongvibulsin S, Yan MJ, Pahalyants V, Murphy W, Daneshjou R, Rotemberg V. Current State of Dermatology Mobile Applications With Artificial Intelligence Features: A Scoping Review. JAMA Dermatology. Published online March 7, 2024. doi:https://doi.org/10.1001/jamadermatol.2024.0468