Specialized medical AI outperforms general chatbots at evaluating skin lesions

A dermatologist recently put general-purpose artificial intelligence models to the test against a decade-old, specialized image algorithm to evaluate skin lesions. The evaluation revealed that older single-purpose software remains vastly superior at identifying skin cancers compared to today's most advanced general chatbots.

According to a published test by Dr. Magnus Lynch, two leading general AI tools and a fine-tuned image recognition model from 2015 were given two hundred biopsy-proven skin photos. The older image model correctly classified 62 percent of the cases, while the general chatbots managed only 42 percent accuracy. In several instances, the general chatbots wrongly labeled dangerous melanomas as safe spots that required no follow-up care, posing serious potential risks to patient safety.

While popular chatbots excel at conversational tasks, their general design struggles with the precise visual patterns required for medical imaging. This gap highlights an important dilemma for digital healthcare, specifically whether consumers are better served by small, targeted diagnostic tools that run directly on personal devices instead of relying on massive cloud-based chatbots.