10-Year-Old CNN Outperforms Frontier LLMs for Skin Cancer Diagnosis

A Surprising Reversal: Why an Old CNN Beats New Frontier LLMs

This week, researchers at Lynch Lab published findings that challenged conventional wisdom about AI progress: a convolutional neural network (CNN) trained over a decade ago now outperforms frontier large language models (LLMs) at diagnosing skin cancer. The discovery, detailed in their latest technical report, has sparked urgent conversations across medicine, AI development, and enterprise adoption strategies.

The headline feels counterintuitive. Frontier LLMs—models like GPT-4o, Claude 3.5, and Gemini 2.0—represent the cutting edge of artificial intelligence, trained on trillions of parameters and deployed across millions of applications. Yet when tasked with classifying melanoma and other skin lesions from dermoscopic images, these sophisticated generalists were outmatched by a specialized tool built before most people owned a smartphone.

What the Lynch Lab Study Actually Found

The research compared a 2016-era CNN architecture against multiple state-of-the-art frontier LLMs when applied to skin lesion classification tasks. The older CNN achieved 94.7% accuracy in melanoma detection, while frontier LLMs ranged from 78–89% depending on the model and prompt engineering approach.

Why does this matter now? Three reasons:

1. Enterprise validation: Organisations investing billions in LLM infrastructure need clarity on when specialised tools outperform general-purpose models.
2. Medical liability: Clinical decision-making demands proven accuracy. A 5–16 percentage point accuracy gap could mean hundreds of missed diagnoses at scale.
3. Resource efficiency: Smaller, older models require a fraction of the computational overhead—critical for hospitals with limited budgets.

The findings suggest that LLMs excel at reasoning and contextual understanding, but struggle with pixel-level visual discrimination tasks where CNNs were specifically architected to succeed. Frontier models are generalists; the 2016 CNN was a specialist.

Why Frontier LLMs Struggle with Skin Cancer Diagnosis

Frontier LLMs were designed to process and generate text. When adapted for image tasks, they typically operate by:

• Converting images to text descriptions or embeddings
• Processing visual information through a generalised transformer architecture
• Relying on prompt engineering to guide diagnosis logic

This indirect approach introduces multiple failure points. A CNN, by contrast, applies convolutional filters optimised specifically for detecting edges, textures, and spatial patterns—exactly the visual features dermatologists use to identify malignancy.

Researchers tested various prompt strategies with LLMs, including zero-shot, few-shot, and chain-of-thought prompting. Performance improved modestly (2–4 percentage points) but never closed the gap. The architectural mismatch was fundamental, not a prompting problem.

"We're seeing a critical lesson: bigger and more recent doesn't always mean better for every task. The AI landscape requires nuanced tool selection, not blanket LLM deployment." – Lynch Lab findings summary

The Practical Implications for Healthcare AI

For healthcare leaders and developers, this study reshapes procurement and strategy:

1. Task-Model Fit Matters More Than Frontier Status A decade-old CNN is more reliable for dermatology than a $100M frontier LLM. The inverse may be true for clinical note generation or patient communication. Organisations must match models to specific workflows.

2. Older Models Offer Regulatory Advantages The 2016 CNN has a decade of validation, peer-reviewed evidence, and established performance profiles. FDA and clinical trials favour proven models. A frontier LLM's performance is still being characterised in real-world healthcare contexts.

3. Cost-Benefit Economics Flip Deploying a 2016 CNN requires minimal infrastructure compared to frontier LLM inference. For a hospital processing 10,000 skin biopsies annually, this could represent millions in operational savings while improving accuracy.

What This Means for AI Tool Builders

If you're exploring AI solutions on platforms like ListmyAI, which curates 1,000+ AI tools across categories, this study is a reminder to evaluate based on capability, not recency or brand prestige.

Developers building healthcare AI should consider:

• Hybrid architectures: Use CNNs for imaging, LLMs for clinical narrative and reasoning.
• Transparency over scale: Prove performance on your specific dataset, even if your model is smaller.
• Specialisation as competitive advantage: The race to build general-purpose AI is crowded. Domain-specific models can command premium pricing and trust.

The Broader Lesson: Specialisation vs. Generalisation

This isn't the first time frontier models have underperformed specialists. Similar patterns have emerged in:

• Code generation: Smaller, task-specific code models often outperform general LLMs for specific programming languages.
• Financial forecasting: Domain-trained models beat general-purpose LLMs at time-series prediction.
• Scientific research: Specialised models trained on chemistry or protein data outpace general transformers.

The emerging consensus: frontier LLMs are powerful reasoning engines, but they're not universal solutions. They excel at synthesis, explanation, and context-bridging. For tasks requiring precision in bounded domains, specialised models remain superior.

What Organisations Should Do Right Now

1. Audit your AI stack: Are you using frontier LLMs for tasks where specialised models might be better?
2. Invest in model evaluation: Before deploying any AI system in healthcare, run head-to-head performance tests on your specific data.
3. Don't discard legacy systems: If your 2016 CNN works, rigorously validate before replacing it with a flashier alternative.
4. Plan for hybrid deployment: Modern healthcare AI likely needs LLMs and CNNs, each solving different problems.

The Road Ahead: Integration, Not Replacement

The Lynch Lab findings don't mean frontier LLMs are a failure—they're simply better at different tasks. A frontier model could excel at:

• Integrating skin cancer diagnosis with patient history and risk factors
• Drafting differential diagnoses
• Explaining recommendations to patients

A CNN excels at the core detection task. The future likely involves orchestrating multiple models: CNN for diagnosis, LLM for contextualization and communication.

As AI adoption accelerates across healthcare, regulatory bodies and enterprises will increasingly demand task-specific validation rather than generic capability benchmarks. This shifts the competitive landscape—specialisation becomes an asset, not a sign of obsolescence.

Conclusion: Right Tool, Right Job

A 10-year-old CNN outperforming frontier LLMs for skin cancer diagnosis is not a condemnation of progress; it's a validation that progress is task-dependent. The medical AI community now has a clear roadmap: match model architecture to diagnostic challenge, validate rigorously, and resist the temptation to deploy frontier models universally.

For organisations building, deploying, or selecting AI tools—whether you're exploring options on ListmyAI or evaluating in-house—this research offers one clear lesson: benchmark thoroughly, specialise where possible, and remember that the newest model isn't always the best model.