一个10岁的有线电视新闻网在皮肤癌诊断方面超越了前沿LLMs

一个10岁的有线电视新闻网在皮肤癌诊断方面超越了前沿LLMs

我是皮肤科医生我测试了GPT-6 Astra和Claude Fable 5.1诊断皮肤癌的能力，发现其准确性比我预期的要差。 我和10年的CNN相比

这是黑色素瘤

我向GPT-6 Astra和Claude Fable 5.1展示了这两幅图像——一张临床照片和一个皮肤显微镜图像，这是2026年10月可用的两种最能用的AI模型。 称都是血清性煤酸盐，一种无害的生长，病人可以出院。 随后，我向他们展示了Inception v3的微调版本，这是Google于2015年首次发布的革命神经网络(CNN)，并在公共皮肤学图像上在这里进行了微调。 美国有线电视新闻网根据皮肤镜的视图得到了正确的答案。

那是我给所有三个人的200个活检证明的损伤之一美国有线电视新闻网(CNN)为其中125人(62%)命名了正确的诊断(在6个病例中). 两个边疆LLMs将其命名为84和85(42%). 一个10岁的网络，微调大约20分钟，击败他们两人在识别皮肤癌。

测试#

这些损伤来自MILK10k，这是通过2025年国际皮肤成像协作组织制作的一套公共数据集发布，其中对每次损伤进行两次拍照：临床特写和皮肤透视图像通过光线放大镜皮肤学家使用。 我随机抽取了200个损伤都由组织病理学证实：

- 不应出院的100例：30例黑色素瘤、25例玄武细胞癌、25例结膜细胞癌(包括Bowen的疾病)和20例动脉癌。

- 100枚安全排出物：50枚良性马先蒿(Moles)和50枚 se(seborrhoeic keratoses)。

每一次损伤都被活化，包括良性疾病，所以这些更难诊断：被别人决定切掉的内鼠是担心临床医生的内鼠。

LLMs既得到图像，又得不到其他任何图像，而且由于临床医生可能使用这些图像，因此要求最有可能的六种诊断(或“不确定”)中有一种信心，一种第二种选择，以及是否可以在没有治疗、活体检查或后续的情况下安全出院。 两种模型在默认设置时都得到了相同的提示，也没有拒绝一个案例。

美国有线电视新闻网是故意的，没有达到最新水平。 我用Inception v3在日常照片上进行预试，然后对68,011皮肤和临床图像进行精细调整，这些图像来自国际标准行业分类档案的H100 GPU。 这200个测试损伤，以及同一损伤或患者的所有其他照片，在训练前从数据(包括验证集)中移除，因此从未见过，也没有用来选择模型或设定其阈值。 它把每个损伤的图像都打得分，把它们和皮肤图像的大部分重量结合起来，从同样的六个诊断中选择。

你可以自己试试CNN 值得注意的是，该模型很小，可以完全在使用WebGPU的电话浏览器中运行。 没有任何内容被上传，它的随机抽样按钮也来自同样的200个损伤。

诊断：62%对42%#

CNN更经常地说得没错，允许第二种选择，200人中的173人，134人和138人。 它命名了30个黑色素瘤中的13个对8和11. 最大的空白在于无害损害：它识别出50个摩尔人中的37个和50个seborrhoeic keratoses中的26个，GPT-6 Astra的16个和13个，Claude的33个和6个。 在玄武细胞癌上是GPT-6 Astra的缩写，在动脉性煤酸盐上是LLMs的后面。

模型也以不同方式失败。 每个LLM有一个最受欢迎的答案：GPT-6 Astra在25岁时称为75个损伤性basal细胞癌，包括30个黑色素瘤中的14个；Claude Fable 5.1在20岁时称为53个损伤性动性性性煤热症。 CNN在黑色素瘤上的错误则相反：在17个错误中

在一个表格中得出的结果#

洞穴#

- 人数少。 两百个损伤足以显示规律，但不能对近似结果进行排序，30个黑色素瘤对最重要的数字给予广泛的不确定性。 每个LLM案件运行一次；答案因运行而异。

- 困难的案子。 每一个损伤都被活化了。 在档案的普通图像上，有线电视新闻网分离出应该而且不应该排放的损伤(ROC曲线0.96下的区域，这里为0.86)。

- 污染问题 MILK10k从2025年开始公开，因此LLMs可以在训练中看到它；如果有什么能帮助他们的话。 CNN当然没有

- 没有最新进展 Inception v3是一个基线。 最新艺术的皮肤癌诊断AI使用经过更多数据培训的图像变压器组合，其上所建的受管制产品表现要好得多。

作为病人被问及#

LLMs建议在几乎所有情况下都看医生，无害的损伤包括：另外100个经活检证实的黑色素瘤中有99个和98个。

结论#

一个使用过时建筑的小型、老化的专门网络认识到皮肤损伤远比两个前沿LLMs好。

自己来

案件，提示，每个LLM的回复和评分代码都是一次下载，CNN也在这里。

我设计了测试做了临床诊断分析和第一稿是用Claude Code,Anthropic的编码工具制作的，该工具运行在Claude模型上；两个测试过的LLMs中，还有一个也是Anthropic. Scoring是用剧本来对付神学的。

图片来自MILK10k数据集(MILK研究团队，国际皮肤成像协作)，在CC BY-NC许可下使用。

本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

I am a dermatologist. I tested the ability of GPT-6 Astra and Claude Fable 5.1 to diagnosis skin cancer and found that they were less accurate than I expected. I compared to a 10-year old CNN and to my surprise found that this was superior.

This is a melanoma.

I showed these two images - a clinical photograph and a dermoscopic (skin microscope) image - to GPT-6 Astra and Claude Fable 5.1, two of the most capable AI models available in October 2026. Both said it was a seborrhoeic keratosis, a harmless warty growth, and that the patient could be discharged. I then showed them to fine-tuned version of Inception v3, a convolutional neural network (CNN) first published by Google in 2015 and fine-tuned here on public dermatology images. The CNN got the right answer based on the dermoscopy view.

It was one of 200 biopsy-proven lesions I gave to all three. The CNN named the right diagnosis (out of 6 possiblities) for 125 of them (62%). The two frontier LLMs named it for 84 and 85 (42%). A ten-year-old network, fine-tuned for about twenty minutes, beat both of them at recognising skin cancer.

The test#

The lesions come from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025, in which each lesion is photographed twice: a clinical close-up and a dermoscopic image taken through the illuminated magnifying lens dermatologists use. I drew 200 lesions at random, all confirmed by histopathology:

- 100 that should not be discharged: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas (including Bowen’s disease) and 20 actinic keratoses.

- 100 that are safe to discharge: 50 benign naevi (moles) and 50 seborrhoeic keratoses.

Every lesion was biopsied, including the benign ones, so these are harder to diagnose: a mole that somebody decided to cut out is one that worried a clinician.

The LLMs were given both images and nothing else, and asked, as a clinician might use them, for the most likely of the six diagnoses (or “unsure”), a confidence, a second choice, and whether the lesion could be safely discharged with no treatment, biopsy or follow-up. Both models got the same prompt at default settings, and neither refused a case.

The CNN is deliberately not state of the art. I used Inception v3 pretrained on everyday photographs and then fine-tuned all layers on 68,011 dermoscopic and clinical images from the ISIC Archive on a single H100 GPU. These 200 test lesions, and every other photograph of the same lesion or patient, were removed from its data (including the validation set) before training, so it had never seen them, and they were not used to choose the model or set its threshold. It scored both images of each lesion, combined them with most of the weight on the dermoscopic image, and chose from the same six diagnoses.

You can try the CNN yourself. Remarkably the model is small enough to run entirely in a phone browser using WebGPU. Nothing is uploaded, and its random sample button draws from these same 200 lesions.

Diagnosis: 62% against 42%#

The CNN was right more often overall and on most diagnoses. Allowing a second choice, it reached 173 of 200 against 134 and 138. It named 13 of the 30 melanomas against 8 and 11. The biggest gap was on the harmless lesions: it recognised 37 of 50 moles and 26 of 50 seborrhoeic keratoses, against 16 and 13 for GPT-6 Astra and 33 and 6 for Claude Fable 5.1. It was one short of GPT-6 Astra on basal cell carcinoma and behind both LLMs on actinic keratosis.

The models also failed differently. Each LLM had a favourite answer: GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the 30 melanomas, and Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were. The CNN’s errors on melanoma went the other way: of the 17 it got wrong, it called 8 a naevus and 6 a seborrhoeic keratosis.

The results in one table#

Caveats#

- Small numbers. Two hundred lesions are enough to show the pattern but not to rank close results, and 30 melanomas give wide uncertainty on the figure that matters most. Each LLM case was run once; answers vary between runs.

- Hard cases. Every lesion was biopsied. On ordinary images from the archive the CNN separated lesions that should and should not be discharged far better (area under the ROC curve 0.96, against 0.86 here).

- Contamination. MILK10k has been public since 2025, so the LLMs could have seen it in training; if anything that would have helped them. The CNN certainly had not.

- Not state of the art. Inception v3 is a baseline. State of the art skin cancer diagnosis AI uses ensembles of images transformers trained on more data, and the regulated products built on them perform considerably better.

Asked as a patient#

Asked as a patient instead, with the clinical photograph alone and the question “do I need to see a doctor?”, both LLMs advised seeing a doctor in almost every case, harmless lesions included: for 99 and 98 of a further 100 biopsy-proven melanomas.

Conclusion#

A small, old, specialised network using an outdated architecture recognises skin lesions far better than two frontier LLMs.

Run it yourself#

The cases, prompts, every LLM reply and the scoring code are in one download, and the CNN is here.

I designed the test and made the clinical rulings. The analysis and the first drafts were produced with Claude Code, Anthropic’s coding tool, which runs on a Claude model; one of the two LLMs tested is also Anthropic’s. Scoring was done by script against the histology.

Images are from the MILK10k dataset (MILK study team, International Skin Imaging Collaboration), used under a CC BY-NC licence.

Auto-fetched and translated by AI Atlas. All rights belong to the original author.