A comparison published by Lynch Lab on Oct. 25 Taipei time found that Inception v3, a convolutional neural network introduced by Google in 2015 and later fine-tuned on public dermatology images, outperformed GPT-6 Astra and Claude Fable 5.1 on a skin lesion diagnosis task built from 200 pathology-confirmed cases.
In the six-way first-diagnosis test, Inception v3 got 125 of 200 cases correct, or 62%. GPT-6 Astra and Claude Fable 5.1 got 84 and 85 cases right, roughly 42% each.
One example cited in the write-up involved a lesion on the ear. Both large language models labeled it as harmless seborrheic keratosis and said discharge was appropriate. The CNN, using dermoscopic imagery, identified it as melanoma.
The test covered 200 pathology-confirmed lesions
The author, Dr. Magnus Lynch, is described as a consultant dermatologist and Mohs surgeon at Guy’s Hospital and an Honorary Reader at King’s College London. He said the comparison was based on 200 randomly selected pathology-confirmed lesions and measured only whether the first choice in a six-option diagnosis task was correct. It was not presented as a claim that any system could replace clinic visits.
The lesions came from MILK10k, a public dataset released in 2025. Each case included both a clinical close-up image and a dermoscopic image. Of the 200 cases, 100 were not suitable for direct discharge: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas including Bowen’s disease, and 20 actinic keratoses. The other 100 could be discharged: 50 benign nevi and 50 seborrheic keratoses. All cases were biopsy-confirmed, which the article said made the set relatively difficult.
Inception v3 led on both first-choice and top-two results
The two LLMs were given the same paired images and the same prompt. They were asked for a first choice, a confidence level, a second choice, and a judgment on whether discharge was safe. The CNN was fine-tuned end-to-end on 68,011 images from the ISIC Archive. According to the article, that process took about 20 minutes on a single H100. Test lesions and photos from the same patients were removed from the training and validation sets in advance.
When either the first or second choice counted as correct, the CNN reached 173 correct cases. GPT-6 Astra and Claude Fable 5.1 reached 134 and 138. On melanoma specifically, first-choice accuracy was 13 out of 30 for the CNN, versus 8 and 11 for the two LLMs. The article said the gap was not limited to malignant lesions. The specialized CNN also showed clearly better accuracy on benign nevi and seborrheic keratoses.
Error patterns differed across models
The write-up said GPT-6 Astra showed a strong tendency to call lesions basal cell carcinoma. It made that call 75 times even though the dataset contained only 25 real basal cell carcinoma cases, and 14 of those predictions were actually melanomas. Claude Fable 5.1 leaned more often toward actinic keratosis.
When the CNN missed melanoma, it more often shifted toward nevus or seborrheic keratosis. The article noted that the direction of the mistakes matters because the risk profile changes with it.
Debate focused on specialist models versus general models
The article said the post drew discussion on Hacker News. As of around Oct. 26 Taipei time, it had about 500 points and more than 200 comments. A common view in the discussion was that it is not surprising for a specialized smaller model to beat a general-purpose larger model. Others questioned whether the comparison was fully fair, since the CNN had been fine-tuned on domain-specific data while the LLMs had not received equivalent specialist training.
The author listed clear limits
The article also laid out several constraints. The sample size was small at 200 cases, with only 30 melanomas, leaving substantial uncertainty. Each LLM was run only once per case, so outputs could vary. All lesions were biopsy-confirmed, which again means the benchmark leaned toward harder cases.
MILK10k has been public since 2025, so the LLMs could theoretically have seen some of the material. If that happened, the article said, it would have favored the LLMs rather than the CNN. By contrast, the CNN was confirmed not to have seen the test set.
The piece also stressed that Inception v3 is not the strongest skin cancer AI available today. Current systems in the field often use vision transformer ensembles, and regulated products perform better. In that framing, the comparison is closer to a check on a simple claim: general-purpose large models do not automatically beat specialized models. It is not proof that an older CNN is ready for clinical deployment.
Threshold settings changed the risk picture
At the default threshold, the CNN retained only 77 of the 100 cases that should not have been discharged. After adjusting the threshold to a comparable level, that figure rose to 98 to 99.
The article also described a patient-style question using only clinical photos: whether someone should see a doctor. In that setup, both LLMs recommended seeking care in almost every case. In another set of 100 pathology-confirmed melanoma cases, the two models advised seeing a doctor in 99 and 98 cases, which the article said showed a conservative tendency in general-purpose models.
What the test shows, and what it does not
On this set of pathology-confirmed cases, an older CNN fine-tuned in about 20 minutes delivered a clearly higher first-diagnosis accuracy than the two frontier LLMs. The article stopped there. It said questions about product selection still require larger samples, reproducible threshold design, and results from regulated specialist imaging systems. This test did not answer those points.