ICM

DigClaw
2026-08-25 02:36:09

DigClaw says its Rhizome forecasting framework placed three times in FutureX top 10

DigClaw said its Rhizome v1 forecasting framework took the No. 1, No. 3 and No. 7 spots on the FutureX evaluation platform in July, with the results produced separately on different foundation models including Kimi-K3 and DeepSeek-V4-Pro. The company argues the outcome supports a broader thesis: forecasting capability can accumulate outside the base model itself, while model upgrades mainly improve the system’s raw tools for language, reasoning and tool use. In the technical description released alongside the result, DigClaw said Rhizome is built around three engineering choices. The first is to separate search from reasoning, rather than relying on one model to do both. The second is to preserve full prediction trajectories before outcomes are known, then use settlement feedback, Brier Score-based aggregation and Platt scaling to calibrate future probabilities. The third is a causal-chain-aware Bayesian updating system meant to avoid double-counting signals that come from the same underlying driver. The article also links the research to investing. DigClaw said its forecasting stack underpins Newborn Ventures, an AI-native early-stage investment and incubation firm it launched, and that the same system is being used for internal investment practice while also being opened to industrial companies, financial institutions and government-guided funds.

440
DigClaw says its Rhizome forecasting framework placed three times in FutureX top 10
AI
2026-08-20 11:03:11

Terence Tao and Wang Hong say math needs to learn how to absorb AI-generated proofs

Artificial intelligence is producing mathematical proofs faster than the field can comfortably process, and that shift is forcing a debate over what counts as a complete mathematical result. In comments highlighted by MarsBit, Fields Medalists Terence Tao and Wang Hong arrive at much the same conclusion: mathematics cannot ignore AI-generated counterexamples or proofs, but it also cannot stop at machine output. Results still have to be understood, organized, explained, and absorbed by human researchers before they become fully usable. Tao’s recent work on the 67-year-old Sendov conjecture is presented as a concrete example. After math enthusiast Lech Mazur used AI to fill the long-open middle range of the problem and produced a Lean-verified formal proof, Tao spent several days reworking the result into a form mathematicians could actually read and use. According to the source text, that process not only clarified the proof but also extended it to the stronger Phelps–Rodriguez conjecture, while reducing the Lean code from about 90,000 lines to 15,000. Tao is also pushing a broader change in incentives. He argues that the field should place more value on digesting, explaining, reviewing, and integrating proofs, not just being first to announce them. He has also opened Palomar, a registry for Lean-verified results that records proof statements, code, AI involvement, and version details as a bridge between verification and formal publication.

390
Terence Tao and Wang Hong say math needs to learn how to absorb AI-generated proofs
Google Resear
2026-08-14 07:41:09

Google study says GPT-5 and Gemini 3 often know facts but fail to recall them

Google Research says a large share of factual errors in frontier language models may come from retrieval failure rather than missing knowledge. In a study titled "Empty Shelves or Lost Keys?" and accepted at ICML 2026, researchers found that GPT-5, Gemini 3 and other tested models had already stored 95%–98% of benchmark facts in their parameters, yet still failed to produce 26%–34% of those facts in direct question answering. Even with thinking enabled, 11%–12% remained inaccessible. The work introduces a "knowledge states" framework and a new benchmark called WikiProfile, built from 2,150 English Wikipedia facts and 21,500 associated questions. Google evaluated 13 models across the Gemini 3, GPT-5, GPT-4.1 and Gemma 3 families, with and without thinking, sampling each question eight times for roughly 4.5 million responses. The results point to two major choke points: rare facts and reversed questions. The paper argues that in both cases the issue is often not that the model never learned the fact, but that it struggles to retrieve it when wording or direction changes. Google also reports that thinking helps recover 40%–65% of facts that were stored but initially unreachable, while helping only 5%–15% on facts that were never stored, suggesting thinking can function as a recall aid rather than only a reasoning tool.

640
Google study says GPT-5 and Gemini 3 often know facts but fail to recall them
OpenAI
2026-08-09 23:25:08

OpenAI Researcher’s ‘We Don’t Read Papers’ Remark Reignites Debate Over AI Research Quality

An OpenAI researcher’s remark that “we don’t read papers anymore” has sparked fresh debate over how much trust the AI community should place in top conference publications. The discussion picked up after scrutiny of a surprisingly strong ICLR paper led online commenters to question how some headline results are produced. The broader criticism gained traction alongside a large-scale reproducibility review published in July by SAI, a group co-founded by University of Chicago computer science and data science associate professor Tan Chenhao. SAI examined all 168 oral papers from ICML 2026, a conference that received 23,918 submissions and accepted just 168 for oral presentation, or about 0.7%. According to the report, SAI completed 105 full reproductions and found that only 34 papers reproduced more than 40% of their claims, while just eight cleared the 80% mark. The group also pointed to broken code, missing files, incomplete instructions, mismatched results, and four papers that depended on models no longer available. SAI estimated the median cost of fully rerunning one ICML oral paper at about $8,900, with 17 papers costing more than $100,000 and the most expensive nearing $2.2 million. The episode has sharpened a longstanding tension: elite labs may rely less on papers in day-to-day work, but papers still remain a core gatekeeping tool for students and early-career researchers trying to enter those same institutions.

1450
OpenAI Researcher’s ‘We Don’t Read Papers’ Remark Reignites Debate Over AI Research Quality
AI paper repr
2026-08-09 08:20:07

AI tools flag reproducibility and error issues across top research papers

AI-based review systems are beginning to expose how often published research may contain reproducibility problems or objective mistakes, according to a MarsBit report citing several recent projects focused on top artificial intelligence venues. One audit, carried out on all 168 oral papers at ICML 2026 by a U.S. research review company, found that 92 papers had at least five verifiable claims. Among those, 58 could not be reproduced. Only 34 papers had more than 40% of their claims successfully reproduced, and just eight cleared the 80% mark. The report draws a line between failed reproduction and fraud. In the cases cited, failures were tied to missing files, broken software dependencies, mismatches between paper claims and runtime results, and four papers that depended on models no longer available. A separate GPT-5-based paper checking system, introduced in late 2025 to scan already published papers in top AI conferences and journals, reported an average of 4.7 objective errors per paper, with 99.2% of papers flagged for at least one issue. The report says mathematics and formula-related mistakes made up 54.0% of all errors, while 30.8% of NeurIPS papers and 23.8% of ICLR papers contained at least one substantive error that could affect interpretation. Even so, the article notes that AI review tools still need human oversight: one checker cited had 83.2% precision and still missed about 40% of real errors in each run.

1290
AI tools flag reproducibility and error issues across top research papers
AI
2026-07-22 07:54:07

ICML 2026 paper says random noise can serve as a transfer source in low-label learning

A study presented at ICML 2026 argues that transfer learning does not always need a semantically meaningful source domain. The paper, titled "Semi-Supervised Noise Adaptation: Transferring Knowledge from Noise Domain," introduces Semi-Supervised Noise Adaptation (SSNA) and a Noise Adaptation Framework (NAF) that builds class-separable structure from randomly sampled Gaussian noise, then aligns that structure with a target domain using a small number of labeled examples. Under a 4-labels-per-class setting and a ResNet-18 backbone, NAF outperformed the standard empirical risk minimization baseline on CIFAR-10, CIFAR-100, DTD-47, and Caltech-101 by 12.35, 7.61, 4.38, and 2.74 percentage points, respectively. The study also reported gains on fine-grained datasets, ImageNet-1K, and the AG News-4 text classification task with BERT. The code has been open-sourced. Ablation results in the paper suggest the useful part is not randomness itself, but whether the noise domain forms a separable class structure in representation space. When that structure collapses, performance falls sharply. When class centers are pulled farther apart, results improve. The work frames synthetic noise as a low-cost substitute when real source data cannot be shared because of privacy, confidentiality, or copyright limits.

350
ICML 2026 paper says random noise can serve as a transfer source in low-label learning
IMO 2026
2026-07-20 02:47:39

China wins IMO 2026 as reports about GPT-5.6 solving all six problems draw attention

China took first place at the 67th International Mathematical Olympiad in Shanghai with a perfect all-gold team result and a total score of 232, finishing 25 points ahead of the United States. Deng Leyan and Zhang Bairun from Shanghai High School, along with Liu Che from the High School Affiliated to East China Normal University, each earned gold with full marks. The report says this was China’s 26th team title since its first championship in 1989, and the eighth straight edition in which Chinese contestants produced perfect-score gold medals. The United States finished second with 4 golds, 1 silver, 1 bronze and 207 points, while Russia placed third with 4 golds, 2 silvers and 196 points. The article also shifted to AI, noting that no official public results had been released at the time on 2026 IMO problem runs, while separately citing reports that GPT-5.6 Pro solved all six problems on its first attempt without human prompting and that SignalPilot Labs said it produced full solutions for the full set.

2620
China wins IMO 2026 as reports about GPT-5.6 solving all six problems draw attention