A study released by UK fintech company Saturn found that leading AI models produced incorrect answers to personal finance questions 57% of the time. The research tested 18 free and paid models, including ChatGPT, Gemini, Claude, and Copilot, using 121 questions and generating more than 10,000 responses. According to the report, the errors ranged from factual mistakes to missing key information and failing to include necessary warnings.
The error rate climbed to 88% on harder multi-step questions, while free models performed worst with a 63% error rate. Saturn said the best-performing model, Claude Opus 5, still posted a 39% error rate in reasoning mode. Its mistakes included calculation errors, overlooking tax changes, and citing rules that do not exist.
Saturn CEO Amal Jolly said millions of people rely on AI for financial guidance, but wrong answers can lead to financial losses. At the same time, trust in AI tools continues to rise. An EY global survey found that 49% of consumers had used AI to help with saving and investment decisions. A report from the UK Financial Conduct Authority said four in five first-time investors had used AI tools, and 56% said they trusted AI, above the 47% recorded for television and radio.
UK fintech company Saturn said in a research report that leading AI models gave wrong answers to personal finance questions 57% of the time.
The study tested 18 free and paid models, including ChatGPT, Gemini, Claude, and Copilot. Researchers asked 121 questions and generated more than 10,000 answers. The report said the mistakes included factual errors, missing key information, and failing to provide necessary warnings.
On harder multi-step questions, the error rate rose to 88%. Free models performed worst, with a 63% error rate. Saturn said Claude Opus 5 was the best-performing model, but even in reasoning mode it still recorded a 39% error rate. The reported problems included calculation mistakes, ignoring tax changes, and citing rules that do not exist.
Saturn CEO Amal Jolly said millions of people rely on AI for financial advice, but incorrect answers could lead to financial losses.
At the same time, user trust in AI continues to increase. A global survey by EY found that 49% of consumers had used AI to assist with saving and investment decisions. A report from the UK Financial Conduct Authority said four in five new investors had used AI tools, and 56% said they trusted AI, higher than the 47% recorded for television and radio.
The item was cited by Techub and attributed to BeInCrypto.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
