DeepSeek V4 Flash reached the top of a ranking, but its showing in an independent evaluation was far weaker. According to Techub, citing CryptoBriefing, Composio tested the model on complex agent tasks and found it completed only 53.8% of them. The evaluation covered 30 multi-step workflows and included 240 runs across real-world tools such as Gmail, GitHub, and Slack. Results also showed wide performance gaps across agent frameworks. Pi Agent delivered the best outcome, completing 20 tasks. The report added that DeepSeek V4 Flash has an input token price as low as $0.14 per million, roughly one-tenth of comparable products, but said the high failure rate could create extra computing and debugging costs. The findings point to a gap between leaderboard standing and execution in tool-based, multi-step workflows.
DeepSeek V4 Flash has topped a ranking, yet an independent test by Composio showed weak performance on complex agent tasks, with the model completing only 53.8% of them, according to Techub, citing CryptoBriefing.
Test covered 30 workflows and 240 runs
The evaluation spanned 30 multi-step workflows and involved 240 runs. It used real tools including Gmail, GitHub, and Slack.
Results varied across agent frameworks
The report said the model’s performance differed sharply depending on the agent framework. Pi Agent posted the best result, completing 20 tasks.
Low pricing may be offset by failure-related costs
DeepSeek V4 Flash lists input token pricing at $0.14 per million, about one-tenth the level of comparable products. Even so, the report said the model’s high failure rate could lead to added computing and debugging costs.
This article was originally published by Bit.Fan. For more cryptocurrency news and market insights, visit www.bit.fan. Disclaimer:
The market information, project data, and third-party content displayed on this platform are for industry information sharing only and do not constitute any form of investment advice or return commitment.
Cryptocurrency trading carries high risks. Users should fully assess their risk tolerance and make independent decisions. All profits, losses, and legal responsibilities are borne by the users themselves.
