Solana nearly lost its ability to finalize transactions early Wednesday after a routing misconfiguration at a single hosting provider pushed 28.83% of staked SOL offline, according to staking platform Marinade.
Marinade said the network stops finalizing at 33.34%, which meant the incident brought Solana to roughly 86% of the way toward a freeze.
Teraswitch route propagation triggered the outage
The fault began at Teraswitch, Marinade said. A default route from the provider’s Miami site was propagated across its sites in Europe and the Asia-Pacific, leading to a wider disruption.
Marinade’s analysis focused heavily on stake concentration. One autonomous system, AS20326, carries 118,890,767 SOL, more than a quarter of all stake on the network and above the 25% ceiling set by the Solana Foundation’s delegation program. According to Marinade, 94% of that stake went dark during the same stretch.
Another 14.1 million SOL also dropped offline across latitude.sh, Limestone, Butterfly Research and Allnodes. Marinade said the available data did not explain that portion of the outage.
Failover saw little use as validators waited for routes to reconverge
Marinade said failover mechanisms barely engaged. It found that 59 validators holding 80.2 million SOL returned within the same narrow window in Amsterdam, Frankfurt and Tokyo after waiting for routing to reconverge instead of switching elsewhere.
Helius, Solana’s second-largest validator, was down for the full 33 minutes. Of the 74 operators Marinade said it could measure, only three recovered cleanly: Laine and Cogent Crypto, both operated by Sol Strategies, and Lion3d.
Reward losses, Foundation response and Marinade’s own review
The 90 affected validators lost 333 SOL in rewards, Marinade said. Those losses will be covered by validator bonds at the end of the epoch.
Solana Foundation VP Tech Jacob Creech pushed back on the characterization of the event. He said the network continued producing blocks, 597 of 699 staked validators kept voting, affected validators recovered within 40 minutes, and validators in the Foundation’s delegation program were unaffected. He described the outcome as evidence that infrastructure diversity worked.
Marinade also turned the analysis inward. The company said four autonomous systems hold two-thirds of the stake distributed by its allocation model, with one of them accounting for 36.94%.
It said it will review concentration limits by network and by data center, and will begin publishing which validators use hot swap and automatic failover.
The last outright Solana halt was in February 2024, and the network took about five hours to restart.