BackCAPO-SLT

CAPO-SLT

vivo AI Lab
2026-09-09 11:29:10

CAPO-SLT targets RL instability in sign language translation and posts top Chinese benchmark scores

vivo AI Lab has introduced CAPO-SLT, short for Confidence-Aware Policy Optimization for Sign Language Translation, a method aimed at a specific failure mode in automatic sign language translation: a model can align visual inputs and text reasonably well, yet still drift during token-by-token generation when a locally plausible word is not supported by visual evidence. Once such a word enters the decoding context, the semantic error can compound across the rest of the sentence. The paper keeps the visual encoder and reward design unchanged and instead modifies the policy optimization rule used during reinforcement learning. For positive-advantage tokens, the upper clipping bound is adjusted according to the old policy’s confidence: tokens with higher prior confidence get a more conservative cap, while lower-confidence tokens retain more room for positive updates. The method also limits excessive penalties on negative-advantage tokens to reduce the impact of sentence-level reward noise. On the CSL-Daily test set, CAPO-SLT achieved the best scores in all three reported Chinese sign language translation metrics while using pose-only input. The paper reports gains of 2.93, 1.26 and 0.64 points over Geo-Sign, and 4.96, 3.07 and 3.67 points over pose-only Uni-Sign. The study also reports 41.4 BLEU-1, 15.2 BLEU-4 and 34.9 ROUGE-L on How2Sign, plus 63.77% Per-Instance and 61.91% Per-Class accuracy on WLASL2000.

50
CAPO-SLT targets RL instability in sign language translation and posts top Chinese benchmark scores