Authors: Alex Agranovich, Eliya Nachmani, Oleg Rybakov, Yifan Ding, Ye Jia, Nadav Bar, Heiga Zen, Michelle Tadmor Ramanovich

Published on: June 04, 2024

Impact Score: 7.8

Arxiv code: Arxiv:2406.02133

Summary

• What is new: SimulTron introduces a novel, lightweight architecture for real-time speech-to-speech translation on mobile devices, improving upon previous models like Translatotron 1 and 2.
• Why this is important: Accurate, real-time speech-to-speech translation through mobile devices is challenging, impacting effective cross-language communication.
• What the research proposes: SimulTron, using a modified Translatotron framework with streaming capabilities and an adjustable fixed delay, is optimized for mobile use.
• Results: SimulTrron outperforms Translatotron 2 in offline evaluations and Translatotron 1 in real-time settings, showing superior BLEU scores and latency on the MuST-C dataset.

Technical Details

Technological frameworks used: Translatotron

Models used: SimulTron, a direct speech-to-speech translation model

Data used: MuST-C dataset

Potential Impact

Mobile communications, language translation services, international business collaboration platforms

Want to implement this idea in a business?

We have generated a startup concept here: FluentBridge.