Authors: Dat Quoc Nguyen, Linh The Nguyen, Chi Tran, Dung Ngoc Nguyen, Dinh Phung, Hung Bui

Published on: November 06, 2023

Impact Score: 8.22

Arxiv code: Arxiv:2311.02945

Summary

• What is new: The introduction of PhoGPT-4B series, a state-of-the-art 4B-parameter generative model series for Vietnamese, including a base and chat variant.
• Why this is important: The lack of publicly available, high-performance generative models for Vietnamese language tasks.
• What the research proposes: Developing the PhoGPT-4B series by pre-training from scratch on a large Vietnamese corpus and fine-tuning for chat applications.
• Results: The PhoGPT-4B series showcases strong performance compared to prior 7B-parameter models, even when being open-sourced.

Technical Details

Technological frameworks used: Open-source generative model series

Models used: PhoGPT-4B (base), PhoGPT-4B-Chat (chat variant)

Data used: 102B tokens Vietnamese corpus for PhoGPT-4B, additional 360K dialogue dataset for PhoGPT-4B-Chat

Potential Impact

Language technology markets, AI chatbot providers, companies targeting Vietnamese-speaking customers, educational technology

Want to implement this idea in a business?

We have generated a startup concept here: VietAI Chat Solutions.