Authors: Bowen Tan, Yun Zhu, Lijuan Liu, Hongyi Wang, Yonghao Zhuang, Jindong Chen, Eric Xing, Zhiting Hu

Published on: October 25, 2023

Impact Score: 8.22

Arxiv code: Arxiv:2310.16355

Summary

• What is new: Introduction of RedCoast(Redco), a tool designed to simplify distributed training and inference for large language models (LLMs) with an emphasis on automation and user-friendliness.
• Why this is important: Escalating memory requirements of LLMs necessitate partitioning across multiple GPUs or TPUs, a process complicated by the need for extensive coding and configuration efforts.
• What the research proposes: Redco automates model parallelism with two straightforward rules for generating tensor parallel strategies, and allows customization of diverse ML pipelines through the definition of just three functions.
• Results: Application of Redco on LLM architectures (GPT-J, LLaMA, T5, OPT, up to 66B) demonstrated its effectiveness by significantly reducing the amount of code required for distributed training and inference.

Technical Details

Technological frameworks used: RedCoast(Redco)

Models used: GPT-J, LLaMA, T5, OPT up to the size of 66B

Data used: Not specified

Potential Impact

Machine Learning development tools and platforms market, particularly affecting companies and solutions focusing on the development, training, and deployment of large language models.

Want to implement this idea in a business?

We have generated a startup concept here: OptiML.