Authors: Jovan Stojkovic, Esha Choukse, Chaojie Zhang, Inigo Goiri, Josep Torrellas

Published on: March 29, 2024

Impact Score: 7.8

Arxiv code: Arxiv:2403.20306

Summary

• What is new: This paper explores unique energy efficiency strategies for serving large language models (LLMs) without compromising performance.
• Why this is important: The need to expand data centers to accommodate LLMs faces the challenge of limited energy availability.
• What the research proposes: The study presents trade-offs and optimization strategies for energy-efficient deployment of LLMs.
• Results: The research demonstrates how varying input, model parameters, and SLAs can impact energy consumption, providing a path to sustainable LLM serving.

Technical Details

Technological frameworks used: LLM inference serving optimization framework

Models used: Various large language models

Data used: Performance and energy consumption data under different configurations

Potential Impact

Data center providers, cloud computing companies, and enterprises utilizing large language models could benefit or face disruption from these insights

Want to implement this idea in a business?

We have generated a startup concept here: EcoServeAI.