Authors: Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, Jun Wang

Published on: February 27, 2024

Impact Score: 7.8

Arxiv code: Arxiv:2402.17453

Summary

• What is new: DS-Agent combines large language models with case-based reasoning to automate data science tasks more efficiently.
• Why this is important: Existing LLM agents often create unreasonable experiment plans for automating data science tasks.
• What the research proposes: DS-Agent utilizes a novel framework that integrates LLMs with case-based reasoning, harnessing expert knowledge from Kaggle for better performance.
• Results: Achieved 100% success rate in development and 36% improvement in deployment with lower costs.

Technical Details

Technological frameworks used: Case-Based Reasoning (CBR)

Models used: GPT-4

Data used: Expert knowledge from Kaggle

Potential Impact

Data science service providers, AI development companies, and businesses reliant on large-scale data analysis could be disrupted or benefit.

Want to implement this idea in a business?

We have generated a startup concept here: DataWise.