Authors: Sizhe Chen, Julien Piet, Chawin Sitawarin, David Wagner

Published on: February 09, 2024

Impact Score: 8.22

Arxiv code: Arxiv:2402.06363

Summary

• What is new: Introduced structured queries to separate prompts and data, and a fine-tuning strategy for LLMs to ignore user directive instructions in data.
• Why this is important: Prompt injection attacks on Large Language Models (LLMs) that make them follow unintended instructions.
• What the research proposes: A system utilizing structured queries and a specially trained LLM to improve resistance against prompt injection attacks.
• Results: The system showed significant improvement in resisting prompt injection attacks with minimal impact on utility.

Technical Details

Technological frameworks used: Structured queries integration with LLMs

Models used: Specially trained LLM using a novel fine-tuning strategy

Data used: Standard instruction tuning datasets augmented with examples including instructions in data

Potential Impact

Enhanced security in LLM applications could impact markets that rely on language models for text-based tasks, including tech companies providing AI writing aids, search engines, and customer support chatbots.

Want to implement this idea in a business?

We have generated a startup concept here: SecureQueryAI.