Authors: Chi Ian Tang, Lorena Qendro, Dimitris Spathis, Fahim Kawsar, Cecilia Mascolo, Akhil Mathur

Published on: March 30, 2023

Impact Score: 8.22

Arxiv code: Arxiv:2303.17235

Summary

• What is new: The introduction of Kaizen, a training architecture designed specifically for self-supervised continual learning, improving flexibility in leveraging labels and mitigating catastrophic forgetting effectively.
• Why this is important: Existing models trained offline show poor performance in continual learning scenarios due to catastrophic forgetting, and retraining them with new data is inefficient.
• What the research proposes: Kaizen, which incorporates a novel loss function and training process to maintain performance on older tasks while learning new ones, using self-supervised learning principles.
• Results: Kaizen shows up to 16.5% accuracy improvement on split CIFAR-100 and significantly outperforms previous SSL models in continual learning benchmarks.

Technical Details

Technological frameworks used: Self-Supervised Learning (SSL), Continual Learning (CL)

Models used: Kaizen

Data used: Split CIFAR-100

Potential Impact

Tech companies focused on AI and machine learning, particularly those developing systems requiring continual learning abilities such as autonomous vehicles, surveillance systems, and personalized recommendation engines.

Want to implement this idea in a business?

We have generated a startup concept here: EverLearnAI.