Meta-Learning: Teaching Models How to Learn by Miguel Dias

In this presentation, the core ideas behind meta-learning are introduced based on the paper “Model agnostic meta-learning for fast adaptation of deep networks”. It is also discussed why meta-learning remains relevant in the era of foundation models and large-scale pretraining and the presentation finishes with emerging research directions and real-world scenarios where meta-learning offers meaningful advantages over conventional learning approaches.

Key Takeaways

• A meta-model gains general information about the learning process, or meta-knowledge, over multiple learning episodes: It covers a distribution of related tasks VS one task in conventional supervised learning

• Pre-trained foundation models trained on large datasets may generalize poorly when the target dataset is small. Meanwhile Meta-Learning optimizes the model’s intial weights to quickly adapt to a new task with little data.

• Meta Learning can be applied to popular and emerging fields such as Federated Learning and Prompt Learning.

Reference

Finn, C. and Abbeel, P. and & Levine, Sergey. , 2017. Model Agnostic meta-learning for fast adaptation of deep networks.. https://proceedings.mlr.press/v70/finn17a/finn17a.pdf