Meta-ensemble learning with a multi-headed model for few-shot problems

Abstract

Recent meta-learning algorithms for few-shot learning are based on episodic training where each episode consists of only a few support and query samples to imitate a target few-shot task. However, due to the limited number of categories and few samples in each category, this framework suffers from over-fitting to both a meta-training dataset and the support set of each episode. It also causes a large variance in the accuracy of each episode, which reduces reliability and confidence in model performance. To address this problem, we propose a novel meta-ensemble learning approach based on a recent ensemble method: a multi-input multi-output (MIMO) configuration. Our approach is simply applied to existing meta-learning algorithms. Multiple subnetworks in a single model simultaneously learn multiple episodes and ensemble the predictions, leveraging the model capacity. We show that meta-ensemble learning achieves significant improvement in generalization. It also improves the performance of meta-learning algorithms on few-shot classification benchmarks.

Keywords

• Computer vision
• Few-shot learning
• Meta learning