Abstract: Meta-learning (ML) has emerged as a promising learning method under resource
constraints such as few-shot learning. ML approaches typically propose a
methodology to learn generalizable models. In this work-in-progress paper, we
put the recent ML approaches to a stress test to discover their limitations.
Precisely, we measure the performance of ML approaches for few-shot learning
against increasing task complexity. Our results show a quick degradation in the
performance of initialization strategies for ML (MAML, TAML, and MetaSGD),
while surprisingly, approaches that use an optimization strategy (MetaLSTM)
perform significantly better. We further demonstrate the effectiveness of an
optimization strategy for ML (MetaLSTM++) trained in a MAML manner over a pure
optimization strategy. Our experiments also show that the optimization
strategies for ML achieve higher transferability from simple to complex tasks.