Model-based meta-learning in neural networks

Aluno: Willian Wang

Orientadora: Nina S. T. Hirata

Meta-learning, often referred to as “learning to learn,” aims to improve the performance and efficiency of learning algorithms by enabling them to learn more effectively new tasks. This approach is particularly relevant in the context of deep learning, where classic challenges such as data and computational bottlenecks often limit model performance and training speed.^[1]

Unlike optimization-based methods, which focus on improving gradient backpropagation, and metric-based methods, which seek to find an optimal metric function for a specific problem, model-based (or “black box”) meta-learning relies primarily on its architecture. This approach leverages the design of the network to update its weights more efficiently.

The goal of this study is to focus on model-based methods of meta-learning within neural networks. It will examine a range of historically significant architectures in deep learning to establish a foundation, explore recent advancements in meta-learning using these techniques, and implement a few pertinent algorithms to understand them better. The plan is:

1. Study of Memory-Augmented Neural Networks (MANNs): These techniques are particularly important to this work because there are multiple attempts of using memory storage enhanced networks to improve the learning process.

2. Analysis of Meta-Learning Methods Relative to Conventional Techniques: This involves a detailed examination of meta-learning models influenced by established frameworks, including MANNs and other models. ^[2][3]

3. Investigation of Non-Traditional Meta-Learning Techniques: This includes the study of strategies that diverge from mainstream algorithms, such as hypernetworks and self-referential networks ^[4][5]

4. Selective Implementation of Algorithms: Given the impracticality of implementing every algorithm, a few selected algorithms will be implemented to understand this field better.