MemoryINP: memory enhanced neural processes for understanding-based informed meta-learning

Although informed machine learning and meta-learning have been studied for many years as ways to improve the inductive biases of neural networks and enhance their performance under limited data availability, it has not been until recently that the two paradigms have been combined in the form of informed meta-learning. Despite its high potential for automating knowledge incorporation and enabling rapid adaptation to novel tasks, current informed meta-learning methods do not leverage shared knowledge across similar tasks, which could further enhance performance. In this paper, we introduce MemoryINP, a novel model architecture based on Neural Processes that enables explicit knowledge sharing across tasks. The model uses a fully differentiable memory to read from and write to task-specific knowledge representations, allowing it to transfer relevant information based on task similarity. Preliminary results on regression benchmarks highlight the effectiveness of MemoryINP in low-data regimes and reducing uncertainty in the presence of observational noise.