Clare Lyle | Notes for my EEML talk on continual learning
For some context: I gave a talk earlier this month at the Eastern European Machine Learning summer school about continual learning. Since I don’t like a good talk prep to go to waste, I’ve adapted the content I covered into a blog post so that folks who missed out on the Montenegrin beaches this summer won’t also miss out on the chance to learn about continual learning. You are attending this talk (reading this blog post) because your brain is able to do something that we have yet to figure out how to replicate in machines: efficiently incorporate new information about the world without erasing prior knowledge. If you were a(n artificial) neural network, updating your parameters using only the content of this blog post would immediately lead to catastrophic drops in performance on everything that is not Clare Lyle’s Blog. But somehow, a mushy ball of mostly fat and water is able to do this using the energy required to power a retro lightbulb for less than a second. More than that, it’s