Micah Goldblum, Assistant Professor of Electrical Engineering at Columbia Engineering, has received research funding from the Columbia Center for AI and Responsible Financial Innovation (CAIRFI) for a project aimed at improving how large language models learn new information after they have already been trained.
The proposal, “Data Synthesis for Continual Learning in Language Models,” addresses a persistent challenge in artificial intelligence: keeping deployed language models current as information changes. Large language models may need to incorporate new developments in software, recent events, or information specific to individual users without being retrained from scratch.
Recent approaches to this problem, known as continual learning, have used techniques such as self-distillation, in which a model with access to new documents helps train another model to internalize that information. Much of the existing research has focused on improving the fine-tuning algorithms themselves, while relying on relatively simple synthetic training data, such as question-and-answer pairs based on individual documents.
Goldblum’s project will investigate whether the way training data is synthesized may play a more important role in continual learning performance than the particular fine-tuning algorithm being used.
The research team plans to develop richer mixtures of synthetic data that require models to work across multiple documents, connect information through several reasoning steps, revise existing beliefs when presented with new evidence, and answer questions for which relevant information may not be immediately obvious. The researchers will combine these datasets with established training approaches, including self-distillation and supervised fine-tuning.
A central goal is to determine whether more sophisticated synthetic data can enable continual learning to improve as the amount of generated training data increases. By contrast, simpler single-document question-and-answer approaches may quickly reach a point where producing additional data provides limited benefits.
As part of the project, Goldblum plans to make the resulting models, data-synthesis pipeline, and evaluation suite openly available, providing resources that other researchers can use to study and advance continual learning in language models.
