Large language models (LLMs) for artificial intelligence (AI) often struggle with long inputs. Traditional models handle limited context windows, but real tasks like analyzing huge documents or long conversations need better ways. To fix this, MIT researchers have introduced recursive language models (RLMs).
The researchers have published an arXiv preprint titled "Recursive Language Models." Researcher Alex Zhang has posted a research summary and a codebase to Github. AI commentator Matthew Berman has posted a YouTube video titled "MIT Researchers DESTROY the Context Window Limit."
RLMs let an AI call itself or other models repeatedly during processing. This acts like a wrapper around the main model, treating input as a variable in a coding setup, allowing the AI to peek, search, or split data without seeing everything at once.
In one setup, RLMs use a loop where code runs interactively and stores results. The main model gets the query but not the full context; instead, it interacts with the environment to run code, grep for keywords (search for patterns), or make sub-calls to smaller models. When done, it outputs a final answer. This avoids clogging the model's memory and scales to millions of tokens.
Overcoming context challenges
Tests on benchmarks like OOLONG, which involves answering questions over thousands of data entries, show RLMs outperform standard models. For example, an RLM using a smaller model beat a larger one by over double the correct answers on hard tasks, at similar cost. On BrowseComp-Plus, with multi-hop queries needing info from many documents, RLMs handled 1000 documents well, maintaining high accuracy where others failed due to size limits.
RLMs also shine in patterns like summarizing chunks or applying code to tasks, such as tracking long file changes. Unlike agents that follow fixed steps, RLMs let the AI decide how to decompose problems. Future work could train models for this, improving efficiency. This approach promises better AI for real-world scenarios with vast data, without needing huge context windows.