RAG isn't all the time quick sufficient or clever sufficient for contemporary agentic AI workflows. As groups transfer from short-lived chatbots to long-running, tool-heavy brokers embedded in manufacturing methods, these limitations have gotten tougher to work round.
In response, groups are experimenting with different reminiscence architectures — generally known as contextual reminiscence or agentic reminiscence — that prioritize persistence and stability over dynamic retrieval.
One of many more moderen implementations of this strategy is "observational reminiscence," an open-source know-how developed by Mastra, which was based by the engineers who beforehand constructed and offered the Gatsby framework to Netlify.
In contrast to RAG methods that retrieve context dynamically, observational reminiscence makes use of two background brokers (Observer and Reflector) to compress dialog historical past right into a dated remark log. The compressed observations keep in context, eliminating retrieval completely. For textual content content material, the system achieves 3-6x compression. For tool-heavy agent workloads producing massive outputs, compression ratios hit 5-40x.
The tradeoff is that observational reminiscence prioritizes what the agent has already seen and determined over looking a broader exterior corpus, making it much less appropriate for open-ended information discovery or compliance-heavy recall use instances.
The system scored 94.87% on LongMemEval utilizing GPT-5-mini, whereas sustaining a totally steady, cacheable context window. On the usual GPT-4o mannequin, observational reminiscence scored 84.23% in comparison with Mastra's personal RAG implementation at 80.05%.
"It has this nice attribute of being each easier and it’s extra highly effective, prefer it scores higher on the benchmarks," Sam Bhagwat, co-founder and CEO of Mastra, informed VentureBeat.
The way it works: Two brokers compress historical past into observations
The structure is easier than conventional reminiscence methods however delivers higher outcomes.
Observational reminiscence divides the context window into two blocks. The primary comprises observations — compressed, dated notes extracted from earlier conversations. The second holds uncooked message historical past from the present session.
Two background brokers handle the compression course of. When unobserved messages hit 30,000 tokens (configurable), the Observer agent compresses them into new observations and appends them to the primary block. The unique messages get dropped. When observations attain 40,000 tokens (additionally configurable), the Reflector agent restructures and condenses the remark log, combining associated gadgets and eradicating outmoded info.
"The way in which that you simply're form of compressing these messages over time is you're truly simply form of getting messages, after which you could have an agent form of say, 'OK, so what are the important thing issues to recollect from this set of messages?'" Bhagwat mentioned. "You type of compress it, and then you definately get in one other 30,000 tokens, and also you compress that."
The format is text-based, not structured objects. No vector databases or graph databases required.
Steady context home windows reduce token prices as much as 10x
The economics of observational reminiscence come from immediate caching. Anthropic, OpenAI, and different suppliers scale back token prices by 4-10x for cached prompts versus these which might be uncached. Most reminiscence methods can't reap the benefits of this as a result of they modify the immediate each flip by injecting dynamically retrieved context, which invalidates the cache. For manufacturing groups, that instability interprets immediately into unpredictable value curves and harder-to-budget agent workloads.
Observational reminiscence retains the context steady. The remark block is append-only till reflection runs, which implies the system immediate and present observations type a constant prefix that may be cached throughout many turns. Messages hold getting appended to the uncooked historical past block till the 30,000 token threshold hits. Each flip earlier than that may be a full cache hit.
When remark runs, messages are changed with new observations appended to the present remark block. The remark prefix stays constant, so the system nonetheless will get a partial cache hit. Solely throughout reflection (which runs occasionally) is your complete cache invalidated.
The typical context window measurement for Mastra's LongMemEval benchmark run was round 30,000 tokens, far smaller than the total dialog historical past would require.
Why this differs from conventional compaction
Most coding brokers use compaction to handle lengthy context. Compaction lets the context window fill all the way in which up, then compresses your complete historical past right into a abstract when it's about to overflow. The agent continues, the window fills once more, and the method repeats.
Compaction produces documentation-style summaries. It captures the gist of what occurred however loses particular occasions, choices and particulars. The compression occurs in massive batches, which makes every move computationally costly. That works for human readability, nevertheless it usually strips out the precise choices and gear interactions brokers have to act persistently over time.
The Observer, however, runs extra regularly, processing smaller chunks. As an alternative of summarizing the dialog, it produces an event-based choice log — a structured listing of dated, prioritized observations about what particularly occurred. Every remark cycle handles much less context and compresses it extra effectively.
The log by no means will get summarized right into a blob. Even throughout reflection, the Reflector reorganizes and condenses the observations to search out connections and drop redundant knowledge. However the event-based construction persists. The end result reads like a log of choices and actions, not documentation.
Enterprise use instances: Lengthy-running agent conversations
Mastra's prospects span a number of classes. Some construct in-app chatbots for CMS platforms like Sanity or Contentful. Others create AI SRE methods that assist engineering groups triage alerts. Doc processing brokers deal with paperwork for conventional companies shifting towards automation.
What these use instances share is the necessity for long-running conversations that preserve context throughout weeks or months. An agent embedded in a content material administration system must do not forget that three weeks in the past the consumer requested for a selected report format. An SRE agent wants to trace which alerts have been investigated and what choices have been made.
"One of many large objectives for 2025 and 2026 has been constructing an agent inside their internet app," Bhagwat mentioned about B2B SaaS firms. "That agent wants to have the ability to do not forget that, like, three weeks in the past, you requested me about this factor, otherwise you mentioned you needed a report on this sort of content material sort, or views segmented by this metric."
In these situations, reminiscence stops being an optimization and turns into a product requirement — customers discover instantly when brokers overlook prior choices or preferences.
Observational reminiscence retains months of dialog historical past current and accessible. The agent can reply whereas remembering the total context, with out requiring the consumer to re-explain preferences or earlier choices.
The system shipped as a part of Mastra 1.0 and is out there now. The group launched plug-ins this week for LangChain, Vercel's AI SDK, and different frameworks, enabling builders to make use of observational reminiscence exterior the Mastra ecosystem.
What it means for manufacturing AI methods
Observational reminiscence presents a special architectural strategy than the vector database and RAG pipelines that dominate present implementations. The easier structure (text-based, no specialised databases) makes it simpler to debug and preserve. The steady context window allows aggressive caching that cuts prices. The benchmark efficiency means that the strategy can work at scale.
For enterprise groups evaluating reminiscence approaches, the important thing questions are:
How a lot context do your brokers want to keep up throughout periods?
What's your tolerance for lossy compression versus full-corpus search?
Do you want the dynamic retrieval that RAG gives, or would steady context work higher?
Are your brokers tool-heavy, producing massive quantities of output that wants compression?
The solutions decide whether or not observational reminiscence suits your use case. Bhagwat positions reminiscence as one of many prime primitives wanted for high-performing brokers, alongside software use, workflow orchestration, observability, and guardrails. For enterprise brokers embedded in merchandise, forgetting context between periods is unacceptable. Customers anticipate brokers to recollect their preferences, earlier choices and ongoing work.
"The toughest factor for groups constructing brokers is the manufacturing, which may take time," Bhagwat mentioned. "Reminiscence is a extremely necessary bit in that, as a result of it's simply jarring when you use any form of agentic software and also you form of informed it one thing after which it simply type of forgot it."
As brokers transfer from experiments to embedded methods of document, how groups design reminiscence could matter as a lot as which mannequin they select.
