Why Your AI Agent’s Memory Dose Matters
If you run a Malaysian SME, you may already use AI to reply to customers, prepare quotations, organise appointments, summarise documents or support your sales team. The next question is often: should you give the AI agent access to every previous instruction, conversation and lesson?
New research from IBM Research suggests that more memory is not automatically better. The useful amount depends on the model, the task and how the information is selected. The study evaluated agentic memory across eight models and found three broad patterns: stronger models with room to improve benefited from a complete guideline set, weaker models performed better with a compact core and task-specific retrieval, while already-saturated models showed no measurable improvement. Source: IBM Research via Hugging Face
For you, this has a practical implication. When an AI agent gives an incorrect answer, the solution may not be to add more company documents to its prompt. It may be better to give it fewer, more relevant operating rules at the exact moment they are needed.
What Happened
IBM Research examined ALTK-Evolve, a system that allows an agent to learn from its own previous work without changing the model’s underlying weights. The system reviews successful and unsuccessful task trajectories, extracts reusable guidelines, consolidates them and presents those guidelines during future tasks. In other words, the agent does not permanently retrain itself; it receives better guidance around the existing model. Source: IBM Research via Hugging Face
The researchers tested the approach on AppWorld, a benchmark containing 585 multi-step tasks across nine simulated applications, including calendars, messaging and payments. They compared three approaches: no memory, the full guideline set added at every reasoning step, and curated retrieval that combined a fixed high-confidence core with a few task-relevant guidelines. Source: IBM Research via Hugging Face
The results were not uniform. The 117-billion-parameter gpt-oss-120b model improved task completion by 16.1 percentage points with curated retrieval, while its full guideline configuration produced a smaller improvement and used approximately 51% more tokens than the no-memory baseline. DeepSeek-V3.2 improved task completion by 9.5 percentage points with the full guideline set, but its stricter scenario completion score rose by 16.1 percentage points. Source: IBM Research via Hugging Face
Other models showed smaller gains. Claude Opus 4.6 improved task completion by 4.1 percentage points and scenario completion by 7.1 percentage points with the full guideline set. GPT-5.5 improved by 2.9 and 7.2 percentage points respectively. GLM-5 showed no measurable improvement in the reported test. Source: IBM Research via Hugging Face
Why This Matters for Malaysian SMEs
Many Malaysian businesses operate with a small team handling a wide range of responsibilities. A property agency may need AI to qualify leads, arrange viewings and follow up with prospects. A clinic may need help managing appointment enquiries while following communication rules. A distributor may want an agent to check stock, prepare a quotation and escalate unusual requests. These tasks involve different information, so sending one large instruction set to every interaction can make the agent less focused.
Imagine that your customer asks, “Can I change tomorrow’s delivery address?” The agent may need your delivery-change policy, the order record and the escalation rule for goods already dispatched. It does not need your entire employee handbook, every historical customer complaint or all product descriptions. A selective memory system can bring forward the relevant rules while leaving unrelated material out of the active context.
This is especially useful when your business handles both English and Bahasa Malaysia, local delivery instructions, service-area restrictions and industry-specific requirements. You can create a compact core of rules that always applies, such as tone of voice, approval limits and privacy boundaries. Then you can retrieve additional guidance based on the task, such as “refund request”, “late delivery”, “quotation follow-up” or “appointment rescheduling”.
The research also highlights an important operational point: memory should be tested by business outcome, not by how much information the agent can see. For an SME, that means measuring whether enquiries are routed correctly, whether staff receive complete handovers and whether customers get consistent answers. A longer prompt is not a performance metric.
Key lessons for your AI workflow
| Situation | Practical approach | What to monitor |
|---|---|---|
| New or less capable agent | Use a small trusted rule set plus task-specific retrieval | Wrong answers, irrelevant instructions and missed steps |
| Strong agent with recurring edge cases | Test whether a broader guideline set improves completion | Exceptions handled correctly and fewer escalations |
| Agent already performing well | Identify the remaining failure modes before adding more memory | Whether additional guidance changes results at all |
| Multi-step operational task | Retrieve instructions at each relevant stage | End-to-end completion, not just the first response |
How You Can Apply This in Your Business
Start by listing the ten tasks where your AI assistant makes the biggest difference. Examples include answering product questions, checking order status, drafting service replies, qualifying leads and preparing internal summaries. For each task, record the rules the agent must always follow and the information it only needs sometimes.
Next, separate permanent guidance from situational guidance. Permanent guidance might say that the agent must not confirm a refund without checking the approved workflow. Situational guidance might explain how to handle a damaged parcel, a corporate account or a customer requesting delivery outside your normal service area.
Then build a simple evaluation set using real but anonymised examples. Include routine requests, ambiguous requests and difficult exceptions. Compare the agent with no memory, with all guidelines and with a selected set. Record the result in a spreadsheet. You want evidence that the chosen memory method improves completed tasks and reduces avoidable handovers.
“Agentic memory is not a feature you switch on. It is a dose you calibrate to the model.” Source: IBM Research via Hugging Face
Do not allow an agent to learn from every past interaction without review. A previous response may have been successful because a staff member corrected it manually, or because the customer accepted an incomplete answer. Before turning a past lesson into a reusable guideline, check whether it is accurate, safe and relevant to your current process.
The Bigger Picture
The research points towards a more disciplined way to build AI automation. Instead of treating memory as a large archive, businesses can treat it as an operational control layer. The agent model provides general capabilities, while your approved guidelines shape how those capabilities are applied to your workflows.
This approach can also make AI projects easier to maintain. When your return policy, delivery process or lead qualification criteria changes, you update the relevant guidance instead of rebuilding the entire system. You can also identify which rules are genuinely useful by measuring whether they improve task completion.
For Malaysian SMEs, the best starting point is not the most advanced model or the largest knowledge base. It is a clearly defined workflow with approved rules, realistic test cases and a review process for failures. Your AI agent should know what to remember, when to use it and when to ask a human.
As AI agents move from answering questions to completing multi-step business tasks, memory selection will become an important part of automation design. The winning setup will not necessarily be the one with the most information. It will be the one that gives your agent the right instruction, in the right context, at the right time.
Source: IBM Research, “How Much Memory Does Your Agent Actually Need?”
Ready to Streamline Your Operations?
Technology moves fast. Your operations should keep up. AutoRunBiz builds AI systems that run your daily workflows — from WhatsApp order capture to accounting. Book a free 15-min ops audit →