Why Your AI Workflow May Be Slowing Your Business Down

You may already be using AI for customer replies, document summaries, sales enquiries or internal operations. The problem often appears after the first successful experiment: one model handles simple requests, another handles difficult reasoning, and a third may be used for fast follow-up conversations.

That sounds sensible, but switching between AI models can create hidden delays. The new model may need to read the entire conversation again before it can continue. For a Malaysian SME, this can mean slower replies, less consistent service and more pressure on your team when several customer conversations are active at once.

A recent Nvidia research project suggests that some of this repeated work can be avoided. Instead of making a new model rebuild the whole conversation memory, a mathematical mapping can transfer useful information from one model to another. The reported approach ran between 2.7 and 25 times faster than recomputing the conversation on compatible model pairs, while retaining up to 98% of the target model’s standalone accuracy. Source: VentureBeat

TL;DR

AI systems often lose time when they hand a long conversation from one model to another because the receiving model must process the history again.

Nvidia researchers found that simple linear mathematics can transfer part of the model’s working memory between compatible models, potentially making long-running AI workflows faster and more practical for business use.

What This Means

To understand the idea, imagine an AI assistant managing a customer enquiry. It first reads the customer’s messages, your company policy, product information and previous replies. This initial reading is called the “prefill” stage. The system stores a temporary working memory called a Key-Value, or KV, cache, which helps it respond without repeatedly rereading everything.

During a short conversation, this memory saves time. However, the problem appears when your system changes models. For example, a smaller model might handle a simple product question, while a larger model takes over when the customer asks for a detailed comparison or a customised recommendation. Because the two models organise information differently, the second model normally has to process the full conversation from the beginning.

This repeated processing becomes more noticeable as the conversation grows. A long support case may contain product details, delivery information, customer preferences, previous troubleshooting steps and internal instructions. If every handoff starts from zero, your workflow becomes slower even though the information has already been read.

Nvidia’s researchers studied whether the first model’s KV cache could be mathematically converted into a format the second model understands. Their method uses per-head ridge regression, selected information from different layers and a technique that removes position-specific encoding before mapping the information. It is not a new business application by itself. Rather, it is infrastructure that could help developers build more efficient multi-model AI systems.

Research finding	Reported result	Why it matters to you
Speed compared with recomputing the conversation	2.7 to 25 times faster	Potentially quicker AI handoffs and responses
Accuracy retained on compatible model pairs	Up to 98%	Less need to choose between speed and response quality
Calibration data used in testing	500 sequences of 1,024 tokens each	Shows the mapper did not require a huge calibration set
Model families tested	Qwen3, Llama 3.1 and Ministral 3	Initial use is focused on compatible model families
Largest model-size transition tested	Llama 3.1 8B to 70B	Shows the approach was tested across a substantial size difference

All figures in the table are reported in the VentureBeat coverage of Nvidia’s research. Source: VentureBeat

The practical lesson is simple: changing AI models should not mean throwing away the conversation memory that has already been processed.

How This Applies to Malaysian SMEs

Customer service is the clearest use case. Suppose you operate an online store, repair service, clinic, training centre or local distributor. A smaller AI model could answer routine questions about operating hours, product availability, appointment preparation or delivery areas. When the customer asks about a complicated warranty issue or a combination of products, the workflow could transfer the conversation to a stronger model. If the handoff preserves the useful context, the customer should not need to repeat the same information.

This is especially useful when your customers communicate in mixed language. A customer may begin in English, switch to Bahasa Malaysia and include informal shorthand or industry terms. A smooth handoff could preserve the conversation context rather than forcing the next model to reconstruct it imperfectly. You would still need proper testing for Bahasa Malaysia, Chinese, Tamil and industry-specific vocabulary, because the reported research initially focuses on compatible model families rather than every language and model combination.

Sales qualification can also benefit. A lightweight model could collect basic information from a lead: business type, preferred product, required quantity, delivery location and desired timeline. If the lead asks for a detailed quotation explanation or a comparison between several solutions, a larger model could take over. The important point is that the stronger model would ideally receive the existing qualification context instead of asking the lead to answer the same questions again.

Document-heavy work is another practical area. Your team may ask AI to review supplier terms, summarise tender requirements, compare purchase specifications or extract information from a long PDF. A larger model may be useful for understanding the document initially, while a smaller and faster model handles follow-up questions such as “Which section mentions delivery?” or “List the required attachments.” A transferred working memory could make that sequence feel more responsive.

Internal operations may become more structured. An SME could use one model to interpret a staff request, another to check company procedures and a third to format the result into a task or approval note. For example, a sales representative might ask for a delivery exception. The system could preserve the original order details, customer explanation and relevant policy while moving the decision step to a more capable model. This reduces the risk of missing context during automation.

There are important limits. Nvidia’s experiment focused first on model pairs within compatible families, where the models share important architectural characteristics. You should not assume that any two AI providers can exchange their internal memory. In many cases, the safest approach remains transferring a clean, structured summary rather than an internal KV cache.

Practical Takeaways for Your Business

• Map your current AI handoffs. List every point where one model, software tool or automation service passes work to another.
• Measure the context being repeated. Track how much customer, product or document information the second step must reread.
• Separate routine and complex work. Use a faster model for predictable tasks and reserve stronger reasoning for cases that genuinely need it.
• Keep structured business data outside the model. Store order numbers, customer IDs, product codes and approval status in your business system rather than relying only on conversation memory.
• Test handoff quality with real examples. Include Bahasa Malaysia, mixed-language messages, abbreviations, spelling mistakes and incomplete customer questions.
• Check privacy requirements. Avoid sending sensitive personal, payroll or financial information into an AI workflow unless your controls and provider arrangements are appropriate.
• Ask vendors specific questions. Find out whether they preserve context between model changes, use summaries, or make the receiving model process the entire conversation again.
• Start with one workflow. Customer support, lead qualification or document review is easier to measure than an organisation-wide rollout.
• Use fallback summaries. If direct memory transfer is not supported, create a concise, structured handoff containing the customer’s goal, known facts, actions taken and unresolved questions.

What You Should Ask Before Adopting This Approach

First, ask whether the models in your workflow are compatible. The reported method was tested across selected model families, including Qwen3, Llama 3.1 and Ministral 3, rather than across every available model. Source: VentureBeat

Second, ask how accuracy is measured. A fast response is not useful if it forgets a delivery condition, changes a product specification or gives the wrong procedural advice. Test the output against a defined set of real cases and have a staff member review difficult examples.

Third, ask what happens when the transfer fails. Your system should have a clear fallback, such as rebuilding the context, using a structured summary or routing the case to a person. Never design an important customer workflow around a single untested handoff technique.

The Bigger Picture

This research points towards a more flexible way to design AI systems. Instead of choosing one large model for every task, businesses may combine different models according to the job: a smaller model for simple, high-volume interactions and a larger model for complex analysis. The challenge has always been moving between them without losing time or context.

The research also shows why AI performance is not only about model size. The way systems manage memory, context and handoffs can have a direct effect on the experience your customers and employees receive. A well-designed workflow with sensible routing may perform better operationally than a single model used for everything.

For Malaysian SMEs, this does not mean you need to build a research system yourself. It means you should pay closer attention to the workflow behind an AI feature. When evaluating an automation provider, ask how it handles long conversations, model switching, document context, data protection and failure recovery.

Over time, AI tools may make model selection invisible to you. Your assistant could quietly use a fast model for routine questions, move difficult cases to a stronger model and preserve the relevant context throughout. The business benefit would be a smoother customer journey and less repetitive work for your team.

For now, the sensible step is to identify where your business is making AI repeat work it has already completed. That is where better context management, structured handoffs and newer model-transfer techniques may offer the most practical improvement.

Ready to Streamline Your Operations?

Your business should run itself. AutoRunBiz deploys AI agents to automate your daily operations — WhatsApp orders, invoicing, customer follow-ups, and accounting. Book a free 15-min ops audit to see where automation fits your business →