Your Last Bug Report Was Probably From a Customer

You know the feeling. A client calls at 6:45pm because their invoice total is wrong. Your IT support digs in, finds a small error in the code that calculates sales tax. It affects three months of records. Your team works late, you apologise to the client, and you promise yourself: we need better testing. But the next release happens under pressure, and testing gets cut again.

If that sounds familiar, you are not alone. Most Malaysian SMEs run with tiny tech teams — sometimes just one developer. And even the best developer cannot predict every scenario, especially when they are also handling support calls and WhatsApp groups. That is exactly the gap Microsoft just addressed by open sourcing code-testing-generator, an AI agent that writes unit tests for your software and proves they actually work.

The agent is available in the MIT-licensed dotnet/skills repository, inside the dotnet-test plugin. It is not a hosted service; it runs inside your existing coding environment, so your code stays private and local.

TL;DR: Microsoft released an open-source AI agent that automatically writes and verifies software tests. In their internal benchmark, it completed 92.1% of tasks versus 78.9% for stock GitHub Copilot on the same model, cutting testing failures by 63%. The biggest wins come from vague, real-world requests and pull-request-specific targets — exactly the situations small teams face daily.

What This Means: An AI That Reads Your Code Before Writing Tests

A unit test is a small script that checks one piece of your software behaves correctly: this function adds prices properly, that login rejects invalid passwords, this delivery fee calculator works for every zone. Unit tests are the difference between a system that quietly breaks and one that tells your developer “something is wrong here” before a customer ever sees it.

For years, AI coding tools only generated tests if you asked precisely. But a prompt like “write tests” is vague. Which file? Which framework? What should the test assert? The new agent solves this by doing research first. It reads your repository, detects the language and test framework, studies existing tests for naming conventions, and finds the real build and test commands. Then it writes tests, runs them, and verifies they can catch bugs. It uses a Research-Plan-Implement (RPI) pipeline — a structured mini-project instead of a one-shot guess.

This matters for a small team: you do not need a senior architect to define standards. The agent learns conventions from the codebase in front of it. And because it never modifies production code, your business logic is safe.

How This Applies to Malaysian SMEs

You may be thinking: “I am not a software company — why does this matter?” But every SME in Malaysia runs on software today, whether it is a restaurant reservation system, an e-commerce storefront, a logistics dispatch app, or an accounting portal. When that software misbehaves, it directly eats your margin and your reputation.

Consider what happens when a payment gateway integration update rolls out without tests. A customer in Penang tries to check out with FPX. The request times out. The order is lost. They take their purchase to a bigger competitor instead. That loss is silent — you never hear about it, you just see lower daily revenue. The new agent can automatically generate tests for every new code change, so a developer knows before a release whether an integration actually responds correctly. Microsoft designed it specifically to find and verify the real test commands, so the checks that run in CI are the same checks that run locally.

Another angle: most Malaysian SMEs outsource development to freelancers or small agencies. One of the biggest headaches is continuity — a freelancer moves on, and your next developer inherits a codebase with no tests and is terrified to touch anything. This open-source agent can backfill tests on untested modules. In Microsoft’s benchmark, it generated 2.3% fewer tests than stock Copilot but achieved effectively identical coverage — 72.4% versus 72.2%. That means fewer, better-targeted checks rather than volume for volume’s sake.

For Malaysian industries with compliance requirements — finance, insurance, healthcare — this is even more relevant. Regulators and auditors care about evidence that systems work correctly. Automated tests are documentation you can point to. The agent maps every requested scenario to a test and runs five verification checks before reporting completion, so what gets committed is far less likely to be sayap (weak). It even avoids tests that call external URLs or depend on timing, which are the common causes of flaky results.

“The key insight: the agent’s biggest gains come from vague prompts — an everyday reality for business owners who do not speak developer jargon. On 89 vague prompts, it resolved 88.8% versus 66.3% for stock Copilot, cutting failures from 30 to 10.”

The Numbers Behind the Claims

Metric	code-testing-generator	Stock GitHub Copilot
Task completion (152-task benchmark)	140 (92.1%)	120 (78.9%)
Completion on vague prompts	79/89 (88.8%)	59/89 (66.3%)
Completion on diff-targeted tasks	15/15 (100%)	0/15 (0%)
Average task time	359 seconds	380 seconds
Line coverage	72.4%	72.2%
Tests generated	6,963	7,129

Data from Microsoft’s internal 152-task benchmark as reported by MarkTechPost: tasks were drawn from real repositories, and both setups used the same model and prompts. The agent also scored higher across multiple model backends — on 45 .NET tasks, Claude Opus 4.8 reached 43/45 with the agent versus 35/45 stock, and GPT-5.5 reached 41/45 versus 36/45.

Practical Takeaways For Your Business

• If you have a tech team, make sure they know this exists. An open-source tool that writes and verifies tests means developer time goes back to features that grow revenue, not manual test-writing.
• If you outsource development, add one simple requirement to your next scope. Ask for automated tests on any changed module. Now your vendor has no excuse — the tool is available globally.
• If you are preparing for an audit or seeking external investment, automated tests are a cheap way to show your system is stable and your team follows disciplined practices.
• Start with one risky module. Payment flow, inventory count, or delivery fee calculator — each is a high-impact, low-complexity area to start automating.
• Remember it is built for small teams. Microsoft explicitly states it is viable from solo maintainers upward — which is exactly the staffing reality of most Malaysian SMEs.

The Bigger Picture: Quality Becomes Automatic

What we are seeing is a shift from generation to verification. Earlier AI tools produced code fast but left quality up to the developer. This agent does something more important: it proves the code still behaves correctly before a release. As this style of agent becomes standard, the cost of producing reliable software drops dramatically. That is good news for any owner who has been burned by a “minor update” that somehow took down the whole system.

Long-term, the bottleneck for Malaysian SMEs will no longer be “we do not have a senior developer to write tests.” It will be “we have not told our systems to do it yet.” Your competitors are already adopting AI quality agents — their apps will break less often while you are still apologising for last month’s bug. The Microsoft open-source release is one concrete way to close that gap now, with your code staying in your own hands.

Ready to Streamline Your Operations?

Your business should run itself. AutoRunBiz deploys AI agents to automate your daily operations — WhatsApp orders, invoicing, customer follow-ups, and accounting. Book a free 15-min ops audit to see where automation fits your business →