Small AI That Works Without the Internet
You’ve got a chatbot that answers customer questions, but only when it can reach the cloud. You’ve thought about adding voice commands to your shop or warehouse, but every option you find asks for a subscription, a fast internet line, and a monthly bill that climbs with every extra user. When the line drops, the AI goes quiet. You end up with something that looks smart and fails at the worst possible moment.
That’s the trap of big AI models. They need big servers. They need connections. They need constant upkeep. But there’s another path — a tiny model that fits inside a 14MB file and runs a full session in about 28MB of RAM, even on a Raspberry Pi.
Cactus Compute’s Needle 2 is an open 45-million-parameter model built for one specific job: turning a messy sentence into a clean, typed action. No open-ended conversation, no world knowledge, no poetry. Just “turn on the aircond” mapped to a command your system understands.
TL;DR: Needle 2 is a 45M-parameter tool-calling AI in a 14MB binary that runs on hardware without a GPU and without an internet connection. It extracts fields from invoices, triggers voice actions, and routes uncertain requests to the cloud only when needed. For Malaysian SMEs, this means AI features you can actually deploy on the phone, tablet, or kiosk you already own.
What This Means
Most AI tools you’ve seen rely on massive models with billions of parameters. Cactus’s own report says a comparable model, LFM2.5-230M, was pretrained on 19 trillion tokens. By contrast, Needle 2 used a proprietary 115B-token corpus with 38B tokens of post-training. It’s a fraction of the size, and that’s intentional.
The team’s premise is simple: “mapping a messy sentence onto a typed function signature needs no world knowledge and no open-ended prose.” In plain language — when a customer says “please book a table for two at 8pm,” you don’t need the AI to know the capital of France. You need it to fill in a form: party_size=2, time=20:00, action=book_table.
Needle 2 uses what they call a Simple Attention Network — 27 layers, 512 wide — with a 256-token sliding window. It keeps system instructions and tool declarations pinned as “KV sinks,” so memory stays near 28MB regardless of conversation length. That’s a fixed ceiling, not a growing curve.
How This Applies to Malaysian SMEs
Think about your own operations. You are a business owner with 1 to 50 employees. You don’t have a server room, and you don’t want to babysit a model. What you have is old desktop computers, work phones, or a tablet at the counter. That’s exactly the hardware Needle 2 targets — sub-$200 phones, low-end mobile, and even a Raspberry Pi 5, where it reportedly pushes 500 tokens/sec.
Take a retail shop with a POS system. You deal with supplier invoices, delivery orders, and warranty forms. Every one of those requires manual entry into a computer. Needle 2 can do what the developers call “receipt and invoice field extraction” — reading a messy document and pulling out the supplier name, date, amount, and item codes into a structured format. In Malaysia, where many suppliers still send paper invoices via WhatsApp photos, this becomes a real way to cut data-entry time without building a custom OCR pipeline.
Then think about voice-to-action on screenless devices. A typical SME office has control panels, lighting timers, or even air-conditioning remotes. With a 14MB model sealed inside a C++ engine, you could build a voice trigger on a cheap Android phone placed at the front desk. “Call the delivery driver” or “open the gate” becomes a local, offline action. The company says Pebble already runs Needle locally in its Index 01 app for offline voice actions, and Cactus lists smart home, wearables, low-end mobile, and retail kiosks as target industries.
Even the escalation logic fits real Malaysian usage habits. Needle 2 returns a confidence value with every response. Off-topic requests return an empty call []. Your edge device can decide: act above a confidence threshold, or escalate to a bigger cloud model when confidence drops. That means a small restaurant chain could run a local ordering bot on a Pi, and only send tricky requests to the cloud when needed. No constant connection. No data leaving your shop for trivial commands.
“A model that fits in 28MB of RAM might look like a toy next to the giant LLMs. But for a business owner, it’s a tool that actually gets used without breaking your workflow.”
Practical Takeaways: What You Can Do Today
- Inventory your repetitive actions. List the 3 to 5 things your staff do repeatedly — extracting invoice numbers, tagging complaints, routing customer messages to departments. Each one can be a “tool” for a local model.
- Start with the field-extraction case. If you receive vendor invoices in multiple formats, test whether a 45M-parameter model can pull the key fields into a spreadsheet. Cactus notes Needle 2 ships as a prebuilt binary and static library for Linux, Windows, Android, iOS, and WebAssembly, so you or a freelance developer can embed it without installing a runtime.
- Keep the tool count under five. Below five tools, the model renders them directly. Above that, it uses retrieval to pick the top five. For a small business, five actions are enough — book, check, open, add, call.
- Use confidence as an auto-escalation trigger. Design the system so every low-confidence request falls back to a human or a larger cloud model. That way you control cost and maintain customer experience.
- Test on old equipment. If you have an unused Android phone or a single-board computer, that’s your demo hardware. The model ships as a 14MB binary, so there’s no “runtime to install and no download at inference time” — the weights never even decompress into RAM.
The Numbers Behind Needle 2
Cactus evaluated Needle 2 on five public function-calling benchmarks using “ordered strict exact match” — names, order, and every argument must match. Needle 2 runs end-to-end through the shipped engine at CQ2-bit, while baselines ran at f16 under vLLM. Here’s how it compares:
| Benchmark | Needle 2 (CQ2) | LFM2.5 230M | FunctionGemma 270M | Apple FM |
|---|---|---|---|---|
| Mobile Actions (961) | 63.7 | 69.1 | 64.0 | 57.6 |
| DroidCall (200) | 17.0 | 11.0 | 17.5 | — |
| Seal-Tools in-domain (700) | 32.6 | 26.9 | 16.3 | — |
| Seal-Tools OOD (654) | 28.7 | 17.0 | 15.6 | — |
| BFCL v4 single-turn (3,641) overall | 42.6 | 60.8 | 46.1 | 61.7 |
Needle 2 leads both Seal-Tools splits, posts a 98.3 function-name accuracy on Mobile Actions, and has a well-formed output rate of 93.4 across the BFCL rows. It trails on BFCL v4 overall — Cactus attributes that to distribution: “its corpus is consumer device actions, not general or enterprise APIs.” Source: MarkTechPost
The Bigger Picture
What matters here isn’t the benchmark score. It’s the direction. The story of AI for small business has been “pay a subscription, get a cloud API, and stay forever connected.” Needle 2 points a different way: a model so small and so specific that a business owner can own it outright.
When AI costs are tied to per-token cloud usage, every conversation can surprise you. When AI runs locally, the connection is the only variable you remove. In Malaysia, where internet coverage varies — from offices in Klang Valley to warehouses in industrial estates with flaky connections, to rural branches — an offline tool that does one thing well protects your operations instead of adding another point of failure.
This is not a revolution. It’s a return to basics: your business probably doesn’t need a model that can write a poem and file your taxes in Malay, English, and Chinese simultaneously. You need a model that can read a delivery order, match it to a customer name, and log it into your system. That’s the kind of boring, dependable, useful AI that keeps a business running.
Long-term, expect to see more narrow models like this embedded in point-of-sale software, accounting tools, and even aircond control systems sold in Malaysia. The smart move is to start identifying which of your current manual tasks are really just sentence-to-function mappings. Those are the first candidates for a 28MB conversation that never drops.
Check out the GitHub repo, Hugging Face model card, and the Cactus Needle page if you want to explore further. For a deeper technical look, the Underlying study on attention-only transformers is referenced in the article.
Ready to Streamline Your Operations?
Your business should run itself. AutoRunBiz deploys AI agents to automate your daily operations — WhatsApp orders, invoicing, customer follow-ups, and accounting. Book a free 15-min ops audit to see where automation fits your business →