Meet the AI That Reads Your Screens and Does the Work

Meet the AI That Reads Your Screens and Does the Work — featured image

by

Stop Copy-Pasting. This AI Reads Your Screens

You know the drill. A stack of delivery orders arrives. Your staff opens one system, reads a number, switches to another window, types it in. Then repeats it for the next order. At the end of the day, someone’s eyes are sore, and you still aren’t sure if the numbers match.

That’s the hidden burden of running a small business in Malaysia. Not the grand strategy, but the small repetitive work: copying invoice numbers, checking name spellings, filling the same government form every month.

A new AI model from Liquid AI might change how you handle that burden. It’s small enough to run on a laptop or phone, it can look at a screen like you do, understand what it sees, and even take action. So let’s talk about what it actually means for your daily operations.

TL;DR

  • LFM2.5-VL-3B is a 3.1-billion-parameter vision-language model that runs on-device, reading screens, documents, and charts.
  • It can ground objects to coordinates and call tools directly from text or image input, opening the door to real automation for SMEs.
  • It supports multilingual and offline use, which matters for Malaysian businesses that deal with Malay, Chinese, Tamil, and English data.

What This Means

In plain language, a vision-language model is AI that can “see” an image — like a screenshot or a photo of a receipt — and answer questions about it. Most such models live in the cloud, so you upload an image to a server and wait for a response. LFM2.5-VL-3B is different. It is built to run right on your device, fitting into roughly 3 GB of memory. That means the image you scan never leaves your phone or computer.

Two features make this relevant to you:

  • Screen and UI understanding: The model scores 80.7 on ScreenSpot-v2, meaning it can identify buttons, fields, and menus on desktop, mobile, and web interfaces. It can find where a button is on the screen, not just describe it.
  • Function calling: It can output a command like “click this button” or “type this value into that box.” The tool calls are formatted between special tokens, so software can react to them and perform the action.

Combined, these capabilities let an AI “operate” a computer interface — not just chat with you. That’s a different experience from the AI you’ve probably tried, where you copy and paste the information it gives you.

“The same screen-reading AI that runs on a phone could automate the boring, repetitive tasks that eat your staff’s hours — and it can do it without sending sensitive data anywhere.”

How This Applies to Malaysian SMEs

Let’s get specific. The most immediate use is receipt and invoice processing. According to the benchmarks, the model scores 91.1 on DocVQA and 84.2 on OCRBench v1. That means it handles the messy layout of real receipts and invoices — the kind your vendors send you. Your staff can photograph a supplier invoice, and the model extracts the date, amount, and supplier name as structured text. No more re-typing. In Malaysia, many SMEs still process physical receipts from suppliers, especially in the F&B, retail, and logistics industries. This gives you a practical way to digitize them without relying on a cloud subscription.

Second, think about online form filling and government portals. Many small business owners spend hours on portals that refuse to integrate with anything. LFM2.5-VL-3B can read a form, match labels with the data you have, and fill in the fields. Since it supports 16 languages, it can handle both Bahasa Malaysia and English interfaces. The fact that it runs on-device means you can use it for sensitive documents like payroll records and tax filings without uploading them to a third-party server.

Third, there’s customer-facing operations. If you run a small e-commerce store, you probably receive orders through WhatsApp, Shopee, and your website. Your team copies those orders into your inventory system. A screen-reading model can look at a WhatsApp chat screenshot, extract the product code and quantity, and update your spreadsheet. It can also do multi-image comparison — 61.5 on BLINK and 58.3 on MuirBench — which is useful if you need to compare product photos or check an item’s condition across different images.

Then there are less obvious but reliable use cases: reading road signs and menus offline for logistics drivers, and checking objects in near-real time in vehicles. That last capability matters for a small fleet operator or a courier service. You don’t need to buy a special camera; the AI just handles the image on the device.

What the Numbers Tell You

Here’s how LFM2.5-VL-3B compares on the tasks above, based on data from the release report:

Capability Score What It Means For You
ScreenSpot-v2 (screen understanding) 80.7 Finds buttons and fields on phone, web, and desktop apps
RefCOCO-avg (object grounding) 87.9 Points at objects precisely, so it can tap or click accurately
DocVQA (document reading) 91.1 Reads invoices, delivery notes, and forms well
ChartQA (chart reading) 81.3 Extracts numbers from graphs and dashboards
TextVQA (text in images) 84.3 Reads signage, labels, and product packaging

Practical Takeaways

You don’t need to be an engineer to start. Here’s a simple path:

  • Pick one task. Choose a single manual data-entry job, such as entering supplier invoices into Excel. Take screenshots or photos of twenty examples.
  • Run the model locally. The checkpoint ships in GGUF, ONNX, and MLX formats. Use tools like LM Studio or llama.cpp on a desktop with at least 8GB RAM to load it.
  • Test extraction and check the output. Verify that it gets the date, amount, and supplier name right. If it struggles with a particular receipt layout, try higher-resolution images.
  • Combine it with a macro or automation tool. Once the model identifies a screen element, let a simple script perform the click or insert the text.
  • Keep a human in the loop. Use it as a speed-up for your best staff, not a replacement. Review occasional samples.

The Bigger Picture

This is part of a larger move: push AI from the cloud to the device. When the AI runs on your laptop or phone, you keep your data close, get faster responses, and don’t depend on an internet connection. For Malaysian SMEs, that means automation is no longer something you buy from a huge vendor for a long-term contract. It’s software you can download, run on hardware you already own, and point at a single boring task.

Long-term, expect to see models like this embedded in accounting software, inventory tools, and even WhatsApp Business. Your ability to adapt won’t depend on your IT budget, but on how clearly you can name that one repetitive task you want to eliminate first. Start with the receipts, then show your team what’s possible.

Ready to Streamline Your Operations?

Your business should run itself. AutoRunBiz deploys AI agents to automate your daily operations — WhatsApp orders, invoicing, customer follow-ups, and accounting. Book a free 15-min ops audit to see where automation fits your business →