Most people assume that powerful AI requires a cloud subscription, a fast internet connection, and a company with deep pockets running the servers. Saluki 27B challenges that assumption. Conway Research, the company behind the on-device assistant called Underdog, released a model that squeezes a 54GB AI system down to 7.89GB, small enough to run on a laptop with a decent graphics card, and it actually beats the original at one of the most useful tasks AI can do.
What Saluki 27B Actually Is
Saluki 27B is a compressed version of a large AI language model called Qwen3.8-27B. Think of the original model as a detailed encyclopedia stored on a shelf that takes up an entire room. Saluki 27B is a condensed edition that fits in a backpack, and for most practical tasks it reads just as well.
The compression technique is called 2-bit quantization. Quantization is a way of storing numbers with less precision, similar to rounding a price from $14.87 to $15. You lose a little detail, but you save a lot of space. According to the source article, the original model needs 54GB of storage and memory. Saluki 27B needs 7.89GB. That is a reduction of more than 85 percent.
Conway Research released Saluki 27B under an Apache 2.0 license, which means anyone can download and use it for free, including for business purposes.
Why Tool Calling Matters More Than Raw Power
The most interesting finding is not the size reduction. It is that Saluki 27B actually outperforms the full-size original on tool calling.
Tool calling is the ability of an AI model to use outside resources, like searching a database, running a calculation, or triggering an action in another app. It is what turns a simple chat assistant into an AI agent, a system that can take steps and complete tasks rather than just answer questions. According to the source article, Saluki 27B scored 88 on Underdog’s tool-calling benchmark while the full-size model scored 84. On a parallel tool-calling test, where the model handles multiple tools at once, Saluki scored 42 compared to 35 for the original.
For small businesses exploring AI agents that can automate workflows, this matters. You can read more about how AI agents work in practice in AI Agents That Speak First: What Meta, OpenAI and Uber Got Right.
The tradeoff is real, though. On a difficult math competition benchmark called AIME 2025, Saluki scored 79.2 compared to 96.7 for the full model. If your use case involves complex multi-step reasoning or advanced mathematics, the compressed version falls noticeably short. For most business and personal tasks, that gap is unlikely to matter.
What This Looks Like in Real Life
A freelance consultant who wants AI help without a monthly subscription
- Problem: Cloud AI tools charge monthly fees and send your data to outside servers.
- Usual approach: Pay for a subscription or avoid AI entirely.
- With AI: Run Saluki 27B locally on a laptop with enough memory, keeping data on your own machine.
- What to provide: A compatible laptop with a capable graphics card and the free llama.cpp software.
- Benefit: No subscription cost, no data leaving your device, and strong performance on practical tasks like drafting emails or summarizing documents.
- Limit: Setup requires some technical comfort. This is not a one-click install for most people today.
A small retail business owner who wants to automate repetitive tasks
- Problem: Connecting AI to your inventory system, calendar, or order tracker usually requires expensive software or a developer.
- Usual approach: Hire someone to build a custom integration or skip automation entirely.
- With AI: A local agent model with strong tool-calling ability can be configured to check stock levels, draft reorder emails, or pull data from a spreadsheet on command.
- What to provide: Clear instructions about which tools the agent should use and what actions it should take.
- Benefit: According to the source article, Saluki 27B handles parallel tool calls better than the full-size model, meaning it can manage more than one task at a time.
- Limit: Building an agent still requires technical setup. The model itself is free, but connecting it to your business tools takes additional work.
An employee who wants to experiment with AI without using company systems
- Problem: Your employer restricts which AI tools you can use at work, and you want to practice on your own time.
- Usual approach: Use a free cloud chatbot that may store your conversations.
- With AI: Run a local model on a personal laptop to practice writing prompts, summarizing notes, or drafting documents without any data leaving your machine.
- What to provide: A personal computer with enough memory and the free llama.cpp software.
- Benefit: Private, free, and a good way to build AI skills before applying them at work.
- Limit: The setup process is still technical. Easier consumer options exist if privacy is not the main concern.
The Bigger Trend: AI Getting Cheaper and Smaller
Saluki 27B is one example of a broader shift. AI models are getting smaller, faster, and cheaper to run without losing much of their usefulness. This trend directly affects the cost of using AI, whether you pay for cloud access or run models yourself. Liquid Inference: AI Cost Auctions That Could Lower Your Bills covers another angle of this same shift.
For small businesses, the practical takeaway is that the barrier to running capable AI is dropping. A model that needed a server farm two years ago now fits on a gaming laptop. That does not mean everyone should rush to set up local AI today, but it does mean the options are expanding and costs are falling.
According to the source article, Saluki 27B achieves 96 percent average performance retention across nine benchmarks compared to the full-size model. That is a strong result for a file that is 85 percent smaller.
Limits and Honest Cautions
Saluki 27B is a tool for technically inclined users right now. Running it requires downloading software, configuring a system, and troubleshooting errors. If you are not comfortable with that kind of setup, a cloud-based AI assistant is still the easier starting point.
The compression does cause real quality loss in some areas. Math, logic puzzles, and complex multi-step reasoning are weaker. Do not rely on any AI model, compressed or not, for high-stakes decisions without human review. Always check AI-generated content for accuracy before using it professionally.
Privacy is one of the genuine advantages of local models. Because the model runs on your own machine, your prompts and documents do not travel to an outside server. That said, setting up any new software carries its own risks. Only download models from sources you trust, and understand what you are installing before you run it.
Conway Research has not published the full details of their compression method, so independent researchers cannot fully verify every claim. Benchmark scores from different companies also use different testing methods, so cross-company comparisons are not always apples to apples, as the source article notes.
What This Means If You Are Just Getting Started
You do not need to run Saluki 27B yourself to benefit from understanding what it represents. The story here is that AI capable of handling real business tasks is becoming small enough, cheap enough, and accessible enough that it no longer belongs only to large companies with large budgets.
If you are still figuring out where AI fits in your work, the most useful thing you can do is start with a clear problem rather than a tool. The Problem-First Implementation Framework: How To Map Business Pain Points Before You Buy AI walks through exactly that process.
The gap between what AI can do and what most people are using it for is still wide. Models like Saluki 27B are evidence that the technical barriers are shrinking. The remaining barrier is mostly familiarity, and that one you can close on your own schedule.
Your first step
Open a free cloud AI assistant like ChatGPT or Claude and type a prompt that names a specific task you repeat at work, the format you want the output in, and any constraints you have. For example: Summarize this meeting agenda as five bullet points for someone who was not in the meeting, keeping each bullet under 20 words. Notice how giving the AI a clear task, format, and audience changes the quality of the result. This is the same kind of structured instruction that makes agent models like Saluki 27B useful.