3 days ago

Kompact AI bets the CPU can give India a cheaper ticket to the AI race

In the expensive world of Artificial Intelligence, where giant models guzzle computing power and GPUs are treated almost like the new oil, Ziroh Labs is making a simpler pitch: perhaps you do not need a Ferrari to go to the neighbourhood shop.

Its Kompact AI technology can cut the cost of running AI models by 80 to 90 per cent, depending on the application, by allowing smaller, specialised models to run on conventional Central Processing Units (CPUs), Hrishikesh Dewan, Co-Founder & CEO of Ziroh Labs, said here on Monday. Dewan is not claiming that the humble CPU is about to send the mighty GPU into retirement. Far from it.

“Nothing can replace GPU and GPU also cannot replace CPU. You have to find the right fit,” he said.

His argument is rather more practical. If all you want from an AI model is a joke, why build one that can also diagnose a disease, write a novel and edit your photograph against the Eiffel Tower?

That, in essence, is the philosophy behind Kompact AI.

Today’s large general-purpose AI models are designed to do an astonishing variety of things. Their versatility, however, comes at a price. The bigger the model, the greater the computing power required to run it, and the greater the appetite for GPUs.

Cost of AI

A smaller model trained for a particular task does fewer things. However, it can do those things without demanding an army of expensive processors.

Dewan explained that an AI model is, at its heart, a vast mathematical machine. When a user types, “tell me something about Bangalore”, the words are converted into numbers. Those numbers pass through layers of mathematical operations before an answer emerges.

“When you say a model has one trillion parameters, it means there are one trillion variables inside the function,” he said.

This, he pointed out, is fundamentally different from a conventional search engine, which searches an enormous database and retrieves information matching a query. In an AI model, he said, “English is over at the first step”.

What remains is mathematics. And mathematics, unlike human beings, does not necessarily need a palace in which to live.

Dewan believes this distinction gives India an unusual opportunity. Instead of trying immediately to build another gigantic general-purpose AI model, India could concentrate on thousands of smaller models solving thousands of specific problems.

India, with its huge population and millions of engineering students, has the raw manpower for such an experiment.

“If even 500,000 start developing small models on different topics, you are developing 500,000 different types of small models,” he said.

“Once that is done, you buy yourself—you are an AI leader. Nobody can stop you.”

For Dewan, the attraction of Kompact AI is not simply that it can make AI cheaper. It can make failure cheaper.

In technology, failure is often the tuition fee for success. But when the tuition becomes prohibitively expensive, fewer people enrol.

“If I have to build a model and it costs me USD 100,000, then I’ll think for three months—do I spend that money?” he said. “But if I tell you, you will only spend Rs 100 for that experiment, you will do the experiment even today.”

That, he said, is the real contribution of Kompact AI: allowing individuals, researchers, universities and startups to experiment without being frightened by the price tag.

He illustrated the point with his own laptop.

He bought it a year ago. He has been running Kompact AI on it for the past month. Since the laptop was already paid for, his additional computing cost was effectively zero.

The saving, he cautioned, depends on the application and the infrastructure available. But in suitable cases, it could be 80 to 90 per cent.

The CPU, however, is no magic wand.

A basic two- or four-core processor cannot suddenly become a supercomputer. Modern server processors with dozens or hundreds of cores can handle considerably larger workloads, and several such machines can be connected through high-speed networks to scale the system.

Ziroh Labs is experimenting with this approach, including tests involving the Qwen 2.0 model. Dewan was careful to add that an experiment is still an experiment; success is not guaranteed.

He is equally relaxed about larger chip companies copying the technology.

“They should replicate,” he said. Competition, he argued, gives consumers more choices, while the technology itself keeps moving.

He cited Google’s PageRank paper, published by Sergey Brin and Larry Page in 1998, as an example. Nobody expects Google in 2026 to operate exactly as it did in 1998.

“Somebody copies what we have done six months back. We are also evolving,” Dewan said.

And what about money?

Dewan declined to disclose the investment made in the technology. Nor did he sound particularly interested in making revenue the headline of his entrepreneurial journey.

“The goal is not revenue. The goal is the value that you deliver. Revenue is automatic,” he said.

It is an appealing proposition in an industry increasingly obsessed with ever larger models, ever bigger data centres and ever more powerful chips.

Dewan’s message is less grandiose: AI need not always be a sledgehammer. Sometimes, a screwdriver will do.