Cactus Is Building the Low-Latency AI Engine That Mobile and Wearables Have Been Missing

Cactus is a Y Combinator Summer 2025 company built on a simple but hard conviction: the next frontier of consumer AI isn’t in the cloud — it’s on the device in your pocket or on your wrist. Founded in 2025 and based in San Francisco, the eight-person team is building a low-latency AI engine optimized for mobile and wearable hardware, releasing the core of that work in the open at github.com/cactus-compute/cactus. The company was co-founded by Roman Shemet and Henry Ndubuaku and is taking aim at one of the most consequential unsolved problems in consumer AI: how to run capable models directly on phones, wearables, and other constrained devices without round-tripping every request to a cloud GPU.

The thesis is timely. Over the last 18 months, on-device AI has gone from a research curiosity to a competitive front line. Apple, Google, Qualcomm, and Meta are all pushing model architectures, accelerators, and runtimes designed to keep more inference on the device, both for latency reasons and to address a steadily rising bar around user privacy, offline reliability, and per-query cost. Yet the practical reality for most app developers is still that on-device inference is hard. Tooling is fragmented across hardware vendors, kernels are uneven, and squeezing competitive latency out of a phone or a smartwatch can take a specialist team months. Cactus is positioning its engine as the layer that absorbs that complexity.

Shemet is a former quant and economist with a background in product and data engineering — and, for color, a pilot, triathlete, and chess enthusiast. Ndubuaku stepped away from an AI/ML role at Nvidia to work on Cactus, a signal worth noting given Nvidia’s central position in the modern AI stack. The combination of low-level systems performance experience and product-minded engineering is exactly the profile that on-device runtime work demands, where every wasted millisecond and every wasted milliwatt shows up directly in the user experience.

By starting with an open-source release, Cactus is making a deliberate community play. Mobile and wearable developers tend to be conservative about adopting new runtime layers, especially ones that sit on the critical path between their app and the user. Open source lowers that bar dramatically: developers can read the code, benchmark it on their own devices, and contribute fixes back. It also creates a natural funnel into the kinds of higher-tier offerings — managed deployments, hardware-specific optimizations, model packaging, on-device fine-tuning tooling — that successful infrastructure companies tend to build on top of an open core.

The strategic context for Cactus is favorable. As more product teams move to embed AI features directly into their applications, the question of where inference physically happens has become a first-class architectural decision. On-device wins on latency, privacy, offline behavior, and unit economics; the cloud wins on raw model size and centralized control. A high-quality engine that makes the on-device side of that trade-off dramatically easier — and works credibly across the messy reality of consumer device fleets — is exactly the kind of pick-and-shovel business that tends to compound quickly once early adopters validate it.