- Based in
- New York City
- Education
- Columbia University, Statistics and Computer Science
- Industry
- Four years at Microsoft and IBM
- Research
- LLM inference, with more than six published articles
What I do
Cloud architecture design
The shape of the system before anyone builds it: where models run, how data reaches them, what it costs, and how it holds up under the security and compliance rules a large company lives with.
AI agents
Agents that do real work inside a business. That means picking the right tools and models, deciding what the agent is allowed to do on its own, and measuring whether it is getting the job right.
Technical delivery
Getting from a promising demo to something people rely on. I scope the work, lead the build alongside your engineers, and stay through launch so the handover is to a team that understands what they own.
Software and technology consulting
Not every problem needs a model. I also take on general software and technology work: system design, build-or-buy decisions, and a second opinion on a technical plan before you commit to it.
Where I've done it
At Microsoft and IBM I worked directly with customers, designing and building AI solutions inside their own environments. The work covered finance, telecom and sport, and each came with its own scale and its own constraints.
Customers I built for include
- Goldman Sachs
- Verizon
- The NBA
Research
Columbia University, with Kostis Kaffes. arXiv, January 2026.
Harvest: Opportunistic Peer-to-Peer GPU Caching for LLM Inference
LLM inference is now limited more by GPU memory than by compute. Harvest treats the unused memory on neighbouring GPUs as a fast cache for model weights and KV cache entries, and more than doubles throughput on the workloads we tested.
I work on the same problems in research that my clients run into in production: what inference costs, why it is slow, and what to do about it.
Away from work
I studied Statistics and Computer Science at Columbia and I'm still in New York. When I'm not working I'm usually watching football, closely enough that I spent part of the 2026 World Cup in the stands in a Brazil shirt. The rest of the time I travel when I can.
Tell me what you're building
A few lines about the problem is enough to start. I read and reply to every message myself.
Prefer email? Write to nikhil@nikhilgo.com.