Nikhil Gopal

AI systems that make it to production.

I'm Nikhil Gopal, an AI consultant in New York City. I spent four years at Microsoft and IBM building AI solutions for customers such as Goldman Sachs, Verizon and the NBA. AI is the focus of my consulting, and I take on general software and technology work too.

Start a conversation Read the research

Nikhil Gopal standing in Piazza di Spagna in Rome at night, smiling.
Based in
New York City
Education
Columbia University, Statistics and Computer Science
Industry
Four years at Microsoft and IBM
Research
LLM inference, with more than six published articles

What I do

Cloud architecture design

The shape of the system before anyone builds it: where models run, how data reaches them, what it costs, and how it holds up under the security and compliance rules a large company lives with.

AI agents

Agents that do real work inside a business. That means picking the right tools and models, deciding what the agent is allowed to do on its own, and measuring whether it is getting the job right.

Technical delivery

Getting from a promising demo to something people rely on. I scope the work, lead the build alongside your engineers, and stay through launch so the handover is to a team that understands what they own.

Software and technology consulting

Not every problem needs a model. I also take on general software and technology work: system design, build-or-buy decisions, and a second opinion on a technical plan before you commit to it.

Where I've done it

At Microsoft and IBM I worked directly with customers, designing and building AI solutions inside their own environments. The work covered finance, telecom and sport, and each came with its own scale and its own constraints.

Customers I built for include

  • Goldman Sachs
  • Verizon
  • The NBA
Nikhil Gopal and another visitor standing beside the Larry O'Brien Championship Trophy in front of an NBA logo wall.
On site at the NBA, next to the Larry O'Brien Trophy.

Research

Columbia University, with Kostis Kaffes. arXiv, January 2026.

Harvest: Opportunistic Peer-to-Peer GPU Caching for LLM Inference

LLM inference is now limited more by GPU memory than by compute. Harvest treats the unused memory on neighbouring GPUs as a fast cache for model weights and KV cache entries, and more than doubles throughput on the workloads we tested.

I work on the same problems in research that my clients run into in production: what inference costs, why it is slow, and what to do about it.

Read the paper All publications on Google Scholar

Away from work

I studied Statistics and Computer Science at Columbia and I'm still in New York. When I'm not working I'm usually watching football, closely enough that I spent part of the 2026 World Cup in the stands in a Brazil shirt. The rest of the time I travel when I can.

Nikhil Gopal in a Columbia blue cap and gown on the Columbia University campus.
Graduation day at Columbia.
Nikhil Gopal in a yellow Brazil shirt in the upper tier of a full stadium at the 2026 World Cup.
At the 2026 World Cup.

Tell me what you're building

A few lines about the problem is enough to start. I read and reply to every message myself.

Prefer email? Write to nikhil@nikhilgo.com.