Four KDD 2026 Papers, One Admission: Serving Generative Recommendation Is the Expensive Part

TL;DR

All four papers land in KDD 2026's Applied Data Science track, not Research — these are write-ups of systems that already shipped in production, not theoretical contributions.
RecCompl and Kunlun (Meta), TransX (LinkedIn), and MTGenRec (Meituan) each target a different bottleneck: compilation (running the model as one optimized graph instead of executing it step by step), GPU utilization (how much of the GPU's compute capacity is actually doing useful work versus sitting idle), per-request latency (how long it takes to serve a single recommendation), and dynamic sparse embeddings (the lookup tables for users/items, which have to keep growing and shrinking as the catalog changes instead of staying a fixed size). Four different problems, but the fixes rhyme more than you'd expect.
The number I keep coming back to is TransX's: roughly 80% less online compute and a 6.0% CTR lift, together rather than as a tradeoff. The other three back it up with real production numbers of their own — Kunlun's Model FLOPs Utilization (MFU) jump from 17% to 37% on B200s, MTGenRec's 1.6–2.4x training throughput past 100 GPUs, and RecCompl's up to 60% higher training throughput.
My honest read: the real achievement isn't a new idea, it's recognizing that LLM serving had already solved these problems and then doing the harder work of making the fix hold up at recsys scale — ragged batches, catalogs in the hundreds of millions, embedding tables that never sit still. Compile the graph, cache what doesn't change per request, let the embedding table grow and shrink the way production actually does. Simple to state, genuinely hard to pull off at this scale, and all four teams did.

I read papers like these four the same way I used to sit down with the engineering team once a model was actually live — not looking for a surprise, exactly. A perf test will usually tell you where a bottleneck is going to show up before an A/B test starts. What I actually wanted to know was whether the fix held once real production traffic hit it in ways the perf test never quite replicated, and what it took to get there. This year the four that stood out to me weren't about better recommendations at all. They were about admitting, in varying degrees of directness, that once you let a model generate recommendations token by token instead of scoring a candidate list, you've bought yourself an LLM-serving problem — the exact class of problem two-tower retrieval let us ignore for a decade — and you don't get to opt out of paying for it.

Two-tower never had this problem, and I think that's worth remembering

A two-tower model is simple: turn the user into a vector, turn the item into a vector, take a dot product between them. That's the whole serving path. No growing context to track, no cache to manage, nothing that gets slower the longer someone has been using the app. It's not an exciting architecture, but that's exactly why it stayed in production for over a decade — it never asked much of the serving stack.

Generative recommendation gives that up on purpose. Modeling a user's history as a sequence is genuinely better for long histories, cold start, and folding retrieval, ranking, and search into one model — I don't disagree with the modeling case at all. What I'd push back on is the idea that you get the better model for free. You don't. The compute graph gets complex enough that you can't run it eagerly anymore. The cache tracking a session keeps growing instead of staying flat. And the embedding tables, which used to be something you sized once and left alone, now have to grow and shrink live because the catalog itself won't hold still. None of that shows up on a slide about model quality. It shows up in whoever's on call once you're serving ads or feed traffic at a QPS a chat product never has to deal with, and that's really what these four papers are about.

The four papers

RecCompl (Meta) — Efficient Model Compilation for Industrial Scale Recommendation Models with PyTorch 2. Good timing, actually — this one went up on ACM's Digital Library on August 8th, open access, right as KDD started, so by the time I sat down to write this I had the actual paper instead of just the title. The story turned out narrower than I'd guessed. PyTorch 2's compiler, when it first shipped, basically choked on deep learning recommendation models — recompiling constantly, breaking the graph in spots it shouldn't. RecCompl is Meta going through and fixing that, and they also bolted on a configuration layer so you can change compilation settings without editing model code — which matters more than it sounds like it should, since having to touch model code just to flip a compiler flag is the kind of friction that quietly discourages people from experimenting with compilation settings at all. Training throughput went up by as much as 60%, which is the number Meta leads with. Compilation got faster too, and apparently it's already running across a big chunk of their internal models. I'd put this next to Kunlun below for the same reason — there's no clever new algorithm here, just someone spending real engineering time making sure PyTorch 2 actually works on models this size and this misshapen, not just the tidy benchmark models it was tested on originally.

Kunlun (Meta) — Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design. Recsys never really got its own Chinchilla moment the way LLM pretraining did, and this paper is framed as the fix for that. Read past the framing, though, and what they actually spent their time on was GPU utilization — Meta's models were running at 17% Model FLOPs Utilization, which is just a bad number, full stop. They reworked a chunk of the architecture to get that to 37% on B200s — Generalized Dot-Product Attention and a "Computation Skip" mechanism are the two pieces that stood out to me, though the paper lists a longer set of changes — and roughly doubled scaling efficiency along the way. It's live in Meta's Ads models now. My honest take: you don't get predictable scaling out of a GPU that's idle two-thirds of the time, so fixing utilization is basically the scaling-laws result here, dressed up in the more publishable name.

TransX (LinkedIn) — Scaling Transformer-based Recommendation via Behavioral and Serving Stream Crossings. This is the one I'd point a friend to first. A user's long-term behavior history barely changes between requests. What's actually new each time is the current serving event — what they're looking at right now. Most "generative rec as one long token stream" designs mash the two together and recompute the whole thing on every single request, which is a strange amount of wasted work once you say it out loud. TransX splits them instead — an encoder-decoder setup with cross-attention between a nearline-encoded behavior stream and the live serving event — and caches the behavior side per user so it isn't redone every time. It's the same idea as a KV cache in LLM inference, just applied to a user's behavior history instead of a token sequence. Results-wise this is the strongest of the four: about 80% less online compute, and in an actual A/B test on LinkedIn traffic, +6.0% CTR and +4.4% conversion on top of that. Most systems papers I read give you one or the other, cheaper or better. Getting both out of the same change is unusual.

MTGenRec (Meituan) — An Efficient Distributed Training System for Generative Recommendation Models in Meituan. This one's a training-side problem, and it's specific to recsys in a way the other three aren't: the sparse embedding tables underneath a generative rec model have to grow and shrink continuously as items and users come and go, and a static table allocation doesn't survive that. MTGenRec's fix is dynamic hash tables that can actually grow and shrink with the catalog. It also rebalances sequences across GPUs so a straggler doesn't leave the rest of the cluster sitting idle, and dedupes feature IDs so the same lookup doesn't get done twice. Training throughput went up 1.6 to 2.4x past 100 GPUs, and it's already running in production at Meituan on hundreds of millions of daily requests — a reported 1.22% lift in order volume, 1.31% in click-through rate. This is the paper that actually gates everything else here: the other three don't matter if the training system underneath them can't keep up with a catalog that won't sit still.

What they have in common

Paper	Company	What's slow	What they built	LLM-serving equivalent
RecCompl	Meta	Eager-mode execution on huge, irregularly shaped models	Full-model PyTorch 2 graph compilation	`torch.compile` / graph-capture inference stacks
Kunlun	Meta	Low GPU utilization (17% MFU) blocking predictable scaling	Attention/pooling redesign + compute-skip, unified architecture	The MFU-chasing that defines LLM pretraining efficiency work
TransX	LinkedIn	Recomputing a user's whole history on every request	Stream-split architecture with per-request KV caching	KV-caching in autoregressive LLM inference
MTGenRec	Meituan	Static embedding tables can't handle real-time insert/delete at scale	Dynamic hash tables + load-balanced sharding	Dynamic/paged memory allocation in LLM serving systems (e.g., PagedAttention-style KV allocation)

Look at that last column — every fix in it is a renamed version of something LLM serving already had a name for. The "tokens" here are user actions and the "vocabulary" is a catalog with hundreds of millions of items, but none of these four teams are solving a new class of problem. They're solving one LLM infra teams solved first, reshaped for a different kind of data.

This isn't borrowing anymore, it's convergence

Four separate bottlenecks, three unrelated companies, and they all landed in the same neighborhood of solutions, seemingly without coordinating with each other. That's the part I actually find interesting: recommendation serving infra and LLM serving infra are turning into the same job, just pointed at differently shaped inputs. Whoever wrote these papers works the ads/feed serving stack day to day — a different crowd from the modeling researchers behind semantic IDs and HSTU — and I'd bet real money a few of them have a vLLM GitHub issue open in another tab right now. If I had to guess what shows up at KDD next, I'd put money on quantization for recommendation models and batching strategies built around how bursty ads/feed traffic actually looks. Both are already standard on the LLM side. Recsys has scattered work on quantization already, but neither one's become the obvious default the way it has for LLM serving. That's roughly where a piece I want to write on distillation and inference cost picks up — this post is the setup for it.

Where I'd push back

Every one of these four papers comes from a company with a dedicated ML systems team and a fleet of B200-class GPUs. Meta twice, then LinkedIn, then Meituan. That's not a knock on the work, but it's worth saying plainly: "the infrastructure is catching up" is, right now, mostly true for the handful of companies who could already afford to build it themselves. None of this shipped as an open framework either — there's no vLLM-for-recsys, just four write-ups of internal systems. I genuinely don't know if that changes — if someone eventually open-sources the recsys equivalent of vLLM, or if every hyperscaler just keeps quietly rebuilding the same thing behind their own firewall.

Where this leaves things

Most of the attention in generative recommendation has gone to the modeling side — semantic IDs, HSTU, Netflix's GenRec, Kuaishou's OneRec. Nobody's giving a conference talk about compiler flags and hash tables. But these four papers are what it actually costs to run any of that at ads or feed QPS, and the answer keeps coming back to rebuilding the LLM serving stack, just aimed at recommendations instead of chat. That's the direction I think this goes: recsys infra and LLM infra stop being two fields that occasionally borrow from each other, and just become the same job.

Sources: