Cold start without the hand waving: what meta-learning gives a recommender

Every recommender has the same weak moment: the first visit. A new user has rated nothing, clicked nothing and bought nothing, and the system still has to decide what to show them. Collaborative filtering, which learns from the overlap between users, has nothing to overlap with. This is the cold-start problem, and it was the subject of my master’s thesis.

My thesis looked at a concrete version of it: recommending places to travellers who have just arrived and have almost no check-ins. This is a practical summary rather than the full thesis: the framing that made the problem tractable, the evaluation choices that mattered most, and what I would tell an engineer considering the approach.

Why the usual answers stall

The standard fallbacks are popularity and content features. Showing the most popular items is a strong baseline that is hard to beat for the very first screen, but it treats every newcomer the same. Content based models use item attributes and whatever we know about the user up front, which helps, but they learn one global mapping and adapt slowly to the few signals a new user does give.

Graph neural networks, which learn from the user and item interaction graph, are among the strongest models for warm users. For a cold user, though, the node has almost no edges, so most of what makes the model strong is missing.

Each user is a small learning task

Meta-learning changes the question. Instead of training one model to fit all users, it treats each user as a separate task: given a handful of that user’s interactions (the support set), predict the rest (the query set). The model is trained across many such tasks to become good at one thing: adapting quickly from a few examples.

In the optimisation based family, of which MAML is the best known, the result of training is not a final model but a starting point. For a new user, we take that shared starting point and run a few gradient steps on their handful of interactions. Because the starting point was learned for exactly this situation, those few steps go a long way.

for users in task_batches:
  meta_loss = 0
  for user in users:
      support, query = user.split(k_shot)

      # Inner loop: personalise a copy of the weights on the few known interactions.
      fast = dict(model.named_parameters())
      for _ in range(inner_steps):
          loss = loss_fn(model(support.x, params=fast), support.y)
          grads = torch.autograd.grad(loss, fast.values(), create_graph=True)
          fast = {name: w - inner_lr * g for (name, w), g in zip(fast.items(), grads)}

      # Outer loss: how well the personalised weights predict what we held back.
      meta_loss += loss_fn(model(query.x, params=fast), query.y)

  meta_optimizer.zero_grad()
  (meta_loss / len(users)).backward()
  meta_optimizer.step()

I trained two meta-learners on exactly the same features. A MAML model adapts with a few gradient steps per user, as above. A prototypical network does something much simpler: it averages the embeddings of a user's few known places into a prototype and ranks candidates by their distance to it. Keeping the features identical meant any difference in the results came from the algorithm, not from the data it was given.

Context the graph cannot see

Place embeddings from a graph neural network carry what the interaction graph knows. For a newcomer that is very little, so I joined them with two kinds of context the graph cannot provide: a geographic vector, so places near each other relate even if nobody has visited both, and a social vector built from a user's connections, which gives a newcomer a starting point from the people they know before they have checked in anywhere.

Evaluation is where most results go wrong

Cold-start results are easy to inflate by accident, and most of my time went into making sure mine were not. The rules I settled on:

• +Split by user, not by interaction. Test users must be absent from training entirely. A random split of interactions leaks every test user into training and quietly turns the problem warm.
• +Fix the support size and report it. Results with 5 known interactions and with 20 are different claims. They should be reported separately rather than as one average.
• +Use ranking metrics. Users see a short list, so NDCG and Recall at 10 and 20 say more than rating error. They reward putting the right places near the top.
• +Test more than one scale. I evaluated on real check-in and friendship data at country, continent and global scale, because a method that only wins on one slice of the world is not a method yet.
• +Remove one piece at a time. An ablation that drops the geographic vector, the social vector or the meta-learning step in turn shows what each part actually contributes.
• +Compare against strong baselines, tuned fairly. Including a graph neural network, the strongest model for warm users, tuned with the same care as the proposed model.
• +Repeat with several seeds. Meta-learning is sensitive to initialisation, so a single run can mislead. I report the mean and the spread.

Under those rules, the meta-learning model improved NDCG@10 by 21.7% over the graph neural network baseline for cold users. That is the kind of gain worth having on the first screens a new user sees, which are also the screens that decide whether they come back.

What I would tell an engineer

Meta-learning is not a replacement for a production recommender. It is a strong component for the first days of a user’s life, after which a warm model with plenty of signal takes over. The practical shape is a handover: popularity on the very first screen, a meta-learned model once the user has a few interactions, and the main model once there is enough history to trust it.

The lesson I carried back into everyday engineering has little to do with recommenders. It is that the evaluation setup is part of the result. Before trusting any improvement, mine included, I want to know exactly how the data was split, what the baseline was and how many times it was run. That habit is as useful in an A/B test for a checkout page as it was in a thesis.