X-algorithm

How X ranks candidates using a tiny Grok-based transformer with candidate isolation masking — scoring all posts in a single forward pass without them interfering with each other.

How X finds ~200 candidate posts from billions in milliseconds — Thunder keeps in-network posts in memory, while Phoenix Retrieval uses a two-tower transformer to discover out-of-network content via embedding similarity search.

X open-sourced their For You feed recommendation algorithm. This is Part 1 of a deep dive into how it works — from the four-codebase architecture to the full pipeline overview, and the one surprising design choice that changes everything.

How X built a reusable, trait-based pipeline framework in Rust that separates orchestration from business logic — with clever error handling that prioritizes availability over correctness.