Artificial Intelligence / AI 0504 · Atom · ~20 seconds
The Counting Wall
Video not yet published
to the StudyWalks catalog
to the StudyWalks catalog
State
Counting-based language models die on sparsity — with a realistic vocabulary, almost every legal word sequence has never been written, so its count is zero.
Show
A 50,000-word vocabulary makes the bigram table two and a half billion cells; three-word contexts push past a hundred trillion.
Watch for
No pile of text on Earth fills those tables — "unseen" and "impossible" collapse into the same zero.