The Bigram Table

Counting how often each word follows each word in a pile of text yields a working language model — the bigram model, built from shares alone.

The corpus, four sentences whole: the cat sat; the cat purred; the dog sat; the dog ran. Count the pairs. After "the": cat twice, dog twice — shares one half and one half. After "cat": sat one half, purred one half. After "dog": sat one half, ran one half. That table is a language model. And it already generates: start at "the," sample from its column, move to the sampled word, repeat. "The cat purred." "The dog sat." Fluent, at toy scale, by counting alone — no grammar written anywhere, no meanings defined, arithmetic on a four-sentence world.