Artificial Intelligence / AI 0513 · Atom · ~20 seconds
Pretraining
Video not yet published
to the StudyWalks catalog
to the StudyWalks catalog
State
A large language model is a transformer pretrained on next-word prediction over vast text collections — billions of knobs descending against prediction error.
Show
More training text than a person reads in ten thousand lifetimes; grammar, style, idiom, and fact-shaped regularity fall out of one objective.
Watch for
"Fact-shaped" is the honest term — descent rewards producing the text that typically follows, which overlaps truth imperfectly.