"Bigger Is Always Better"

Bigger models without more data feed the memorization appetite; bigger data of the same skew polishes the skew; bigger both buys capability on the training distribution and nothing, by itself, beyond it.

Scale is a strategy with costs and preconditions — computation, energy, opacity — not a law of nature.