Here’s what’s really going on inside an LLM’s neural network

Anthropic's conceptual mapping helps explain why LLMs behave the way they do.

See full article...

 

[KT421]

Concepts are encoded as tokens with coordinates in N-dimensional phase space. Concepts that share commonalities share locality. I think I actually grok this.

You got it. The most meaningful difference between a hot dog detector neural net that I can make on my personal computer and ChatGPT4 is the absolute bonkers value of n in their n-dimensional space, and the absurd cost of the hardware needed to calculate and store that n-dimensional space.