Cognitive Heads

◀ Prev | 2026-06-01, access: $$$ Pro

Video interpretation CoT attention text The heads in a multi-head attention transformer architecture tend to specialize for different functions. We can find the cognitive heads, responsible for individual steps in chains of thought, by imitating the techniques used in biology to study animal brains.