{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uncovering-hidden-geometry-in-transformers","title":"Uncovering hidden geometry in Transformers via disentangling position and context","arxiv_id":"2310.04861","date":"2023-10-07","proceeding":null,"authors":["Jiajun Song","Yiqiao Zhong"],"abstract":"Transformers are widely used to extract semantic meanings from input tokens, yet they usually operate as black-box models. In this paper, we present a simple yet informative decomposition of hidden states (or embeddings) of trained transformers into interpretable components. For any layer, embedding vectors of input sequence samples are represented by a tensor $\\boldsymbol{h} \\in \\mathbb{R}^{C \\times T \\times d}$. Given embedding vector $\\boldsymbol{h}_{c,t} \\in \\mathbb{R}^d$ at sequence position $t \\le T$ in a sequence (or context) $c \\le C$, extracting the mean effects yields the decomposition \\[ \\boldsymbol{h}_{c,t} = \\boldsymbol{\\mu} + \\mathbf{pos}_t + \\mathbf{ctx}_c + \\mathbf{resid}_{c,t} \\] where $\\boldsymbol{\\mu}$ is the global mean vector, $\\mathbf{pos}_t$ and $\\mathbf{ctx}_c$ are the mean vectors across contexts and across positions respectively, and $\\mathbf{resid}_{c,t}$ is the residual vector. For popular transformer architectures and diverse text datasets, empirically we find pervasive mathematical structure: (1) $(\\mathbf{pos}_t)_{t}$ forms a low-dimensional, continuous, and often spiral shape across layers, (2) $(\\mathbf{ctx}_c)_c$ shows clear cluster structure that falls into context topics, and (3) $(\\mathbf{pos}_t)_{t}$ and $(\\mathbf{ctx}_c)_c$ are mutually nearly orthogonal. We argue that smoothness is pervasive and beneficial to transformers trained on languages, and our decomposition leads to improved model interpretability.","url_abs":"https://arxiv.org/abs/2310.04861v2","url_pdf":"https://arxiv.org/pdf/2310.04861v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uncovering-hidden-geometry-in-transformers","repo_url":"https://github.com/jiajunsong629/uncover-hidden-geometry","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"dictionary-learning","task_name":"Dictionary Learning"},{"task_slug":"pos","task_name":"POS"},{"task_slug":null,"task_name":"Position"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.04861","atlas_url":"https://app.syntology.ai/?focus=2310.04861","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}