{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-joint-matrix-factorization-analysis-of","title":"A Joint Matrix Factorization Analysis of Multilingual Representations","arxiv_id":"2310.15513","date":"2023-10-24","proceeding":null,"authors":["Zheng Zhao","Yftah Ziser","Bonnie Webber","Shay B. Cohen"],"abstract":"We present an analysis tool based on joint matrix factorization for comparing latent representations of multilingual and monolingual models. An alternative to probing, this tool allows us to analyze multiple sets of representations in a joint manner. Using this tool, we study to what extent and how morphosyntactic features are reflected in the representations learned by multilingual pre-trained models. We conduct a large-scale empirical study of over 33 languages and 17 morphosyntactic categories. Our findings demonstrate variations in the encoding of morphosyntactic information across upper and lower layers, with category-specific differences influenced by language properties. Hierarchical clustering of the factorization outputs yields a tree structure that is related to phylogenetic trees manually crafted by linguists. Moreover, we find the factorization outputs exhibit strong associations with performance observed across different cross-lingual tasks. We release our code to facilitate future research.","url_abs":"https://arxiv.org/abs/2310.15513v1","url_pdf":"https://arxiv.org/pdf/2310.15513v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-joint-matrix-factorization-analysis-of","repo_url":"https://github.com/zsquaredz/joint_multilingual_analysis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.15513","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.15513"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zsquaredz/joint_multilingual_analysis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"880da7cfdc10270d","entry":"merge_attributes","repo":"zsquaredz/joint_multilingual_analysis","repo_kind":"official","path":"morphosyntactic_features/preprocess_treebank.py","file_url":"https://github.com/zsquaredz/joint_multilingual_analysis/blob/HEAD/morphosyntactic_features/preprocess_treebank.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"880da7cfdc10270d"}},{"code_sha256_prefix":"74a002cae3b4adb2","entry":"read_files","repo":"zsquaredz/joint_multilingual_analysis","repo_kind":"official","path":"src/utils/downsample.py","file_url":"https://github.com/zsquaredz/joint_multilingual_analysis/blob/HEAD/src/utils/downsample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74a002cae3b4adb2"}},{"code_sha256_prefix":"c3fc885d622cc62e","entry":"subword_tokenize","repo":"zsquaredz/joint_multilingual_analysis","repo_kind":"official","path":"morphosyntactic_features/preprocess_treebank.py","file_url":"https://github.com/zsquaredz/joint_multilingual_analysis/blob/HEAD/morphosyntactic_features/preprocess_treebank.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3fc885d622cc62e"}},{"code_sha256_prefix":"ff00845230695635","entry":"write_files","repo":"zsquaredz/joint_multilingual_analysis","repo_kind":"official","path":"src/utils/downsample.py","file_url":"https://github.com/zsquaredz/joint_multilingual_analysis/blob/HEAD/src/utils/downsample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff00845230695635"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}