{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/freshman-or-fresher-quantifying-the","title":"Freshman or Fresher? Quantifying the Geographic Variation of Internet Language","arxiv_id":"1510.06786","date":"2015-10-22","proceeding":null,"authors":["Vivek Kulkarni","Bryan Perozzi","Steven Skiena"],"abstract":"We present a new computational technique to detect and analyze statistically\nsignificant geographic variation in language. Our meta-analysis approach\ncaptures statistical properties of word usage across geographical regions and\nuses statistical methods to identify significant changes specific to regions.\nWhile previous approaches have primarily focused on lexical variation between\nregions, our method identifies words that demonstrate semantic and syntactic\nvariation as well.\n  We extend recently developed techniques for neural language models to learn\nword representations which capture differing semantics across geographical\nregions. In order to quantify this variation and ensure robust detection of\ntrue regional differences, we formulate a null model to determine whether\nobserved changes are statistically significant. Our method is the first such\napproach to explicitly account for random variation due to chance while\ndetecting regional variation in word meaning.\n  To validate our model, we study and analyze two different massive online data\nsets: millions of tweets from Twitter spanning not only four different\ncountries but also fifty states, as well as millions of phrases contained in\nthe Google Book Ngrams. Our analysis reveals interesting facets of language\nchange at multiple scales of geographic resolution -- from neighboring states\nto distant continents.\n  Finally, using our model, we propose a measure of semantic distance between\nlanguages. Our analysis of British and American English over a period of 100\nyears reveals that semantic variation between these dialects is shrinking.","url_abs":"http://arxiv.org/abs/1510.06786v2","url_pdf":"http://arxiv.org/pdf/1510.06786v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"freshman-or-fresher-quantifying-the","repo_url":"https://github.com/hjian42/Geo-Twitter2019","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"freshman-or-fresher-quantifying-the","repo_url":"https://github.com/yuxingch/DialectGram","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}