{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/women-also-snowboard-overcoming-bias-in-1","title":"Women also Snowboard: Overcoming Bias in Captioning Models","arxiv_id":"1803.09797","date":"2018-03-26","proceeding":"ECCV 2018 9","authors":["Kaylee Burns","Lisa Anne Hendricks","Kate Saenko","Trevor Darrell","Anna Rohrbach"],"abstract":"Most machine learning methods are known to capture and exploit biases of the\ntraining data. While some biases are beneficial for learning, others are\nharmful. Specifically, image captioning models tend to exaggerate biases\npresent in training data (e.g., if a word is present in 60% of training\nsentences, it might be predicted in 70% of sentences at test time). This can\nlead to incorrect captions in domains where unbiased captions are desired, or\nrequired, due to over-reliance on the learned prior and image context. In this\nwork we investigate generation of gender-specific caption words (e.g. man,\nwoman) based on the person's appearance or the image context. We introduce a\nnew Equalizer model that ensures equal gender probability when gender evidence\nis occluded in a scene and confident predictions when gender evidence is\npresent. The resulting model is forced to look at a person rather than use\ncontextual cues to make a gender-specific predictions. The losses that comprise\nour model, the Appearance Confusion Loss and the Confident Loss, are general,\nand can be added to any description model in order to mitigate impacts of\nunwanted bias in a description dataset. Our proposed model has lower error than\nprior work when describing images with people and mentioning their gender and\nmore closely matches the ground truth ratio of sentences including women to\nsentences including men. We also show that unlike other approaches, our model\nis indeed more often looking at people when predicting their gender.","url_abs":"http://arxiv.org/abs/1803.09797v4","url_pdf":"http://arxiv.org/pdf/1803.09797v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"women-also-snowboard-overcoming-bias-in-1","repo_url":"https://github.com/davidhuji/capdec","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"women-also-snowboard-overcoming-bias-in-1","repo_url":"https://github.com/dtak/local-independence-public","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-captioning","task_name":"Image Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1803.09797","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}