{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/style-transfer-for-texts-to-err-is-human-but","title":"Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites","arxiv_id":"1908.06809","date":"2019-08-19","proceeding":"IJCNLP 2019 11","authors":["Alexey Tikhonov","Viacheslav Shibaev","Aleksander Nagaev","Aigul Nugmanova","Ivan P. Yamshchikov"],"abstract":"This paper shows that standard assessment methodology for style transfer has several significant problems. First, the standard metrics for style accuracy and semantics preservation vary significantly on different re-runs. Therefore one has to report error margins for the obtained results. Second, starting with certain values of bilingual evaluation understudy (BLEU) between input and output and accuracy of the sentiment transfer the optimization of these two standard metrics diverge from the intuitive goal of the style transfer task. Finally, due to the nature of the task itself, there is a specific dependence between these two metrics that could be easily manipulated. Under these circumstances, we suggest taking BLEU between input and human-written reformulations into consideration for benchmarks. We also propose three new architectures that outperform state of the art in terms of this metric.","url_abs":"https://arxiv.org/abs/1908.06809v2","url_pdf":"https://arxiv.org/pdf/1908.06809v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"style-transfer-for-texts-to-err-is-human-but","repo_url":"https://github.com/VAShibaev/text_style_transfer","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"style-transfer","task_name":"Style Transfer"},{"task_slug":"text-style-transfoer","task_name":"Text Style Transfer"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/text-style-transfer-on-yelp-review-dataset","task":"Text Style Transfer","dataset":"Yelp Review Dataset (Small)","model":"SAE+Discriminator","rank_in_archive_order":1,"of":8,"metrics":{"G-Score (BLEU, Accuracy)":"74.56"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1908.06809","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1908.06809"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VAShibaev/text_style_transfer","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"named_in_paper":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d8ab4cdbe05b528e","entry":"clean_text","repo":"VAShibaev/text_style_transfer","repo_kind":"named_in_paper","path":"shiftedae/prepare_manual.py","file_url":"https://github.com/VAShibaev/text_style_transfer/blob/HEAD/shiftedae/prepare_manual.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d8ab4cdbe05b528e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}