{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-bilingual-openworld-video-text-dataset-and","title":"A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer","arxiv_id":"2112.04888","date":"2021-12-09","proceeding":null,"authors":["Weijia Wu","Yuanqiang Cai","Debing Zhang","Sibo Wang","Zhuang Li","Jiahong Li","Yejun Tang","Hong Zhou"],"abstract":"Most existing video text spotting benchmarks focus on evaluating a single language and scenario with limited data. In this work, we introduce a large-scale, Bilingual, Open World Video text benchmark dataset(BOVText). There are four features for BOVText. Firstly, we provide 2,000+ videos with more than 1,750,000+ frames, 25 times larger than the existing largest dataset with incidental text in videos. Secondly, our dataset covers 30+ open categories with a wide selection of various scenarios, e.g., Life Vlog, Driving, Movie, etc. Thirdly, abundant text types annotation (i.e., title, caption or scene text) are provided for the different representational meanings in video. Fourthly, the BOVText provides bilingual text annotation to promote multiple cultures live and communication. Besides, we propose an end-to-end video text spotting framework with Transformer, termed TransVTSpotter, which solves the multi-orient text spotting in video with a simple, but efficient attention-based query-key mechanism. It applies object features from the previous frame as a tracking query for the current frame and introduces a rotation angle prediction to fit the multiorient text instance. On ICDAR2015(video), TransVTSpotter achieves the state-of-the-art performance with 44.1% MOTA, 9 fps. The dataset and code of TransVTSpotter can be found at github:com=weijiawu=BOVText and github:com=weijiawu=TransVTSpotter, respectively.","url_abs":"https://arxiv.org/abs/2112.04888v1","url_pdf":"https://arxiv.org/pdf/2112.04888v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-bilingual-openworld-video-text-dataset-and","repo_url":"https://github.com/weijiawu/BOVText-Benchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"a-bilingual-openworld-video-text-dataset-and","repo_url":"https://github.com/weijiawu/transvtspotter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"a-bilingual-openworld-video-text-dataset-and","repo_url":"https://github.com/weijiawu/MMVText-Benchmark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"CC-BY-4.0"}}],"tasks":[{"task_slug":"text-spotting","task_name":"Text Spotting"},{"task_slug":"text-annotation","task_name":"text annotation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[{"slug":"bovtext","name":"BOVText","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2112.04888","atlas_url":"https://app.syntology.ai/?focus=2112.04888","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.04888"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weijiawu/BOVText-Benchmark","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weijiawu/transvtspotter","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weijiawu/MMVText-Benchmark","reach":{"status":"ok","spdx":"CC-BY-4.0"}}],"summary":{"ran_honours":9,"ran_violates":1,"ran":4,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":19,"ran":16,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":19,"samples":[{"code_sha256_prefix":"75c50b21f10622fe","entry":"add_expensive_edges","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/lap.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/lap.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"75c50b21f10622fe"}},{"code_sha256_prefix":"ac8fe530cdad4d8c","entry":"dice_loss","repo":"weijiawu/transvtspotter","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/weijiawu/transvtspotter/blob/HEAD/models/segmentation.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac8fe530cdad4d8c"}},{"code_sha256_prefix":"934f6f437dd35300","entry":"get_rotate_mat","repo":"weijiawu/transvtspotter","repo_kind":"official","path":"models/save_track.py","file_url":"https://github.com/weijiawu/transvtspotter/blob/HEAD/models/save_track.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"934f6f437dd35300"}},{"code_sha256_prefix":"df7c890d21316ed3","entry":"load_motchallenge","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/io.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/io.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"df7c890d21316ed3"}},{"code_sha256_prefix":"73aafda163f7eb35","entry":"lsa_solve_scipy","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/lap.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/lap.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"73aafda163f7eb35"}},{"code_sha256_prefix":"8aac7a69af7327b2","entry":"norm2squared_matrix","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/distances.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/distances.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8aac7a69af7327b2"}},{"code_sha256_prefix":"c226fad167ab7651","entry":"num_frames","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/metrics.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c226fad167ab7651"}},{"code_sha256_prefix":"fd57be74b7693046","entry":"obj_frequencies","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/metrics.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"fd57be74b7693046"}},{"code_sha256_prefix":"b443457d657ce785","entry":"pred_frequencies","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/metrics.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b443457d657ce785"}},{"code_sha256_prefix":"8c8460f479f8b77e","entry":"quiet_divide","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/math_util.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/math_util.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8c8460f479f8b77e"}},{"code_sha256_prefix":"9a73f2c41ea25d0b","entry":"read_mot_results","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/tracking_utils/io.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/tracking_utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9a73f2c41ea25d0b"}},{"code_sha256_prefix":"3b4276dfe539afa3","entry":"read_results","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/tracking_utils/io.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/tracking_utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3b4276dfe539afa3"}},{"code_sha256_prefix":"65529c5a828c8fac","entry":"read_text_results","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/tracking_utils/io.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/tracking_utils/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"65529c5a828c8fac"}},{"code_sha256_prefix":"d8bed56ba7a09f52","entry":"rect_min_max","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/distances.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/distances.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d8bed56ba7a09f52"}},{"code_sha256_prefix":"5c0711aada67957e","entry":"sigmoid_focal_loss","repo":"weijiawu/transvtspotter","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/weijiawu/transvtspotter/blob/HEAD/models/segmentation.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c0711aada67957e"}},{"code_sha256_prefix":"eedc05e12aec5bd3","entry":"sort_key","repo":"weijiawu/transvtspotter","repo_kind":"official","path":"models/save_track.py","file_url":"https://github.com/weijiawu/transvtspotter/blob/HEAD/models/save_track.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eedc05e12aec5bd3"}},{"code_sha256_prefix":"afa1b0e940655c4c","entry":"linear_sum_assignment","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/lap.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/lap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"afa1b0e940655c4c"}},{"code_sha256_prefix":"5bd3eff2e6eab7ba","entry":"load_detrac_mat","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/io.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/io.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"5bd3eff2e6eab7ba"}},{"code_sha256_prefix":"56d557c272771a4b","entry":"load_vatictxt","repo":"weijiawu/BOVText-Benchmark","repo_kind":"official","path":"Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/io.py","file_url":"https://github.com/weijiawu/BOVText-Benchmark/blob/HEAD/Evaluation_Protocol/Task1_VideoTextTracking/motmetrics/io.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"56d557c272771a4b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}