{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sigmoid-loss-for-language-image-pre-training","title":"Sigmoid Loss for Language Image Pre-Training","arxiv_id":"2303.15343","date":"2023-03-27","proceeding":"ICCV 2023 1","authors":["Xiaohua Zhai","Basil Mustafa","Alexander Kolesnikov","Lucas Beyer"],"abstract":"We propose a simple pairwise Sigmoid loss for Language-Image Pre-training (SigLIP). Unlike standard contrastive learning with softmax normalization, the sigmoid loss operates solely on image-text pairs and does not require a global view of the pairwise similarities for normalization. The sigmoid loss simultaneously allows further scaling up the batch size, while also performing better at smaller batch sizes. Combined with Locked-image Tuning, with only four TPUv4 chips, we train a SigLiT model that achieves 84.5% ImageNet zero-shot accuracy in two days. The disentanglement of the batch size from the loss further allows us to study the impact of examples vs pairs and negative to positive ratio. Finally, we push the batch size to the extreme, up to one million, and find that the benefits of growing batch size quickly diminish, with a more reasonable batch size of 32k being sufficient. We release our models at https://github.com/google-research/big_vision and hope our research motivates further explorations in improving the quality and efficiency of language-image pre-training.","url_abs":"https://arxiv.org/abs/2303.15343v4","url_pdf":"https://arxiv.org/pdf/2303.15343v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/google-research/big_vision","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/apple/ml-mobileclip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/borisdayma/clip-jax","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/filipbasara0/relic","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/filipbasara0/simple-clip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/huggingface/transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/merveenoyan/siglip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/mlfoundations/open_clip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/morrisfl/unifex","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/ramanakshay/clip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"sigmoid-loss-for-language-image-pre-training","repo_url":"https://github.com/shkarupa-alex/tfclip","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"image-to-text-retrieval","task_name":"Image-to-Text Retrieval"},{"task_slug":"zero-shot-text-to-image-retrieval","task_name":"Zero-shot Text-to-Image Retrieval"}],"methods":[{"method_slug":"base","method_name":"BASE"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-to-text-retrieval-on-coco-1","task":"Image-to-Text Retrieval","dataset":"COCO","model":"SigLIP (ViT-L, zero-shot)","rank_in_archive_order":1,"of":1,"metrics":{"Recall@1":"70.6"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.15343","atlas_url":"https://app.syntology.ai/?focus=2303.15343","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.15343"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mlfoundations/open_clip","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/google-research/big_vision","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ramanakshay/clip","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shkarupa-alex/tfclip","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/filipbasara0/relic","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/merveenoyan/siglip","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/filipbasara0/simple-clip","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/morrisfl/unifex","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/apple/ml-mobileclip","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/borisdayma/clip-jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":11,"ran_draft_wrong":3,"unverified":15},"by_repo_kind":{"listed":{"samples":25,"ran":12,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":25,"samples":[{"code_sha256_prefix":"af515d1162a90440","entry":"GlobalPool","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"af515d1162a90440"}},{"code_sha256_prefix":"21e0abb626a7ae62","entry":"GlobalPool2D","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"21e0abb626a7ae62"}},{"code_sha256_prefix":"c4c619fef7d59d34","entry":"LayerNormFP32","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"c4c619fef7d59d34"}},{"code_sha256_prefix":"0748e821730b3fc1","entry":"LearnablePositionalEmbedding","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0748e821730b3fc1"}},{"code_sha256_prefix":"88f8bcf4b76c2844","entry":"MultiHeadAttention","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"88f8bcf4b76c2844"}},{"code_sha256_prefix":"997398ad737e7cd2","entry":"NeighbourExchange","repo":"mlfoundations/open_clip","repo_kind":"listed","path":"src/open_clip/loss.py","file_url":"https://github.com/mlfoundations/open_clip/blob/HEAD/src/open_clip/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"997398ad737e7cd2"}},{"code_sha256_prefix":"7caa2b64e6a6f021","entry":"NeighbourExchangeBidir","repo":"mlfoundations/open_clip","repo_kind":"listed","path":"src/open_clip/loss.py","file_url":"https://github.com/mlfoundations/open_clip/blob/HEAD/src/open_clip/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7caa2b64e6a6f021"}},{"code_sha256_prefix":"1e38402044297bef","entry":"PositionalEmbedding","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1e38402044297bef"}},{"code_sha256_prefix":"2d2c2d56e476e772","entry":"SigLipLoss","repo":"mlfoundations/open_clip","repo_kind":"listed","path":"src/open_clip/loss.py","file_url":"https://github.com/mlfoundations/open_clip/blob/HEAD/src/open_clip/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"2d2c2d56e476e772"}},{"code_sha256_prefix":"59434b67e9a74429","entry":"get_normalization_layer","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"59434b67e9a74429"}},{"code_sha256_prefix":"3586d486792b020d","entry":"postprocess","repo":"merveenoyan/siglip","repo_kind":"listed","path":"clip_siglip.py","file_url":"https://github.com/merveenoyan/siglip/blob/HEAD/clip_siglip.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3586d486792b020d"}},{"code_sha256_prefix":"ad49d2b1bad09561","entry":"postprocess_siglip","repo":"merveenoyan/siglip","repo_kind":"listed","path":"clip_siglip.py","file_url":"https://github.com/merveenoyan/siglip/blob/HEAD/clip_siglip.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ad49d2b1bad09561"}},{"code_sha256_prefix":"eb62100582261eb4","entry":"relic_loss","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"eb62100582261eb4"}},{"code_sha256_prefix":"67c4ef5004396e02","entry":"siglip_loss","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"67c4ef5004396e02"}},{"code_sha256_prefix":"7f362a66c1752126","entry":"CLIP","repo":"ramanakshay/clip","repo_kind":"listed","path":"src/model/model.py","file_url":"https://github.com/ramanakshay/clip/blob/HEAD/src/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f362a66c1752126"}},{"code_sha256_prefix":"a8fa6ab303249a21","entry":"CLIP","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a8fa6ab303249a21"}},{"code_sha256_prefix":"e80ed1f8e5c3f5a8","entry":"CLIPModel","repo":"ramanakshay/clip","repo_kind":"listed","path":"src/model/model.py","file_url":"https://github.com/ramanakshay/clip/blob/HEAD/src/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e80ed1f8e5c3f5a8"}},{"code_sha256_prefix":"df9eaf41a5c09a07","entry":"ConvFFN","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"df9eaf41a5c09a07"}},{"code_sha256_prefix":"579a47ae5bfe8248","entry":"MCi","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"579a47ae5bfe8248"}},{"code_sha256_prefix":"695f06cd58c0597e","entry":"MobileOneBlock","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"695f06cd58c0597e"}},{"code_sha256_prefix":"43cafd406a2da3f3","entry":"RepMixer","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"43cafd406a2da3f3"}},{"code_sha256_prefix":"efcf8d9368ee9d08","entry":"RepMixerBlock","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"efcf8d9368ee9d08"}},{"code_sha256_prefix":"49fac99e1f1a0270","entry":"SEBlock","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"49fac99e1f1a0270"}},{"code_sha256_prefix":"88fa799b7ff64ccb","entry":"TextTransformer","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"88fa799b7ff64ccb"}},{"code_sha256_prefix":"35094b7d325013c5","entry":"TransformerEncoder","repo":"apple/ml-mobileclip","repo_kind":"listed","path":"mobileclip/clip.py","file_url":"https://github.com/apple/ml-mobileclip/blob/HEAD/mobileclip/clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"35094b7d325013c5"}},{"code_sha256_prefix":"e332856e3c2fc814","entry":"neighbour_exchange","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e332856e3c2fc814"}},{"code_sha256_prefix":"5b1fd364afcf3c05","entry":"neighbour_exchange_bidir","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"5b1fd364afcf3c05"}},{"code_sha256_prefix":"4a8006bab44bb658","entry":"neighbour_exchange_bidir_with_grad","repo":"mlfoundations/open_clip","repo_kind":"listed","path":"src/open_clip/loss.py","file_url":"https://github.com/mlfoundations/open_clip/blob/HEAD/src/open_clip/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4a8006bab44bb658"}},{"code_sha256_prefix":"12c8728ab37fdce8","entry":"neighbour_exchange_with_grad","repo":"mlfoundations/open_clip","repo_kind":"listed","path":"src/open_clip/loss.py","file_url":"https://github.com/mlfoundations/open_clip/blob/HEAD/src/open_clip/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"12c8728ab37fdce8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}