{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-tokenizer-and-decoder-in-masked-1","title":"Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules","arxiv_id":"2310.14753","date":"2023-10-23","proceeding":"NeurIPS 2023 11","authors":["Zhiyuan Liu","Yaorui Shi","An Zhang","Enzhi Zhang","Kenji Kawaguchi","Xiang Wang","Tat-Seng Chua"],"abstract":"Masked graph modeling excels in the self-supervised representation learning of molecular graphs. Scrutinizing previous studies, we can reveal a common scheme consisting of three key components: (1) graph tokenizer, which breaks a molecular graph into smaller fragments (i.e., subgraphs) and converts them into tokens; (2) graph masking, which corrupts the graph with masks; (3) graph autoencoder, which first applies an encoder on the masked graph to generate the representations, and then employs a decoder on the representations to recover the tokens of the original graph. However, the previous MGM studies focus extensively on graph masking and encoder, while there is limited understanding of tokenizer and decoder. To bridge the gap, we first summarize popular molecule tokenizers at the granularity of node, edge, motif, and Graph Neural Networks (GNNs), and then examine their roles as the MGM's reconstruction targets. Further, we explore the potential of adopting an expressive decoder in MGM. Our results show that a subgraph-level tokenizer and a sufficiently expressive decoder with remask decoding have a large impact on the encoder's representation learning. Finally, we propose a novel MGM method SimSGT, featuring a Simple GNN-based Tokenizer (SGT) and an effective decoding strategy. We empirically validate that our method outperforms the existing molecule self-supervised learning methods. Our codes and checkpoints are available at https://github.com/syr-cn/SimSGT.","url_abs":"https://arxiv.org/abs/2310.14753v2","url_pdf":"https://arxiv.org/pdf/2310.14753v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-tokenizer-and-decoder-in-masked-1","repo_url":"https://github.com/syr-cn/simsgt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.14753","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.14753"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/syr-cn/SimSGT","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/syr-cn/simsgt","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_draft_wrong":1,"ran_honours":1,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"bacb3b25951416fd","entry":"drop_nodes","repo":"syr-cn/simsgt","repo_kind":"official","path":"chem/dataloader.py","file_url":"https://github.com/syr-cn/simsgt/blob/HEAD/chem/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bacb3b25951416fd"}},{"code_sha256_prefix":"535cc0f8059b3dda","entry":"get_activation","repo":"syr-cn/SimSGT","repo_kind":"official","path":"chem/model.py","file_url":"https://github.com/syr-cn/SimSGT/blob/HEAD/chem/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"535cc0f8059b3dda"}},{"code_sha256_prefix":"287484b87212f47d","entry":"graph_data_obj_to_nx_simple","repo":"syr-cn/simsgt","repo_kind":"official","path":"chem/loader.py","file_url":"https://github.com/syr-cn/simsgt/blob/HEAD/chem/loader.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"287484b87212f47d"}},{"code_sha256_prefix":"233f83f1418b20df","entry":"block_mask","repo":"syr-cn/simsgt","repo_kind":"official","path":"chem/dataloader.py","file_url":"https://github.com/syr-cn/simsgt/blob/HEAD/chem/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"233f83f1418b20df"}},{"code_sha256_prefix":"c9a2b2998ed108da","entry":"get_subgraph_id","repo":"syr-cn/simsgt","repo_kind":"official","path":"chem/cal_pred_acc.py","file_url":"https://github.com/syr-cn/simsgt/blob/HEAD/chem/cal_pred_acc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9a2b2998ed108da"}},{"code_sha256_prefix":"04a0caf58c30b529","entry":"preproces","repo":"syr-cn/simsgt","repo_kind":"official","path":"chem/cal_pred_acc.py","file_url":"https://github.com/syr-cn/simsgt/blob/HEAD/chem/cal_pred_acc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"04a0caf58c30b529"}},{"code_sha256_prefix":"49adc441cd42d7af","entry":"subgraph","repo":"syr-cn/simsgt","repo_kind":"official","path":"chem/dataloader.py","file_url":"https://github.com/syr-cn/simsgt/blob/HEAD/chem/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49adc441cd42d7af"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}