{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/molmix-a-simple-yet-effective-baseline-for","title":"MolMix: A Simple Yet Effective Baseline for Multimodal Molecular Representation Learning","arxiv_id":"2410.07981","date":"2024-10-10","proceeding":null,"authors":["Andrei Manolache","Dragos Tantaru","Mathias Niepert"],"abstract":"In this work, we propose a simple transformer-based baseline for multimodal molecular representation learning, integrating three distinct modalities: SMILES strings, 2D graph representations, and 3D conformers of molecules. A key aspect of our approach is the aggregation of 3D conformers, allowing the model to account for the fact that molecules can adopt multiple conformations-an important factor for accurate molecular representation. The tokens for each modality are extracted using modality-specific encoders: a transformer for SMILES strings, a message-passing neural network for 2D graphs, and an equivariant neural network for 3D conformers. The flexibility and modularity of this framework enable easy adaptation and replacement of these encoders, making the model highly versatile for different molecular tasks. The extracted tokens are then combined into a unified multimodal sequence, which is processed by a downstream transformer for prediction tasks. To efficiently scale our model for large multimodal datasets, we utilize Flash Attention 2 and bfloat16 precision. Despite its simplicity, our approach achieves state-of-the-art results across multiple datasets, demonstrating its effectiveness as a strong baseline for multimodal molecular representation learning.","url_abs":"https://arxiv.org/abs/2410.07981v2","url_pdf":"https://arxiv.org/pdf/2410.07981v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"molmix-a-simple-yet-effective-baseline-for","repo_url":"https://github.com/andreimano/MolMix","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"molecular-representation","task_name":"molecular representation"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.07981","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.07981"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/andreimano/MolMix","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b4119914cc11c41e","entry":"Jn","repo":"andreimano/MolMix","repo_kind":"official","path":"layers/gemnet_layers/basis_utils.py","file_url":"https://github.com/andreimano/MolMix/blob/HEAD/layers/gemnet_layers/basis_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4119914cc11c41e"}},{"code_sha256_prefix":"d3c0454b979dd9e0","entry":"Jn_zeros","repo":"andreimano/MolMix","repo_kind":"official","path":"layers/gemnet_layers/basis_utils.py","file_url":"https://github.com/andreimano/MolMix/blob/HEAD/layers/gemnet_layers/basis_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3c0454b979dd9e0"}},{"code_sha256_prefix":"1c163e2b8e374294","entry":"get_edge_encoder","repo":"andreimano/MolMix","repo_kind":"official","path":"layers/encoders.py","file_url":"https://github.com/andreimano/MolMix/blob/HEAD/layers/encoders.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c163e2b8e374294"}},{"code_sha256_prefix":"c94ad99c5f6191c3","entry":"he_orthogonal_init","repo":"andreimano/MolMix","repo_kind":"official","path":"layers/gemnet_layers/base_layers.py","file_url":"https://github.com/andreimano/MolMix/blob/HEAD/layers/gemnet_layers/base_layers.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c94ad99c5f6191c3"}},{"code_sha256_prefix":"6a886f8b611f61b7","entry":"spherical_bessel_formulas","repo":"andreimano/MolMix","repo_kind":"official","path":"layers/gemnet_layers/basis_utils.py","file_url":"https://github.com/andreimano/MolMix/blob/HEAD/layers/gemnet_layers/basis_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a886f8b611f61b7"}},{"code_sha256_prefix":"002f85b24d33fbf9","entry":"cat_list_varlen","repo":"andreimano/MolMix","repo_kind":"official","path":"layers/models.py","file_url":"https://github.com/andreimano/MolMix/blob/HEAD/layers/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"002f85b24d33fbf9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}