{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/all-in-one-image-coding-for-joint-human","title":"All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation","arxiv_id":"2409.19660","date":"2024-09-29","proceeding":null,"authors":["Xu Zhang","Peiyao Guo","Ming Lu","Zhan Ma"],"abstract":"Image coding for multi-task applications, catering to both human perception and machine vision, has been extensively investigated. Existing methods often rely on multiple task-specific encoder-decoder pairs, leading to high overhead of parameter and bitrate usage, or face challenges in multi-objective optimization under a unified representation, failing to achieve both performance and efficiency. To this end, we propose Multi-Path Aggregation (MPA) integrated into existing coding models for joint human-machine vision, unifying the feature representation with an all-in-one architecture. MPA employs a predictor to allocate latent features among task-specific paths based on feature importance varied across tasks, maximizing the utility of shared features while preserving task-specific features for subsequent refinement. Leveraging feature correlations, we develop a two-stage optimization strategy to alleviate multi-task performance degradation. Upon the reuse of shared features, as low as 1.89% parameters are further augmented and fine-tuned for a specific task, which completely avoids extensive optimization of the entire model. Experimental results show that MPA achieves performance comparable to state-of-the-art methods in both task-specific and multi-objective optimization across human viewing and machine analysis tasks. Moreover, our all-in-one design supports seamless transitions between human- and machine-oriented reconstruction, enabling task-controllable interpretation without altering the unified model. Code is available at https://github.com/NJUVISION/MPA.","url_abs":"https://arxiv.org/abs/2409.19660v1","url_pdf":"https://arxiv.org/pdf/2409.19660v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"all-in-one-image-coding-for-joint-human","repo_url":"https://github.com/NJUVISION/MPA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"data-compression","task_name":"Data Compression"},{"task_slug":"feature-importance","task_name":"Feature Importance"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-compression","task_name":"Image Compression"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2409.19660","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.19660"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/NJUVISION/MPA","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":4,"ran_honours":2,"ran":3,"ran_violates":1,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"b47790190bfa0514","entry":"accuracy","repo":"njuvision/mpa","repo_kind":"official","path":"examples/eval_cls_real_bpp.py","file_url":"https://github.com/njuvision/mpa/blob/HEAD/examples/eval_cls_real_bpp.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b47790190bfa0514"}},{"code_sha256_prefix":"7839fb1aca0933e2","entry":"batch_index_fill","repo":"NJUVISION/MPA","repo_kind":"official","path":"compressai/layers/layers.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/compressai/layers/layers.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7839fb1aca0933e2"}},{"code_sha256_prefix":"908619be92da9903","entry":"batch_index_select","repo":"NJUVISION/MPA","repo_kind":"official","path":"compressai/layers/layers.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/compressai/layers/layers.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"908619be92da9903"}},{"code_sha256_prefix":"f2cea3e9848ae52d","entry":"conv","repo":"NJUVISION/MPA","repo_kind":"official","path":"compressai/models/utils.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/compressai/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f2cea3e9848ae52d"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"NJUVISION/MPA","repo_kind":"official","path":"seg_model/resnet.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/seg_model/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"a94389b554ddc179","entry":"find_named_buffer","repo":"NJUVISION/MPA","repo_kind":"official","path":"compressai/models/utils.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/compressai/models/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a94389b554ddc179"}},{"code_sha256_prefix":"feb29b32fc1ec50e","entry":"get_scale_table","repo":"njuvision/mpa","repo_kind":"official","path":"compressai/models/mpa.py","file_url":"https://github.com/njuvision/mpa/blob/HEAD/compressai/models/mpa.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"feb29b32fc1ec50e"}},{"code_sha256_prefix":"cba60dba29f40ee7","entry":"gumbel_sigmoid","repo":"NJUVISION/MPA","repo_kind":"official","path":"compressai/layers/layers.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/compressai/layers/layers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"cba60dba29f40ee7"}},{"code_sha256_prefix":"b8f76281c8efae48","entry":"resnet18","repo":"NJUVISION/MPA","repo_kind":"official","path":"seg_model/resnet.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/seg_model/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"b8f76281c8efae48"}},{"code_sha256_prefix":"81e6f33194021bdd","entry":"resnet34","repo":"NJUVISION/MPA","repo_kind":"official","path":"seg_model/resnet.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/seg_model/resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"81e6f33194021bdd"}},{"code_sha256_prefix":"ecf287c12bb06741","entry":"str2bool","repo":"njuvision/mpa","repo_kind":"official","path":"examples/eval_cls_real_bpp.py","file_url":"https://github.com/njuvision/mpa/blob/HEAD/examples/eval_cls_real_bpp.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ecf287c12bb06741"}},{"code_sha256_prefix":"07013938ed5a3d55","entry":"configure_optimizers","repo":"njuvision/mpa","repo_kind":"official","path":"examples/train_stage1_wo_gan.py","file_url":"https://github.com/njuvision/mpa/blob/HEAD/examples/train_stage1_wo_gan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"07013938ed5a3d55"}},{"code_sha256_prefix":"0b4d7c9d1baaf828","entry":"find_named_module","repo":"NJUVISION/MPA","repo_kind":"official","path":"compressai/models/utils.py","file_url":"https://github.com/NJUVISION/MPA/blob/HEAD/compressai/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b4d7c9d1baaf828"}},{"code_sha256_prefix":"6656404be8df535e","entry":"init","repo":"njuvision/mpa","repo_kind":"official","path":"examples/train_stage1_wo_gan.py","file_url":"https://github.com/njuvision/mpa/blob/HEAD/examples/train_stage1_wo_gan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6656404be8df535e"}},{"code_sha256_prefix":"ffb7e6e59725022b","entry":"init","repo":"njuvision/mpa","repo_kind":"official","path":"examples/eval_cls_real_bpp.py","file_url":"https://github.com/njuvision/mpa/blob/HEAD/examples/eval_cls_real_bpp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ffb7e6e59725022b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}