{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adcare-vlm-leveraging-large-vision-language","title":"AdCare-VLM: Leveraging Large Vision Language Model (LVLM) to Monitor Long-Term Medication Adherence and Care","arxiv_id":"2505.00275","date":"2025-05-01","proceeding":null,"authors":["Md Asaduzzaman Jabin","Hanqi Jiang","Yiwei Li","Patrick Kaggwa","Eugene Douglass","Juliet N. Sekandi","Tianming Liu"],"abstract":"Chronic diseases, including diabetes, hypertension, asthma, HIV-AIDS, epilepsy, and tuberculosis, necessitate rigorous adherence to medication to avert disease progression, manage symptoms, and decrease mortality rates. Adherence is frequently undermined by factors including patient behavior, caregiver support, elevated medical costs, and insufficient healthcare infrastructure. We propose AdCare-VLM, a specialized Video-LLaVA-based multimodal large vision language model (LVLM) aimed at visual question answering (VQA) concerning medication adherence through patient videos. We employ a private dataset comprising 806 custom-annotated tuberculosis (TB) medication monitoring videos, which have been labeled by clinical experts, to fine-tune the model for adherence pattern detection. We present LLM-TB-VQA, a detailed medical adherence VQA dataset that encompasses positive, negative, and ambiguous adherence cases. Our method identifies correlations between visual features, such as the clear visibility of the patient's face, medication, water intake, and the act of ingestion, and their associated medical concepts in captions. This facilitates the integration of aligned visual-linguistic representations and improves multimodal interactions. Experimental results indicate that our method surpasses parameter-efficient fine-tuning (PEFT) enabled VLM models, such as LLaVA-V1.5 and Chat-UniVi, with absolute improvements ranging from 3.1% to 3.54% across pre-trained, regular, and low-rank adaptation (LoRA) configurations. Comprehensive ablation studies and attention map visualizations substantiate our approach, enhancing interpretability.","url_abs":"https://arxiv.org/abs/2505.00275v1","url_pdf":"https://arxiv.org/pdf/2505.00275v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adcare-vlm-leveraging-large-vision-language","repo_url":"https://github.com/asad14053/AdCare-VLM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.00275","atlas_url":"https://app.syntology.ai/?focus=2505.00275","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.00275"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asad14053/AdCare-VLM","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2,"unverified":7},"by_repo_kind":{"official":{"samples":10,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"07371c331d41d4c5","entry":"LanguageBindImageTower","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07371c331d41d4c5"}},{"code_sha256_prefix":"745b178342051072","entry":"build_video_tower","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"745b178342051072"}},{"code_sha256_prefix":"ef4db0da176c674c","entry":"build_vision_projector","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef4db0da176c674c"}},{"code_sha256_prefix":"6422a6162a5e6169","entry":"CLIPVisionTower","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6422a6162a5e6169"}},{"code_sha256_prefix":"8e7e7423f1e4a61f","entry":"LanguageBindVideoTower","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e7e7423f1e4a61f"}},{"code_sha256_prefix":"8bf0e6b04d962b11","entry":"LlavaLlamaForCausalLM","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bf0e6b04d962b11"}},{"code_sha256_prefix":"b77d45f0407b91ff","entry":"LlavaLlamaModel","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b77d45f0407b91ff"}},{"code_sha256_prefix":"236fb9c779e786df","entry":"LlavaMetaForCausalLM","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"236fb9c779e786df"}},{"code_sha256_prefix":"e57c28d468e2fe09","entry":"LlavaMetaModel","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e57c28d468e2fe09"}},{"code_sha256_prefix":"408c940a53f97c09","entry":"build_image_tower","repo":"asad14053/AdCare-VLM","repo_kind":"official","path":"videollava/model/language_model/llava_llama.py","file_url":"https://github.com/asad14053/AdCare-VLM/blob/HEAD/videollava/model/language_model/llava_llama.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"408c940a53f97c09"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}