{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visil-fine-grained-spatio-temporal-video","title":"ViSiL: Fine-grained Spatio-Temporal Video Similarity Learning","arxiv_id":"1908.07410","date":"2019-08-20","proceeding":"ICCV 2019 10","authors":["Giorgos Kordopatis-Zilos","Symeon Papadopoulos","Ioannis Patras","Ioannis Kompatsiaris"],"abstract":"In this paper we introduce ViSiL, a Video Similarity Learning architecture that considers fine-grained Spatio-Temporal relations between pairs of videos -- such relations are typically lost in previous video retrieval approaches that embed the whole frame or even the whole video into a vector descriptor before the similarity estimation. By contrast, our Convolutional Neural Network (CNN)-based approach is trained to calculate video-to-video similarity from refined frame-to-frame similarity matrices, so as to consider both intra- and inter-frame relations. In the proposed method, pairwise frame similarity is estimated by applying Tensor Dot (TD) followed by Chamfer Similarity (CS) on regional CNN frame features - this avoids feature aggregation before the similarity calculation between frames. Subsequently, the similarity matrix between all video frames is fed to a four-layer CNN, and then summarized using Chamfer Similarity (CS) into a video-to-video similarity score -- this avoids feature aggregation before the similarity calculation between videos and captures the temporal similarity patterns between matching frame sequences. We train the proposed network using a triplet loss scheme and evaluate it on five public benchmark datasets on four different video retrieval problems where we demonstrate large improvements in comparison to the state of the art. The implementation of ViSiL is publicly available.","url_abs":"https://arxiv.org/abs/1908.07410v1","url_pdf":"https://arxiv.org/pdf/1908.07410v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visil-fine-grained-spatio-temporal-video","repo_url":"https://github.com/MKLab-ITI/visil","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"ISVR"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":null,"task_name":"Triplet"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"},{"task_slug":"video-similarity","task_name":"Video Similarity"}],"methods":[{"method_slug":"triplet-loss","method_name":"Triplet Loss"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-retrieval-on-fivr-200k","task":"Video Retrieval","dataset":"FIVR-200K","model":"ViSiLv (pt)","rank_in_archive_order":5,"of":17,"metrics":{"mAP (CSVR)":"0.854","mAP (DSVR)":"0.899","mAP (ISVR)":"0.723"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-fivr-200k","task":"Video Retrieval","dataset":"FIVR-200K","model":"ViSiLv (tf)","rank_in_archive_order":7,"of":17,"metrics":{"mAP (CSVR)":"0.841 ","mAP (DSVR)":"0.892","mAP (ISVR)":"0.702"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-fivr-200k","task":"Video Retrieval","dataset":"FIVR-200K","model":"ViSiLf","rank_in_archive_order":9,"of":17,"metrics":{"mAP (CSVR)":"0.797","mAP (DSVR)":"0.843","mAP (ISVR)":"0.660"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-fivr-200k","task":"Video Retrieval","dataset":"FIVR-200K","model":"ViSiLsym","rank_in_archive_order":10,"of":17,"metrics":{"mAP (CSVR)":"0.792","mAP (DSVR)":"0.833","mAP (ISVR)":"0.654"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1908.07410","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1908.07410"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MKLab-ITI/visil","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1495cfaac81f2e80","entry":"chamfer_similarity","repo":"MKLab-ITI/visil","repo_kind":"official","path":"model/similarity.py","file_url":"https://github.com/MKLab-ITI/visil/blob/HEAD/model/similarity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1495cfaac81f2e80"}},{"code_sha256_prefix":"b35644780c2d36b7","entry":"preprocess_for_eval","repo":"MKLab-ITI/visil","repo_kind":"official","path":"model/nets/vgg_preprocessing.py","file_url":"https://github.com/MKLab-ITI/visil/blob/HEAD/model/nets/vgg_preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b35644780c2d36b7"}},{"code_sha256_prefix":"5348d906683c3ddb","entry":"preprocess_for_train","repo":"MKLab-ITI/visil","repo_kind":"official","path":"model/nets/vgg_preprocessing.py","file_url":"https://github.com/MKLab-ITI/visil/blob/HEAD/model/nets/vgg_preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5348d906683c3ddb"}},{"code_sha256_prefix":"f9400a8b074c96ca","entry":"preprocess_image","repo":"MKLab-ITI/visil","repo_kind":"official","path":"model/nets/vgg_preprocessing.py","file_url":"https://github.com/MKLab-ITI/visil/blob/HEAD/model/nets/vgg_preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9400a8b074c96ca"}},{"code_sha256_prefix":"60fc9288d2497144","entry":"symmetric_chamfer_similarity","repo":"MKLab-ITI/visil","repo_kind":"official","path":"model/similarity.py","file_url":"https://github.com/MKLab-ITI/visil/blob/HEAD/model/similarity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"60fc9288d2497144"}},{"code_sha256_prefix":"cb3be0da48af7c2b","entry":"triplet_loss","repo":"MKLab-ITI/visil","repo_kind":"official","path":"model/similarity.py","file_url":"https://github.com/MKLab-ITI/visil/blob/HEAD/model/similarity.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cb3be0da48af7c2b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}