{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/online-real-time-multiple-spatiotemporal","title":"Online Real-time Multiple Spatiotemporal Action Localisation and Prediction","arxiv_id":"1611.08563","date":"2016-11-25","proceeding":"ICCV 2017 10","authors":["Gurkirt Singh","Suman Saha","Michael Sapienza","Philip Torr","Fabio Cuzzolin"],"abstract":"We present a deep-learning framework for real-time multiple spatio-temporal\n(S/T) action localisation, classification and early prediction. Current\nstate-of-the-art approaches work offline and are too slow to be useful in real-\nworld settings. To overcome their limitations we introduce two major\ndevelopments. Firstly, we adopt real-time SSD (Single Shot MultiBox Detector)\nconvolutional neural networks to regress and classify detection boxes in each\nvideo frame potentially containing an action of interest. Secondly, we design\nan original and efficient online algorithm to incrementally construct and label\n`action tubes' from the SSD frame level detections. As a result, our system is\nnot only capable of performing S/T detection in real time, but can also perform\nearly action prediction in an online fashion. We achieve new state-of-the-art\nresults in both S/T action localisation and early action prediction on the\nchallenging UCF101-24 and J-HMDB-21 benchmarks, even when compared to the top\noffline competitors. To the best of our knowledge, ours is the first real-time\n(up to 40fps) system able to perform online S/T action localisation and early\naction prediction on the untrimmed videos of UCF101-24.","url_abs":"http://arxiv.org/abs/1611.08563v6","url_pdf":"http://arxiv.org/pdf/1611.08563v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"online-real-time-multiple-spatiotemporal","repo_url":"https://github.com/gurkirt/realtime-action-detection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"online-real-time-multiple-spatiotemporal","repo_url":"https://github.com/gurkirt/corrected-UCF101-Annots","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"online-real-time-multiple-spatiotemporal","repo_url":"https://github.com/gurkirt/3D-RetinaNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"online-real-time-multiple-spatiotemporal","repo_url":"https://github.com/gurkirt/AMTNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"online-real-time-multiple-spatiotemporal","repo_url":"https://github.com/salmank255/ROAD_Waymo_Baseline","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"early-action-prediction","task_name":"Early Action Prediction"},{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"non-maximum-suppression","method_name":"Non Maximum Suppression"},{"method_slug":"ssd","method_name":"SSD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1611.08563","atlas_url":"https://app.syntology.ai/?focus=1611.08563","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1611.08563"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/salmank255/ROAD_Waymo_Baseline","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gurkirt/3D-RetinaNet","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gurkirt/realtime-action-detection","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gurkirt/corrected-UCF101-Annots","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gurkirt/AMTNet","reach":null}],"summary":{"ran_draft_wrong":5,"ran_fixture":1,"ran_violates":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"2a8545e480332b47","entry":"add_extras","repo":"gurkirt/realtime-action-detection","repo_kind":"official","path":"ssd.py","file_url":"https://github.com/gurkirt/realtime-action-detection/blob/HEAD/ssd.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a8545e480332b47"}},{"code_sha256_prefix":"36cb625ab21d7c5c","entry":"add_extras","repo":"gurkirt/AMTNet","repo_kind":"listed","path":"AMTNet.py","file_url":"https://github.com/gurkirt/AMTNet/blob/HEAD/AMTNet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"36cb625ab21d7c5c"}},{"code_sha256_prefix":"40dc7296d52276ff","entry":"multibox","repo":"gurkirt/realtime-action-detection","repo_kind":"official","path":"ssd.py","file_url":"https://github.com/gurkirt/realtime-action-detection/blob/HEAD/ssd.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40dc7296d52276ff"}},{"code_sha256_prefix":"e7cae6a12359a3e9","entry":"multibox","repo":"gurkirt/AMTNet","repo_kind":"listed","path":"AMTNet.py","file_url":"https://github.com/gurkirt/AMTNet/blob/HEAD/AMTNet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e7cae6a12359a3e9"}},{"code_sha256_prefix":"25c7475539e39da4","entry":"str2bool","repo":"gurkirt/realtime-action-detection","repo_kind":"official","path":"test-ucf24.py","file_url":"https://github.com/gurkirt/realtime-action-detection/blob/HEAD/test-ucf24.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25c7475539e39da4"}},{"code_sha256_prefix":"6921f7ed00595a4f","entry":"vgg","repo":"gurkirt/realtime-action-detection","repo_kind":"official","path":"ssd.py","file_url":"https://github.com/gurkirt/realtime-action-detection/blob/HEAD/ssd.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6921f7ed00595a4f"}},{"code_sha256_prefix":"ff68a2d579e9a3cf","entry":"vgg","repo":"gurkirt/AMTNet","repo_kind":"listed","path":"AMTNet.py","file_url":"https://github.com/gurkirt/AMTNet/blob/HEAD/AMTNet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ff68a2d579e9a3cf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}