{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/markerless-tracking-of-user-defined-features","title":"Markerless tracking of user-defined features with deep learning","arxiv_id":"1804.03142","date":"2018-04-09","proceeding":null,"authors":["Alexander Mathis","Pranav Mamidanna","Taiga Abe","Kevin M. Cury","Venkatesh N. Murthy","Mackenzie W. Mathis","Matthias Bethge"],"abstract":"Quantifying behavior is crucial for many applications in neuroscience.\nVideography provides easy methods for the observation and recording of animal\nbehavior in diverse settings, yet extracting particular aspects of a behavior\nfor further analysis can be highly time consuming. In motor control studies,\nhumans or other animals are often marked with reflective markers to assist with\ncomputer-based tracking, yet markers are intrusive (especially for smaller\nanimals), and the number and location of the markers must be determined a\npriori. Here, we present a highly efficient method for markerless tracking\nbased on transfer learning with deep neural networks that achieves excellent\nresults with minimal training data. We demonstrate the versatility of this\nframework by tracking various body parts in a broad collection of experimental\nsettings: mice odor trail-tracking, egg-laying behavior in drosophila, and\nmouse hand articulation in a skilled forelimb task. For example, during the\nskilled reaching behavior, individual joints can be automatically tracked (and\na confidence score is reported). Remarkably, even when a small number of frames\nare labeled ($\\approx 200$), the algorithm achieves excellent tracking\nperformance on test frames that is comparable to human accuracy.","url_abs":"http://arxiv.org/abs/1804.03142v1","url_pdf":"http://arxiv.org/pdf/1804.03142v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"markerless-tracking-of-user-defined-features","repo_url":"https://github.com/DeepLabCut/DeepLabCut","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"LGPL-3.0"}}],"tasks":[{"task_slug":"animal-pose-estimation","task_name":"Animal Pose Estimation"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.03142","atlas_url":"https://app.syntology.ai/?focus=1804.03142","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}