{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-learning-under-privileged-information","title":"Deep Learning under Privileged Information Using Heteroscedastic Dropout","arxiv_id":"1805.11614","date":"2018-05-29","proceeding":"CVPR 2018 6","authors":["John Lambert","Ozan Sener","Silvio Savarese"],"abstract":"Unlike machines, humans learn through rapid, abstract model-building. The\nrole of a teacher is not simply to hammer home right or wrong answers, but\nrather to provide intuitive comments, comparisons, and explanations to a pupil.\nThis is what the Learning Under Privileged Information (LUPI) paradigm\nendeavors to model by utilizing extra knowledge only available during training.\nWe propose a new LUPI algorithm specifically designed for Convolutional Neural\nNetworks (CNNs) and Recurrent Neural Networks (RNNs). We propose to use a\nheteroscedastic dropout (i.e. dropout with a varying variance) and make the\nvariance of the dropout a function of privileged information. Intuitively, this\ncorresponds to using the privileged information to control the uncertainty of\nthe model output. We perform experiments using CNNs and RNNs for the tasks of\nimage classification and machine translation. Our method significantly\nincreases the sample efficiency during learning, resulting in higher accuracy\nwith a large margin when the number of training examples is limited. We also\ntheoretically justify the gains in sample efficiency by providing a\ngeneralization error bound decreasing with $O(\\frac{1}{n})$, where $n$ is the\nnumber of training examples, in an oracle case.","url_abs":"http://arxiv.org/abs/1805.11614v1","url_pdf":"http://arxiv.org/pdf/1805.11614v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-learning-under-privileged-information","repo_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1805.11614","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1805.11614"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"johnwlambert/dlupi-heteroscedastic-dropout","repo_kind":"official","path":"cnns/base_networks/resnet.py","file_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout/blob/HEAD/cnns/base_networks/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"ac62432dc5134b0d","entry":"make_layers","repo":"johnwlambert/dlupi-heteroscedastic-dropout","repo_kind":"official","path":"cnns/base_networks/random_gauss_dropout_vgg.py","file_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout/blob/HEAD/cnns/base_networks/random_gauss_dropout_vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac62432dc5134b0d"}},{"code_sha256_prefix":"ea507a2fc08fc9dc","entry":"resnet152","repo":"johnwlambert/dlupi-heteroscedastic-dropout","repo_kind":"official","path":"cnns/base_networks/resnet.py","file_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout/blob/HEAD/cnns/base_networks/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea507a2fc08fc9dc"}},{"code_sha256_prefix":"c4e826c3aa297821","entry":"resnet50","repo":"johnwlambert/dlupi-heteroscedastic-dropout","repo_kind":"official","path":"cnns/base_networks/resnet.py","file_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout/blob/HEAD/cnns/base_networks/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4e826c3aa297821"}},{"code_sha256_prefix":"9df6792345c6f27b","entry":"vgg11","repo":"johnwlambert/dlupi-heteroscedastic-dropout","repo_kind":"official","path":"cnns/base_networks/vgg.py","file_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout/blob/HEAD/cnns/base_networks/vgg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9df6792345c6f27b"}},{"code_sha256_prefix":"e4905952077a97ff","entry":"vgg13","repo":"johnwlambert/dlupi-heteroscedastic-dropout","repo_kind":"official","path":"cnns/base_networks/vgg.py","file_url":"https://github.com/johnwlambert/dlupi-heteroscedastic-dropout/blob/HEAD/cnns/base_networks/vgg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4905952077a97ff"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}