{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sketchparse-towards-rich-descriptions-for","title":"SketchParse : Towards Rich Descriptions for Poorly Drawn Sketches using Multi-Task Hierarchical Deep Networks","arxiv_id":"1709.01295","date":"2017-09-05","proceeding":null,"authors":["Ravi Kiran Sarvadevabhatla","Isht Dwivedi","Abhijat Biswas","Sahil Manocha","R. Venkatesh Babu"],"abstract":"The ability to semantically interpret hand-drawn line sketches, although very\nchallenging, can pave way for novel applications in multimedia. We propose\nSketchParse, the first deep-network architecture for fully automatic parsing of\nfreehand object sketches. SketchParse is configured as a two-level fully\nconvolutional network. The first level contains shared layers common to all\nobject categories. The second level contains a number of expert sub-networks.\nEach expert specializes in parsing sketches from object categories which\ncontain structurally similar parts. Effectively, the two-level configuration\nenables our architecture to scale up efficiently as additional categories are\nadded. We introduce a router layer which (i) relays sketch features from shared\nlayers to the correct expert (ii) eliminates the need to manually specify\nobject category during inference. To bypass laborious part-level annotation, we\nsketchify photos from semantic object-part image datasets and use them for\ntraining. Our architecture also incorporates object pose prediction as a novel\nauxiliary task which boosts overall performance while providing supplementary\ninformation regarding the sketch. We demonstrate SketchParse's abilities (i) on\ntwo challenging large-scale sketch datasets (ii) in parsing unseen,\nsemantically related object categories (iii) in improving fine-grained\nsketch-based image retrieval. As a novel application, we also outline how\nSketchParse's output can be used to generate caption-style descriptions for\nhand-drawn sketches.","url_abs":"http://arxiv.org/abs/1709.01295v1","url_pdf":"http://arxiv.org/pdf/1709.01295v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sketchparse-towards-rich-descriptions-for","repo_url":"https://github.com/val-iisc/sketch-parse","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"object","task_name":"Object"},{"task_slug":"pose-prediction","task_name":"Pose Prediction"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"sketch-based-image-retrieval","task_name":"Sketch-Based Image Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}