{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/logic-tensor-networks-for-semantic-image","title":"Logic Tensor Networks for Semantic Image Interpretation","arxiv_id":"1705.08968","date":"2017-05-24","proceeding":null,"authors":["Ivan Donadello","Luciano Serafini","Artur d'Avila Garcez"],"abstract":"Semantic Image Interpretation (SII) is the task of extracting structured\nsemantic descriptions from images. It is widely agreed that the combined use of\nvisual data and background knowledge is of great importance for SII. Recently,\nStatistical Relational Learning (SRL) approaches have been developed for\nreasoning under uncertainty and learning in the presence of data and rich\nknowledge. Logic Tensor Networks (LTNs) are an SRL framework which integrates\nneural networks with first-order fuzzy logic to allow (i) efficient learning\nfrom noisy data in the presence of logical constraints, and (ii) reasoning with\nlogical formulas describing general properties of the data. In this paper, we\ndevelop and apply LTNs to two of the main tasks of SII, namely, the\nclassification of an image's bounding boxes and the detection of the relevant\npart-of relations between objects. To the best of our knowledge, this is the\nfirst successful application of SRL to such SII tasks. The proposed approach is\nevaluated on a standard image processing benchmark. Experiments show that the\nuse of background knowledge in the form of logical constraints can improve the\nperformance of purely data-driven approaches, including the state-of-the-art\nFast Region-based Convolutional Neural Networks (Fast R-CNN). Moreover, we show\nthat the use of logical background knowledge adds robustness to the learning\nsystem when errors are present in the labels of the training data.","url_abs":"http://arxiv.org/abs/1705.08968v1","url_pdf":"http://arxiv.org/pdf/1705.08968v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"logic-tensor-networks-for-semantic-image","repo_url":"https://github.com/sbadredd/semantic-pascal-part","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"relational-reasoning","task_name":"Relational Reasoning"},{"task_slug":"tensor-networks","task_name":"Tensor Networks"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.08968","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}