{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/challenges-in-the-automatic-analysis-of","title":"Challenges in the Automatic Analysis of Students' Diagnostic Reasoning","arxiv_id":"1811.10550","date":"2018-11-26","proceeding":null,"authors":["Claudia Schulz","Christian M. Meyer","Michael Sailer","Jan Kiesewetter","Elisabeth Bauer","Frank Fischer","Martin R. Fischer","Iryna Gurevych"],"abstract":"Diagnostic reasoning is a key component of many professions. To improve\nstudents' diagnostic reasoning skills, educational psychologists analyse and\ngive feedback on epistemic activities used by these students while diagnosing,\nin particular, hypothesis generation, evidence generation, evidence evaluation,\nand drawing conclusions. However, this manual analysis is highly\ntime-consuming. We aim to enable the large-scale adoption of diagnostic\nreasoning analysis and feedback by automating the epistemic activity\nidentification. We create the first corpus for this task, comprising diagnostic\nreasoning self-explanations of students from two domains annotated with\nepistemic activities. Based on insights from the corpus creation and the task's\ncharacteristics, we discuss three challenges for the automatic identification\nof epistemic activities using AI methods: the correct identification of\nepistemic activity spans, the reliable distinction of similar epistemic\nactivities, and the detection of overlapping epistemic activities. We propose a\nseparate performance metric for each challenge and thus provide an evaluation\nframework for future research. Indeed, our evaluation of various\nstate-of-the-art recurrent neural network architectures reveals that current\ntechniques fail to address some of these challenges.","url_abs":"http://arxiv.org/abs/1811.10550v1","url_pdf":"http://arxiv.org/pdf/1811.10550v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"challenges-in-the-automatic-analysis-of","repo_url":"https://github.com/UKPLab/aaai19-diagnostic-reasoning","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1811.10550","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}