{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bobcat-bilevel-optimization-based","title":"BOBCAT: Bilevel Optimization-Based Computerized Adaptive Testing","arxiv_id":"2108.07386","date":"2021-08-17","proceeding":null,"authors":["Aritra Ghosh","Andrew Lan"],"abstract":"Computerized adaptive testing (CAT) refers to a form of tests that are personalized to every student/test taker. CAT methods adaptively select the next most informative question/item for each student given their responses to previous questions, effectively reducing test length. Existing CAT methods use item response theory (IRT) models to relate student ability to their responses to questions and static question selection algorithms designed to reduce the ability estimation error as quickly as possible; therefore, these algorithms cannot improve by learning from large-scale student response data. In this paper, we propose BOBCAT, a Bilevel Optimization-Based framework for CAT to directly learn a data-driven question selection algorithm from training data. BOBCAT is agnostic to the underlying student response model and is computationally efficient during the adaptive testing process. Through extensive experiments on five real-world student response datasets, we show that BOBCAT outperforms existing CAT methods (sometimes significantly) at reducing test length.","url_abs":"https://arxiv.org/abs/2108.07386v1","url_pdf":"https://arxiv.org/pdf/2108.07386v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bobcat-bilevel-optimization-based","repo_url":"https://github.com/arghosh/bobcat","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"bobcat-bilevel-optimization-based","repo_url":"https://github.com/arghosh/NeurIPSEducation2020","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bilevel-optimization","task_name":"Bilevel Optimization"},{"task_slug":"question-selection","task_name":"Question Selection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.07386","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.07386"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arghosh/NeurIPSEducation2020","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arghosh/bobcat","reach":null}],"summary":{"ran":4,"ran_fixture":2,"ran_draft_wrong":3,"ran_honours":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1},"listed":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b86faf0fe0426b7f","entry":"AttentionModel","repo":"arghosh/NeurIPSEducation2020","repo_kind":"listed","path":"model_task_1_2.py","file_url":"https://github.com/arghosh/NeurIPSEducation2020/blob/HEAD/model_task_1_2.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b86faf0fe0426b7f"}},{"code_sha256_prefix":"7a466f4e8e785035","entry":"CosinePositionalEmbedding","repo":"arghosh/NeurIPSEducation2020","repo_kind":"listed","path":"model_task_1_2.py","file_url":"https://github.com/arghosh/NeurIPSEducation2020/blob/HEAD/model_task_1_2.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a466f4e8e785035"}},{"code_sha256_prefix":"1a5ae3c194622d14","entry":"MAMLModel","repo":"arghosh/bobcat","repo_kind":"official","path":"model.py","file_url":"https://github.com/arghosh/bobcat/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a5ae3c194622d14"}},{"code_sha256_prefix":"e28f6e6239277b44","entry":"MultiHeadAttention","repo":"arghosh/NeurIPSEducation2020","repo_kind":"listed","path":"model_task_1_2.py","file_url":"https://github.com/arghosh/NeurIPSEducation2020/blob/HEAD/model_task_1_2.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e28f6e6239277b44"}},{"code_sha256_prefix":"1abee56c1c82b3be","entry":"attention","repo":"arghosh/NeurIPSEducation2020","repo_kind":"listed","path":"model_task_1_2.py","file_url":"https://github.com/arghosh/NeurIPSEducation2020/blob/HEAD/model_task_1_2.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1abee56c1c82b3be"}},{"code_sha256_prefix":"ddaa3dd8a024013d","entry":"compute_loss","repo":"arghosh/bobcat","repo_kind":"official","path":"model.py","file_url":"https://github.com/arghosh/bobcat/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ddaa3dd8a024013d"}},{"code_sha256_prefix":"bf10b1462929ad81","entry":"get_inputs","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"bf10b1462929ad81"}},{"code_sha256_prefix":"98d34936f4d8072b","entry":"get_outputs","repo":"arghosh/bobcat","repo_kind":"official","path":"model.py","file_url":"https://github.com/arghosh/bobcat/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98d34936f4d8072b"}},{"code_sha256_prefix":"b8d6b34baa8e618a","entry":"normalize_loss","repo":"arghosh/bobcat","repo_kind":"official","path":"model.py","file_url":"https://github.com/arghosh/bobcat/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b8d6b34baa8e618a"}},{"code_sha256_prefix":"5e6b5e316fc4d0d6","entry":"pick_random_sample","repo":"arghosh/bobcat","repo_kind":"official","path":"model.py","file_url":"https://github.com/arghosh/bobcat/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e6b5e316fc4d0d6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}