{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/computing-aic-for-black-box-models-using","title":"Computing AIC for black-box models using Generalised Degrees of Freedom: a comparison with cross-validation","arxiv_id":"1603.02743","date":"2016-03-09","proceeding":null,"authors":["Severin Hauenstein","Carsten F. Dormann","Simon N. Wood"],"abstract":"Generalised Degrees of Freedom (GDF), as defined by Ye (1998 JASA\n93:120-131), represent the sensitivity of model fits to perturbations of the\ndata. As such they can be computed for any statistical model, making it\npossible, in principle, to derive the number of parameters in machine-learning\napproaches. Defined originally for normally distributed data only, we here\ninvestigate the potential of this approach for Bernoulli-data. GDF-values for\nmodels of simulated and real data are compared to model complexity-estimates\nfrom cross-validation. Similarly, we computed GDF-based AICc for randomForest,\nneural networks and boosted regression trees and demonstrated its similarity to\ncross-validation. GDF-estimates for binary data were unstable and\ninconsistently sensitive to the number of data points perturbed simultaneously,\nwhile at the same time being extremely computer-intensive in their calculation.\nRepeated 10-fold cross-validation was more robust, based on fewer assumptions\nand faster to compute. Our findings suggest that the GDF-approach does not\nreadily transfer to Bernoulli data and a wider range of regression approaches.","url_abs":"http://arxiv.org/abs/1603.02743v1","url_pdf":"http://arxiv.org/pdf/1603.02743v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"computing-aic-for-black-box-models-using","repo_url":"https://github.com/biometry/GDF","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}