{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pipelineprofiler-a-visual-analytics-tool-for","title":"PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines","arxiv_id":"2005.00160","date":"2020-05-01","proceeding":null,"authors":["Jorge Piazentin Ono","Sonia Castelo","Roque Lopez","Enrico Bertini","Juliana Freire","Claudio Silva"],"abstract":"In recent years, a wide variety of automated machine learning (AutoML) methods have been proposed to search and generate end-to-end learning pipelines. While these techniques facilitate the creation of models for real-world applications, given their black-box nature, the complexity of the underlying algorithms, and the large number of pipelines they derive, it is difficult for their developers to debug these systems. It is also challenging for machine learning experts to select an AutoML system that is well suited for a given problem or class of problems. In this paper, we present the PipelineProfiler, an interactive visualization tool that allows the exploration and comparison of the solution space of machine learning (ML) pipelines produced by AutoML systems. PipelineProfiler is integrated with Jupyter Notebook and can be used together with common data science tools to enable a rich set of analyses of the ML pipelines and provide insights about the algorithms that generated them. We demonstrate the utility of our tool through several use cases where PipelineProfiler is used to better understand and improve a real-world AutoML system. Furthermore, we validate our approach by presenting a detailed analysis of a think-aloud experiment with six data scientists who develop and evaluate AutoML tools.","url_abs":"https://arxiv.org/abs/2005.00160v1","url_pdf":"https://arxiv.org/pdf/2005.00160v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"pipelineprofiler-a-visual-analytics-tool-for","repo_url":"https://github.com/VIDA-NYU/PipelineVis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2005.00160","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.00160"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VIDA-NYU/PipelineVis","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":22},"by_repo_kind":{"official":{"samples":22,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cb700fb13b74cafb","entry":"comm_export_pipelines","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_plot_pipeline_matrix.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_plot_pipeline_matrix.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cb700fb13b74cafb"}},{"code_sha256_prefix":"fd412c5826f4338f","entry":"comm_merge_graphs","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_plot_pipeline_matrix.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_plot_pipeline_matrix.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fd412c5826f4338f"}},{"code_sha256_prefix":"ce3a9407ef5f2b82","entry":"comm_powerset_analysis","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_plot_pipeline_matrix.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_plot_pipeline_matrix.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"ce3a9407ef5f2b82"}},{"code_sha256_prefix":"468aa06640e32104","entry":"compute_edge_type","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_similarity_flooding.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_similarity_flooding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"468aa06640e32104"}},{"code_sha256_prefix":"093743a6ad664cab","entry":"create_pairwise_conn_graph","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_similarity_flooding.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_similarity_flooding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"093743a6ad664cab"}},{"code_sha256_prefix":"981f88dd9636a989","entry":"extract_coef_matrix","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/Experimental/FeatureWeights.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/Experimental/FeatureWeights.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"981f88dd9636a989"}},{"code_sha256_prefix":"02b0b27a25321f61","entry":"extract_primitive_matrix","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_powerset_analysis.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_powerset_analysis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"02b0b27a25321f61"}},{"code_sha256_prefix":"8f8cace5414ad119","entry":"extract_primitive_names","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_powerset_analysis.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_powerset_analysis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"8f8cace5414ad119"}},{"code_sha256_prefix":"e83e9f9bff219c2f","entry":"extract_primitive_names","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/Experimental/FeatureWeights.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/Experimental/FeatureWeights.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e83e9f9bff219c2f"}},{"code_sha256_prefix":"2bdd440663e3b6c7","entry":"extract_scores","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_powerset_analysis.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_powerset_analysis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2bdd440663e3b6c7"}},{"code_sha256_prefix":"78e46d80267263a0","entry":"find_ensemble_weights","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_import_autosklearn.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_import_autosklearn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"78e46d80267263a0"}},{"code_sha256_prefix":"af652d3f85cb2cd0","entry":"find_metric_name","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_import_autosklearn.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_import_autosklearn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"af652d3f85cb2cd0"}},{"code_sha256_prefix":"02ed464496dd4171","entry":"find_pipeline_digest","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_helpers.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"02ed464496dd4171"}},{"code_sha256_prefix":"171e8b337efff784","entry":"graph_data_accessor","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_graph_matching.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_graph_matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"171e8b337efff784"}},{"code_sha256_prefix":"9e6f0e878723c786","entry":"id_generator","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_plot_pipeline_node_link.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_plot_pipeline_node_link.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9e6f0e878723c786"}},{"code_sha256_prefix":"bbed82fca7a991d2","entry":"import_autosklearn","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_import_autosklearn.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_import_autosklearn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"bbed82fca7a991d2"}},{"code_sha256_prefix":"f03844004cdb90bc","entry":"make_html","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_plot_pipeline_node_link.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_plot_pipeline_node_link.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"f03844004cdb90bc"}},{"code_sha256_prefix":"cdf74ce85a1cf951","entry":"merge_pipeline_files","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/pipeline_merge.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/pipeline_merge.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cdf74ce85a1cf951"}},{"code_sha256_prefix":"e257df5f0fbcb877","entry":"node_type","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_similarity_flooding.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_similarity_flooding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"e257df5f0fbcb877"}},{"code_sha256_prefix":"5d81e8b2d80be745","entry":"pipeline_to_graph","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_graph_matching.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_graph_matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5d81e8b2d80be745"}},{"code_sha256_prefix":"64cc10732a5bf54b","entry":"python_path_similarity","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_graph_matching.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_graph_matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"64cc10732a5bf54b"}},{"code_sha256_prefix":"1996a25026aaf84a","entry":"setup_comm_colab","repo":"VIDA-NYU/PipelineVis","repo_kind":"official","path":"PipelineProfiler/_comm_api.py","file_url":"https://github.com/VIDA-NYU/PipelineVis/blob/HEAD/PipelineProfiler/_comm_api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"1996a25026aaf84a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}