{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scidataflow-a-tool-for-improving-the-flow-of","title":"SciDataFlow: A Tool for Improving the Flow of Data through Science","arxiv_id":"2311.04904","date":"2023-09-08","proceeding":null,"authors":["Vince Buffalo"],"abstract":"Managing data and code in open scientific research is complicated by two key problems: large datasets often cannot be stored alongside code in repository platforms like GitHub, and iterative analysis can lead to unnoticed changes to data, increasing the risk that analyses are based on older versions of data. Here, I introduce SciDataFlow: a fast, concurrent command-line tool paired with a simple Data Manifest specification. SciDataFlow streamlines tracking data changes, uploading data to remote repositories, and pulling in all data necessary to reproduce a computational analysis.","url_abs":"https://arxiv.org/abs/2311.04904v1","url_pdf":"https://arxiv.org/pdf/2311.04904v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"scidataflow-a-tool-for-improving-the-flow-of","repo_url":"https://github.com/vsbuffalo/scidataflow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}