{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/designing-replicable-networking-experiments","title":"Designing Replicable Networking Experiments with TriScale","arxiv_id":null,"date":"2021-08-02","proceeding":"JSYS Aug_Papers 2021 1","authors":["Anonymous"],"abstract":"When designing their performance evaluations, networking researchers often encounter questions such as: How long should a run be? How many runs to perform? How to account for the variability across multiple runs? What statistical methods should be used to analyze the data? Despite their best intentions, researchers often answer these questions differently, thus impairing the replicability of their  evaluations and the confidence in their results.  \n\nIn this paper, we propose a concrete methodology for the design and analysis of performance evaluations. Our approach  hierarchically partitions the performance evaluation into three timescales, following the principle of separation of concerns. The idea is to understand, for each timescale, the temporal characteristics of variability sources, and then to apply rigorous statistical methods to derive  performance results with quantifiable confidence in spite of the inherent variability. We implement this methodology in a software  framework called TriScale. For each performance metric, TriScale computes a variability score that estimates, with a given confidence, how similar the results would be if the evaluation were replicated; in other words, TriScale quantifies the replicability of evaluations. We showcase the practicality and usefulness of TriScale on four different case studies demonstrating that TriScale helps to generalize and strengthen published results. \n\nImproving the standards of replicability in networking is a complex challenge. This paper is an important  contribution to this endeavor; it provides networking researchers with a rational and concrete experimental methodology rooted in sound  statistical foundations. The first of its kind.","url_abs":"https://openreview.net/forum?id=c1LNi8CTPy6","url_pdf":"https://openreview.net/pdf?id=c1LNi8CTPy6","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"designing-replicable-networking-experiments","repo_url":"https://github.com/romain-jacob/triscale","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}