SIGCOMM Artifacts Evaluation Committee
A growing fraction of the papers accepted by Computer Communication Review and the SIGCOMM-sponsored conferences includes artifacts such as simulation models, measurement datasets, software implementations, etc. These artifacts are an essential part of many of these papers, and artifacts’ availability encourages other researchers to build upon and extend previous results. The ACM SIGCOMM community showed to be sensible to artifacts availability and research reproducibility, as proven by the ACM SIGCOMM Reproducibility Workshop. Also, a survey conducted among the authors of the accepted papers at SIGCOMM’17, CoNext’17, IMC’17, and ICN’17 revealed that a significant fraction of the accepted papers contained artifacts.
ACM has created a badging system to assess the quality of these artifacts. Similarly to Artifacts Evaluation Committees (AEC) created in several SIGs or conferences, the ACM SIGCOMM Artifacts Evaluation Committee evaluates the artifacts of accepted papers and assign badges to these papers.
The objective of the committee is to assign badges to the papers accepted at the SIGCOMM-sponsored conferences. ACM has defined a range of badges that can be associated to papers. The evaluation will focus on three of these badges :
Artifacts Available: this badge indicates that author-created artifacts relevant to this paper have been placed on a publically accessible archival repository.
Artifacts Evaluated – Functional: this badge indicates that the artifacts associated with the research are found to be documented, consistent, complete, exercisable, and include appropriate evidence of verification and validation.
Artifacts Evaluated – Reusable: this badge indicates that the artifacts associated with the paper are of a quality that significantly exceeds minimal functionality.
SIGCOMM thanks Haroon Rashid, Omid Ardakanian, Anna Brunstrom, Lianjie Cao, Kenjiro Cho, Quentin De Coninck, Benoit Donnet, Paul Emmerich, Christian Esteve Rothenberg, Ronaldo Alves Ferreira, Oliver Gasser, Yossi Gilad, Thomas Given-Wilson, Mohammed Hawari, Ralph Holz, Luigi Iannone, Costas Iordanou, Praveen Kumar, Sebastian Lehnhoff, Simon Leinen, Leonardo Linguaglossa, Jed Liu, Nicole Ludwig, Andra Lutu, Francois Michel, Damien Saucez, Quirin Scheitle, Joel Sommers, Holger Trittenbach, Adrien Wion, Noa Zilberman, Thomas Zinner.