Paper decision: | January 4, 2019 |
Artifact submission: | January 18, 2019 |
Decision announced: | February 22, 2019 |
Camera-ready paper: | TBA |
Conference: | March 31 - April 2, 2019 |
Public discussion: | TBA |
SysML'19 AE Chairs
Gennady Pekhimenko (University of Toronto)
Grigori Fursin (cTuning foundation / dividiti)
SysML'19 AE Committee
TBA
Artifact Evaluation for SysML 2019
SysML promotes reproducibility of experimental results and encourages code and data sharing to help the community quickly validate and compare alternative approaches. Authors of accepted SysML'19 papers are invited to formally describe supporting material (code, data, models, workflows, results) using the standard Artifact Appendix template and submit it to the Artifact Evaluation process (AE). Note that this submission is voluntary, will not influence the final decision regarding the papers, and promotes artifact sharing and reproducible research! For example, ACM Digital Library already allows one to find papers with available artifacts and reproducible results!
AE is run by a separate committee whose task is to assess how submitted artifacts support the work described in accepted papers based on the standard ACM Artifact Review and Badging policy. However, since it may be very time consuming to perform full validation of AI/ML experiments while requiring expensive computational resources, we decided to only validate if submitted artifacts are "available" and "functional" at SysML'19. Thus, depending on evaluation results, camera-ready papers will include artifact appendix and will receive at most two ACM stamps of approval printed on their first page:
You need to prepare your artifacts and appendix using the following guidelines. You can then submit your paper with artifact appendix via dedicated SysML AE website before January 18, 2019. Your submission will be then reviewed according to the following guidelines. Please, do not forget to provide a list of hardware, software, benchmark and data set dependencies in your artifact abstract - this is essential to find appropriate evaluators! At the end of the process we will inform you about how to add badges to your camera-ready paper.
If you plan to use Collective Knowledge framework to automate your workflow and require some assistance, please get in touch with the cTuning foundation and we will register you on the CK slack channel.
If you have questions, please check AE FAQs,
discuss them via the dedicated AE google group
or get in touch with SysML AE chairs!