List of accepted artifacts

• ReQuEST-ASPLOS'18
• PPoPP'18
• CGO'18
• IA3 at SuperComputing'17
• CGO'17
• PACT'17
• PPoPP'17
• ADAPT'16
• PACT'16
• PPoPP'16
• CGO'16
• PPoPP'15
• CGO'15

Paper	Artifact available	Artifact functional	Artifact reusable	Results reproduced	Results replicated
Highly Efficient 8-bit Low Precision Inference of Convolutional Neural Networks with IntelCaffe Jiong Gong, Haihao Shen, Guoming Zhang, Xiaoli Liu, Shane Li, Ge Jin, Niharika Maheshwari, Evarist Fomenko, Eden Segal [ Paper DOI ] [ Artifact DOI ] [ Original artifact ] [ CK workflow ] [ CK results ]
Optimizing Deep Learning Workloads on ARM GPU with TVM Lianmin Zheng, Tianqi Chen [ Paper DOI ] [ Artifact DOI ] [ Original artifact ] [ CK workflow ] [ CK results ]
Real-Time Image Recognition Using Collaborative IoT Devices Ramyad Hadidi, Jiashen Cao, Matthew Woodward, Michael S. Ryoo, Hyesoon Kim [ Paper DOI ] [ Artifact DOI ] [ Original artifact ] [ CK workflow ] [ CK results ]
Leveraging the VTA-TVM Hardware-Software Stack for FPGA Acceleration of 8-bit ResNet-18 Inference Thierry Moreau [ Paper DOI ] [ Artifact DOI ] [ Original artifact ] [ CK workflow ] [ CK results ]
Multi-objective autotuning of MobileNets across the full software/hardware stack Anton Lokhmotov, Nikolay Chunosov, Flavio Vella, Grigori Fursin [ Paper DOI ] [ Artifact DOI ] [ CK workflow ] [ CK results ]

Accepted artifacts for PPoPP 2018

This year we successfully validated the new ACM Artifact Review and Badging policy which we co-authored last year. However, we also noticed that "reusability/customization" criteria in the new guidelines are quite vague and caused ambiguity in evaluation of several complex artifacts. We would like to discuss this issue further with the community and develop more precise guidelines for next artifact evaluation. In the mean time, since our philosophy of artifact evaluation is that it is a cooperative process between authors and reviewers to overcome technical issues together, we helped all authors improve artifacts and pass evaluation.

Paper	Artifact available	Artifact functional	Artifact reusable	Results replicated
Optimizing N-Dimensional, Winograd-Based Convolution for Manycore CPUs Zhen Jia, Aleksandar Zlateski, Frédo Durand and Kai Li	Artifact (10.6084/m9.figshare.5873868)
PAM: Parallel Augmented Maps Yihan Sun, Daniel Ferizovic and Guy Blelloch	Artifact (10.5281/zenodo.1168703)
An Effective Fusion and Tile Size Model for Optimizing Image Processing Pipelines Abhinav Jangda and Uday Bondhugula	Artifact (10.5281/zenodo.1168539)
Cache-Tries: Concurrent Lock-Free Hash Tries with Constant-Time Operations Aleksandar Prokopec	Artifact (10.5281/zenodo.1168402)
swSpTRSV: a Fast Sparse Triangular Solve with Sparse Level Tile Layout on Sunway Architectures Xinliang Wang, Weifeng Liu, Wei Xue and Li Wu	Artifact (10.5281/zenodo.1168762)
VerifiedFT: A Verified, High-Performance Dynamic Race Detector James Wilcox, Cormac Flanagan and Stephen Freund	Artifact (10.5281/zenodo.1171046)
Juggler: A Dependency-Aware Task Based Execution Framework for GPUs Mehmet Belviranli, Seyong Lee, Jeff Vetter and Laxmi Bhuyan	Artifact (10.5281/zenodo.1168558)
Bridging the Gap between Deep Learning and Sparse Matrix Format Selection Yue Zhao, Jiajia Li, Chunhua Liao and Xipeng Shen	Artifact (10.6084/m9.figshare.5873868)
Harnessing Epoch-based Reclamation for Efficient Range Queries Maya Arbel and Trevor Brown	Artifact (10.5281/zenodo.1168726)
Register Optimizations for Stencils on GPUs Prashant Rawat, Aravind Sukumaran-Rajam, Atanas Rountev, Fabrice Rastello, Louis-Noel Pouchet and P. Sadayappan	Artifact (10.5281/zenodo.1168498)
Making Pull-Based Graph Processing Performant Samuel Grossman, Heiner Litz and Christos Kozyrakis	Artifact (10.5281/zenodo.1169388)
Interval-Based Memory Reclamation Haosen Wen, Joseph Izraelevitz, Wentao Cai, H. Alan Beadle and Michael L. Scott	Artifact (10.5281/zenodo.1168572)
Efficient Parallel Race Detection for Two-Dimensional Dags Yifan Xu, Kunal Agrawal and I-Ting Angelina Lee	Artifact (10.5281/zenodo.1169390)
Featherlight On-the-fly False-sharing Detection Milind Chabbi, Shasha Wen and Xu Liu	Artifacts: 1, 2, 3, 4, 5 (10.5281/zenodo.1168535) (10.5281/zenodo.1168520) (10.5281/zenodo.1168529) (10.5281/zenodo.1168533) (10.5281/zenodo.1168526)
Communication-Avoiding Minimum Cuts and Connected Components Pavel Kalvoda, Lukas Gianinazzi, Alessandro De Palma, Maciej Besta and Torsten Hoefler	Artifact (10.5281/zenodo.1169439)

Accepted artifacts for CGO 2018

This year we successfully validated the new ACM Artifact Review and Badging policy which we co-authored last year.

Paper	Artifact available	Artifact functional	Artifact reusable	Results replicated
Poker: Permutation-based SIMD Execution of Intensive Tree Search by Path Encoding F. Zhang, J. Xue
Optimal DNN Primitive Selection with Partitioned Boolean Quadratic Programming A. Anderson, D. Gregg
May-Happen-in-Parallel Analysis with Static Vector Clocks Q. Zhou, L. Li, L. Wang, J. Xue, X. Feng
SIMD Intrinsics on Managed Language Runtimes Alen Stojanov, Ivaylo Toskov, Tiark Rompf, and Markus Püschel
DeLICM: Scalar Memory Dependence Removal at Zero Memory Cost M. Kruse, T. Grosser
CVR: Efficient SpMV Vectorization on X86 Processors B. Xie, J. Zhan, X. Liu, Z. Jia, W. Gao, L. Zhang
Lightweight Detection of Cache Conflicts P. Roy, S. Song, S. Krishnamoorthy, X. Liu
Synthesizing Instruction Selection S. Buchwald, A. Fried, S. Hack
Qubit Allocation M. Siraichi, V. Santos, S. Collange, F. Pereira
High Performance Stencil Code Generation with LIFT B. Hagedorn, L. Stoltzfus, M. Steuwer, S. Gorlatch, C. Dubach
Conflict-free Vectorization of Associative Irregular Applications with Recent SIMD Architectural Advances P. Jiang, G. Agrawal
nAdroid: Statically Detecting Ordering Violations in Android Applications X. Fu, D. Lee, C. Jung
CUDAAdvisor: LLVM-based Runtime Profiling for Modern GPUs D. Shen, S. Song, A. Li, X. Liu
Enabling Enclave Code Secrecy via Self-Modification E. Bauman, H. Wang, M. Zhang, Z. Lin
A Compiler for Cyber-Physical Digital Microfluidic Biochips Christopher Curtis, Daniel Grissom, Philip Brisk We also encourage sharing of artifacts via ACM DL even if they were not submitted for evaluation due to lack of time, etc. However, we now collaborate with ACM to develop common formats and API for such artifacts.

Accepted artifacts for IA3 at SuperComputing 2017

[

• Optimizing Word2Vec Performance on Multicore Systems (dividiti award for portable and reusable experimental workflow implemented using Collective Knowledge Framework)
  Vasudevan Rengasamy, Tao-Yang Fu, Wang-Chien Lee and Kamesh Madduri
  [ GitHub ]
  

Accepted artifacts for CGO 2017

[

This year we successfully tried an "open reviewing model" with a few artifacts, when we asked the community to publicly evaluate several artifacts already available at GitHub, GitLab and other project hosting services. This allowed us to find external reviewers who had access to very rare HPC servers or proprietary benchmarks and tools. We also allowed authors of the accepted artifacts to add up to 2 pages of Artifact Appendix to their camera-ready paper and let readers better understand what was evaluated and how.

• Software Prefetching for Indirect Memory Accesses (dividiti award for distinguished artifact implemented using Collective Knowledge Framework)
  Sam Ainsworth and Timothy M. Jones
  Example of a reusable and customizable artifact with automatic cross-platform software installation and web-based experimental dashboard powered by CK framework: [GitHub , Paper with AE appendix and CK workflow , PDF snapshot of the interactive CK dashboard , CK concepts ]
  Example of a public evaluation: [GitHub discussion]
  
• FinePar: Irregularity-Aware Fine-Grained Workload Partitioning on Integrated Architectures
  Feng Zhang, Bo Wu, Jidong Zhai, Bingsheng He, Wenguang Chen and Feng Zhang
• TwinKernels: An Execution Model to Improve GPU Hardware Scheduling at Compile Time
  Xiang Gong, Zhongliang Chen, Amir Ziabari, Rafael Ubal and David Kaeli
• Optimistic Loop Optimization
  Johannes Doerfert, Tobias Grosser and Sebastian Hack
  Example of a public evaluation (with proprietary benchmark): [GitHub discussions]
• Pointer Disambiguation via Strict Inequalities
  Maroua Maalej, Vitor Paisante, Pedro Ramos, Laure Gonnord and Fernando Pereira
• Taming Warp Divergence
  Jayvant Anantpur and R Govindarajan
• Lift: A Functional Data-Parallel IR for High-Performance GPU Code Generation
  Michel Steuwer, Toomas Remmelg and Christophe Dubach
  Example of a public evaluation (with a bug fix): GitLab discussions
  Example of a paper with AE Appendix and a stamp: PDF
  CK workflow for this artifact: GitHub
  CK concepts: blog
• Legato: End-to-End Bounded Region Serializability Using Commodity Hardware Transactional Memory
  Aritra Sengupta, Man Cao, Michael D. Bond and Milind Kulkarni
• Clairvoyance: Look-Ahead Compile-time Scheduling
  Kim-Anh Tran, Trevor E. Carlson, Konstantinos Koukos, Magnus Själander, Vasileios Spiliopoulos, Stefanos Kaxiras and Alexandra Jimborean
• Synthesizing Benchmarks for Predictive Modeling
  Chris Cummins, Pavlos Petoumenos, Zheng Wang and Hugh Leather
• Cross-ISA Machine Emulation for Multicores
  Emilio Cota, Paolo Bonzini, Alex Bennée and Luca Carloni
• A Space- and Energy-Efficient Code Compression/Decompression Technique for Coarse-Grained Reconfigurable Architectures
  Bernhard Egger, Hochan Lee, Duseok Kang, Mansureh S. Moghaddam, Youngchul Cho, Yeonbok Lee, Sukjin Kim, Soonhoi Ha and Kiyoung Choi
• Automatic Generation of Fast BLAS3-GEMM: A Portable Compiler Approach
  Xing Su, Jingling Xue and Xiangke Liao

Accepted artifacts for PACT 2017

[

• SuperGraph-SLP Auto-Vectorization
  Vasileios Porpodas
  
• End-to-end Deep Learning of Optimization Heuristics
  Chris Cummins, Pavlos Petoumenos, Zheng Wang, and Hugh Leather
  
• RCU-HTM: Combining RCU with HTM to Implement Highly Efficient Concurrent Binary Search Trees
  Dimitrios Siakavaras, Konstantinos Nikas, Georgios Goumas, and Nectarios Koziris
  
• Graphie: Large-Scale Asynchronous Graph Traversals on Just a GPU
  Wei Han, Daniel Mawhirter, Bo Wu, and Matthew Buland
  
• MultiGraph: Efficient Graph Processing on GPUs
  Changwan Hong, Aravind Sukumaran-Rajam, Jinsung Kim, P. Sadayappan
  

Accepted artifacts for PPoPP 2017

[

This year we successfully tried an "open reviewing model" with a few artifacts, when we asked the community to publicly evaluate several artifacts already available at GitHub, GitLab and other project hosting services. This allowed us to find external reviewers who had access to very rare HPC servers or proprietary benchmarks and tools. We also allowed authors of the accepted artifacts to add up to 2 pages of Artifact Appendix to their camera-ready paper and let readers better understand what was evaluated and how.

• Understanding the GPU Microarchitecture to Achieve Bare-Metal Performance Tuning (NVIDIA distinguished artifact award)
  Xiuxia Zhang, Guangming Tan, Shuangbai Xue, Jiajia Li and Mingyu Chen
• Checking Concurrent Data Structures Under the C/C++11 Memory Model
  Peizhao Ou and Brian Demsky
• Combining SIMD and Many/Multi-core Parallelism for Finite State Machines with Enumerative Speculation
  Peng Jiang and Gagan Agrawal
• Groute: An Asynchronous Multi-GPU Programming Model for Irregular Computations
  Tal Ben-Nun, Michael Sutton, Sreepathi Pai and Keshav Pingali
• Self-Checkpoint: An In-Memory Checkpoint Method Using Less Space and its Practice on Fault-Tolerant HPL
  Xiongchao Tang, Jidong Zhai, Bowen Yu, Wenguang Chen and Weimin Zheng
  Example of a public evaluation via HPC and supercomputer mailing lists (with an acess to rare servers): [GitHub discussions]
• Provably Efficient Low Contention Non-Blocking Data Structures for Parallel Computation
  Umut Acar, Naama Ben-David and Mike Rainey
• Synchronized-by-Default Concurrency for Shared Memory Systems
  Martin Bättig and Thomas Gross
• Layout Lock: A Scalable Locking Paradigm for Concurrent Data Layout Modifications
  Nacshon Cohen, Arie Tal and Erez Petrank
• Processor-Oblivious Record and Replay
  Robert Utterback, Kunal Agrawal, I-Ting Angelina Lee and Milind Kulkarni
• Eunomia: A Scalable, Contention-Conscious HTM-Friendly B+Tree
  Xin Wang, Weihua Zhang, Zhaoguo Wang, Ziyun Wei, Haibo Chen and Wenyun Zhao
• Pagoda: Fine-Grained GPU Resource Virtualization for Narrow Tasks
  Tsung Tai Yeh, Amit Sabne, Putt Sakdhnagool, Rudolf Eigenmann and Timothy Rogers
• Sequential Consistency in Languages That Minimize Mutable Shared Heap
  Ryan Newton, Ryan Scott, Michael Vollmer and Madan Musuvathi
• Tapir: Embedding Fork-Join Parallelism into LLVM’s Intermediate Representation
  Tao B. Schardl, William S. Moses and Charles E. Leiserson
• Function Call Re-Vectorization
  Rubens Emilio Alves Moreira, Sylvain Collange and Fernando Magno Quintão Pereira

[ Event website ]

• GEMMbench: a framework for reproducible and collaborative benchmarking of matrix multiplication
  Anton Lokhmotov
  [ Open reviewing at Reddit , ArXiv , Shared experiment workflow as reusable CK artifact ]
• Integrating a large-scale testing campaign in the CK framework (highest ranked artifact)
  Andrei Lascu, Alastair F. Donaldson
  [ Open reviewing at Reddit , ArXiv , Shared experiment workflow as reusable CK artifact ]

Accepted artifacts for PACT 2016

[

• Fusion of Parallel Array Operations (highest ranked artifact)
  Mads R. B. Kristensen, James Avery, Simon Andreas Frimann Lund and Troels Blum
• Auto-tuning Spark Big Data Workloads on POWER8: Prediction-Based Dynamic SMT Threading
  Zhen Jia, Chao Xue and Jianfeng Zhan
• Integrating algorithmic parameters into benchmarking and design space exploration in dense 3D scene understanding
  Bruno Bodin, Luigi Nardi, M Zeeshan Zia, Harry Wagstaff, Govind Sreekar Shenoy, Murali Emani, John Mawer, Christos Kotselidis, Andy Nisbet, Mikel Lujan, Björn Franke, Paul Kelly and Michael O'Boyle
  [ Can be reproduced via Collective Knowledge Framework with online interactive graphs ]
• Sparso: Context-driven Optimizations of Sparse Linear Algebra
  Hongbo Rong, Jongsoo Park, Lingxiang Xiang, Todd Anderson and Mikhail Smelyanskiy
• Online Scalability Characterization of Data-Parallel Programs on Many Cores
  Younghyun Cho, Surim Oh and Bernhard Egger

Accepted artifacts for PPoPP 2016

[

• Optimistic Concurrency with OPTIK
  Rachid Guerraoui and Vasileios Trigonakis
• Drinking from Both Glasses: Combining Pessimistic and Optimistic Tracking of Cross-Thread Dependences
  Man Cao, Minjia Zhang, Aritra Sengupta and Michael Bond
• Keep Calm and React with Foresight: Strategies for Low-Latency and Energy-Efficient Elastic Data Stream Processing
  Tiziano De Matteis and Gabriele Mencagli
• Be My Guest - MCS Lock Now Welcomes Guests
  Tianzheng Wang, Milind M. Chabbi and Hideaki Kimura
• A Wait-free Queue as Fast as Fetch-and-Add (highest ranked artifact - award by dividiti)
  Chaoran Yang and John Mellor-Crummey
• Multi-Core On-The-Fly SCC Decomposition
  Vincent Bloemen, Alfons Laarman and Jaco van de Pol
• Production-guided Concurrency Debugging
  Nuno Machado, Luis Rodrigues and Brandon Lucia
• Declarative coordination of graph-based parallel programs
  Flavio Cruz, Ricardo Rocha and Seth Goldstein
• The Virtues of Conflict: Analysing Modern Concurrency
  Ganesh Narayanaswamy, Saurabh Joshi and Daniel Kroening
• DomLock: A New Multi-Granularity Locking Technique for Hierarchies
  Saurabh Kalikar and Rupesh Nasre
• AUTOGEN : Automatic Discovery of Cache-Oblivious Parallel Recursive Algorithms for Solving Dynamic Programs
  Charles Bachmeier, Rezaul Chowdhury, Pramod Ganapathi, Charles Leiserson, Armando Solar-Lezama, Yuan Tang and Jesmin Jahan Tithi
• Gunrock: A High-Performance Graph Processing Library on the GPU
  Yangzihao Wang, Andrew Davidson, Yuechao Pan, Yuduo Wu, Andy Riffel and John Owens

Accepted artifacts for CGO 2016

[

• Sparse Flow-Sensitive Pointer Analysis for Multithreaded C Programs
  Yulei Sui, Peng Di and Jingling Xue
• Trace-based Affine Reconstruction of Codes
  Gabriel Rodriguez, Jose M. Andion, Mahmut Kandemir and Juan Tourino
• Exploiting Mixed SIMD Parallelism by Reducing Data Reorganization Overhead
  Hao Zhou and Jingling Xue
• Validating Optimizations of Concurrent C/C++ Programs
  Soham Chakraborty and Viktor Vafeiadis
• Atomicity Violation Checker for Task Parallel Programs
  Adarsh Yoga and Santosh Nagarakatte
• A Basic Linear Algebra Compiler for Structured Matrices (highest ranked artifact - award by the cTuning foundation)
  Daniele G. Spampinato and Markus Puschel
• BlackBox: Lightweight Security Monitoring for COTS Binaries
  Byron Hawkins, Brian Demsky and Michael Taylor
• Exploiting Recent SIMD Architectural Advances for Irregular Applications
  Linchuan Chen, Peng Jiang and Gagan Agrawal
• Range Analysis of Pointers
  Vitor Paisante and Fernando Pereira
• Flexible On-Stack Replacement in LLVM
  Daniele Cono D'Elia and Camil Demetrescu
• Inference of Peak Density of Indirect Branches to Detect ROP Attacks
  Mateus Felipe Tymburiba Ferreira, Rubens Emilio Alves Moreira and Fernando Magno Quintao Pereira

Accepted artifacts for PPoPP 2015

[

• NUMA-aware Graph-structured Analytics
  Kaiyuan Zhang, Rong Chen and Haibo Chen
• Predicate RCU: An RCU for Scalable Concurrent Updates
  Maya Arbel and Adam Morrison
• Scalable and Efficient Implementation of 3D Unstructured Meshes Computation: A Case Study on Matrix Assembly
  Loïc Thébault, Eric Petit and Quang Dinh
• VirtCL: A Framework for OpenCL Device Abstraction and Management
  Yi-Ping You, Hen-Jung Wu, Yeh-Ning Tsai and Yen-Ting Chao
• Dynamic deadlock verification for general barrier synchronisation
  Tiago Cogumbreiro, Raymond Hu, Francisco Martins and Nobuko Yoshida
• Low-Overhead Software Transactional Memory with Progress Guarantees and Strong Semantics
  Minjia Zhang, Jipeng Huang, Man Cao and Michael Bond
• The SprayList: A Scalable Relaxed Priority Queue (highest ranked artifact - award by Nvidia)
  Justin Kopinsky, Dan Alistarh, Jerry Li and Nir Shavit
• Performance Implications of Dynamic Memory Allocators on Transactional Memory Systems
  Alexandro Baldassin, Edson Borin and Guido Araujo
• More than You Ever Wanted to Know about Synchronization
  Vincent Gramoli
• Cache-Oblivious Wavefront: Improving Parallelism of Recursive DP Algorithms without Losing Cache-efficiency
  Yuan Tang, Ronghui You, Haibin Kan, Jesmin Tithi, Pramod Ganapathi and Rezaul Chowdhury

Accepted artifacts for CGO 2015

[

• Locality-Centric Thread Scheduling for Bulk-synchronous Programming Models on CPU Architectures
  Hee-Seok Kim, Izzat El Hajj, John Stratton, Steven Lumetta and Wen-Mei Hwu
• MemorySanitizer: fast detector of uninitialized memory use in C++
  Evgeniy Stepanov and Konstantin Serebryany
• A Parallel Abstract Interpreter for JavaScript
  Kyle Dewey, Vineeth Kashyap and Ben Hardekopf
• A Graph-Based Higher-Order Intermediate Representation (highest ranked artifact - award by the cTuning foundation)
  Roland Leißa, Marcel Köster and Sebastian Hack
• Optimizing the flash-RAM energy trade-off in deeply embedded systems
  James Pallister, Kerstin Eder and Simon J. Hollis
• Scalable Conditional Induction Variable (CIV) Analysis
  Cosmin E. Oancea and Lawrence Rauchwerger