CNNGEN: A GENERATOR AND BENCHMARK FOR SUSTAINABLE CONVOLUTIONAL NEURAL NETWORK SEARCH

Neural Architecture Search (NAS) emerged as a promising approach to search for
optimal neural network architectures in a limited, predefined architecture space.
One popular method to form such a space is to derive a known architecture in
which we insert cells where NAS algorithms can automatically combine network
functions and connections. Cell-based methods yielded hundreds of thousands
of trained architectures whose specifications and performance are available to de-
sign performance prediction models. Cell-based approaches come with three main
limitations: i) generated networks have limited diversity resulting in very sim-
ilar performances, in turn hampering the generalization of trained performance
models, ii) networks’ implementations are missing hampering performance un-
derstanding, and iii) they solely focus on performance metrics (e.g., accuracy)
ignoring the growing sustainability concern. We propose CNNGen, an approach
that addresses: i) by leveraging a domain-specific language (DSL) to automat-
ically generate convolutional neural networks (CNNs) without predefined cells
or base skeleton. It allows the exploration of diverse and potentially unknown
topologies; ii) CNNGen’s comprehensive pipeline stores the network description
(textual and image representation) and the fully executable generated Python code
(integrated with popular deep-learning frameworks) for analysis or retraining, and
iii) in addition to training and performance metrics, CNNGen also computes en-
ergy consumption and carbon impact for green machine learning endeavors. We
demonstrate the possibilities of CNNGen by designing two performance predic-
tors and comparing them to the state of the art.