Energy efficient deep learning for medical imaging

Deep learning has produced some of the most accurate and most versatile techniques for many applications in medical image computing and computer-assisted intervention. However, there are very few systematic rules to guide the design and training of deep learning methods. As a result, development of these methods typically involves a lot of guesswork, trial and error, and hyper-parameter search and fine-tuning. It has become very common to train tens or hundreds of models via a near-exhaustive search of the space of parameters that may influence the model performance. Moreover, the trend has been towards using larger and larger models and training them for longer hours. Researchers and practitioners spend a lot of electric energy in the hope of gaining small improvements in model performance. This trend has reached alarming proportions due to the growing popularity of deep learning models, the increasing availability of more powerful computational hardware that naturally consume more energy, and the gloomy outlook of the human-caused global warming. Therefore, there is great incentive for designing energy-efficient deep learning methods to reduce the carbon footprint of this indispensable technology. Specifically, we are in dire need of techniques that reduce the energy requirement of deep learning methods during both training and inference.

The goal of this challenge is to encourage the MICCAI community to innovate energy-efficient deep learning methods. The proposed challenge is timely because it encourages research and development that may address several very important concerns. The current practice in developing deep learning methods, which involves huge models, long training times, and massive architecture search and hyper-parameter fine-tuning, has several critical disadvantages:

1- It wastes much energy, as explained above.
2- It demands a great amount of time from the experts who develop these methods.
3- It makes it difficult to compare and contrast different methods in a fair manner. The seeming advantages of one method over another can be merely due to the extra time and energy spent on training, architecture search, and hyper-parameter selection. This can also slow down the progress of the research community towards discovering new methods that are truly meritorious.
4- In competitions, challenges, and comparisons, it puts the teams with access to less computational resources at an unfair disadvantage and it biases the results of such competitions.
5- It limits the usability of deep learning methods for geographical areas where electric energy is less affordable, such as in many developing countries.

The proposed challenge can serve as a first step by the MICCAI community towards finding innovative solutions that can address these problems.