Recently, there has been a notable increase in energy consumption associated with selecting, training, and deploying deep learning models. 

This research aims to streamline the creation of energy-efficient deep learning models that demand fewer computational resources, focusing on environmental sustainability. By concentrating on energy consumption, the research endeavours to design models that prioritize efficiency. Neural architecture search (NAS) finds support in tabular benchmarks, enabling cost-effective evaluation of NAS strategies via pre-calculated performance statistics.

Convolutional neural network AI models

The researchers conducted a study that estimated the energy required to train over 400,000 convolutional neural network AI models without training each model. Convolutional neural networks are utilized for several purposes, including the analysis of medical images, language translation, and object and face recognition, which you may be familiar with from the camera application on your smartphone.

AI models

According to the calculations, the researchers have developed a set of AI models that require less energy to do a specific task while achieving the same degree of performance. The study demonstrates that 70-80% energy savings can be attained during the training and deployment phase by selecting alternative models or adjusting existing models while experiencing a negligible performance drop of 1% or less. According to the researchers, this estimate is considered cautious and likely lower than the actual value.

The researchers emphasize that model accuracy is crucial for ensuring safety in certain domains, such as autonomous vehicles or specific branches of healthcare. Performance must be prioritized, and no concessions must be made. Nevertheless, this should encourage us to pursue high levels of energy efficiency in other areas.

Recipe book

Compiling the "recipe book" in this study is accessible as an open-source dataset for other researchers to utilize in their experiments. The data regarding over 400,000 architectures is available on GitHub and may be accessed by AI practitioners through uncomplicated Python programs. The researchers from UCPH conducted an estimation of the energy required to train 429,000 convolutional neural networks of the AI subtype models using this dataset. These technologies are utilized for object detection, language translation, and medical image analysis.

Evaluation

The study found that training the 429,000 neural networks would necessitate around 263,000 kWh. It is equivalent to the energy consumption of an average Danish citizen for 46 years. One computer would require around 100 years to complete the training. The authors of this study did not directly train these models but instead estimated them using another AI model, resulting in a 99% reduction in energy use.

Conclusion

Training AI models requires significant energy, resulting in substantial CO2e emissions. This is due to the complex calculations carried out during the training process, usually executed on high-performance computers.

Furthermore, it applies to expansive models, such as the language model powering ChatGPT. AI tasks are typically executed in data centres, requiring substantial power to sustain computer operations and maintain optimal temperatures. The carbon footprint of these centres is influenced by the energy source they use, which may depend on fossil fuels.

Researchers information

• Pedram Bakhtiarifard, Department of Computer Science, University of Copenhagen, Denmark
• Christian Igel, Department of Computer Science, University of Copenhagen, Denmark
• Raghavendra Selvan, Department of Computer Science, University of Copenhagen, Denmark

Source: https://ieeexplore.ieee.org/document/10448303/authors#authors

Image source: Unsplash