PowerAI Revolutionizes Deep Learning (Again!) with Release 4
I’m excited to share with you that IBM has just released PowerAI release 4 which includes a technology preview of the record breaking Distributed Deep Learning technology we announced earlier this week. Drawing on IBM’s deep expertise in AI, in high-performance computing and system design, we have announced breakthrough results in both accuracy and performance in Deep Learning this week.
Using a cluster of 64 IBM advanced Deep Learning Power servers with 256 GPUs, we demonstrated a new speed record for training today’s most advanced neural networks in 50 minutes, a significant improvement over the hour-long training time reported by Facebook last month. At the same time, this work also significantly boosted neural network accuracy by over 13% for networks trained on very large data sets with over 7.5 million images, improving accuracy to 33.8% from the previous best results at 29.8% published by Microsoft (https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-chilimbi.pdf).
Accelerating the training of Deep Neural Networks (DNNs) is not an idle competition but has direct impact on how DNNs can be applied to real-world problems. High-speed training enables AI developers and data scientists to develop better neural models for their applications by interactively exploring and optimizing DNN architectures. Thus, speed records are not about idle competition, but about empowering technology end users to find better solutions. Underscoring that dual commitment to advancing both the state of the art in AI and making the technology available to its users immediately, IBM is the only AI leader making the new technological advances available to all users concurrently with announcing the breakthrough by releasing them as a technology preview as part of PowerAI.
The advances were obtained by applying IBM’s deep expertise in system design and high-performance computing to deep learning with a close collaboration between the IBM Research and product divisions, combining PowerAI software, deep learning servers and excellence in high-performance computing research.
Drawing on IBM’s deep decade-long experience expertise in high-performance parallel systems, the Distributed Deep Learning framework achieves unprecedented scaling efficiency of 95%, ensuring the computing resources are efficiently used (https://www.ibm.com/blogs/research/2017/08/distributed-deep-learning/).
AI users can use the technologies that will power the world’s fastest CORAL supercomputers with the CORAL supercomputer at the US national laboratories (https://openpowerfoundation.org/press-releases/department-of-energy-awards-425-million-for-next-generation-supercomputing-technologies/) to enhance the quality and speed of deep learning applications today. Advances like these are only possible in an open standard-based environment that brings together the industry’s best technologies: IBM’s deep learning servers and CORAL technologies are created in the Open POWER ecosystem for collaborative innovation, together the IBM’s advanced Power system designs, Nvidia’s GPU numerical accelerators and Mellanox high-performance networking.
The advances were obtained with the PowerAI for transforming scientific research and businesses with AI technologies which provides a stable, compatible environment for delivering breakthrough innovations. The Distributed Deep Learning technology is available today to PowerAI users with a technology preview for Caffe and TensorFlow in PowerAI Release 4 which is available for free download at ibm.biz/powerai today.