VAST Data Teams with National Center for Supercomputing Applications to Accelerate AI Research and Discovery

Remote-First-Company | NEW YORK CITY & ATLANTA, GA – November 18, 2024 – VAST Data, the AI data platform company, announced today at SC24, a strategic partnership with the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana-Champaign, a leading research institution dedicated to advancing scientific research and discovery, by providing access to accelerated computing infrastructure.?

NCSA supports a wide range of organizations and use cases, and its users include Fortune 100 companies, U.S. National Science Foundation (NSF) researchers, and University of Illinois scientists. To meet the complex and evolving demands of its diverse user groups, NCSA turned to the VAST Data Platform for its reliable performance, scalable multi-tenancy capabilities, and efficient data compression - to deliver a uniform, consistent experience to every user of its Harbor home and software file system.

Over the last two years, we’ve seen a significant increase in GPU utilization from our Harbor users – and in that time, we’ve consolidated our file systems to a single VAST namespace, which has helped to ensure 100 percent uptime, improved application load times and provided a consistent, more interactive experience,” said J.D. Maloney, Lead HPC Storage Engineer at NCSA. “VAST has helped us to simplify operations while improving performance – accelerating data access, especially for container-heavy AI workloads, maximizing GPU utilization and streamlining AI research pipelines at NCSA.

With the VAST Data Platform, NCSA has seen significant improvements across its operations, including:

• Improved Performance: Users experienced faster load times for software and home directories, which is critical for interactive HPC workloads. Module loads dropped from nine seconds to just 2.5 seconds, leading to a more responsive system overall.?
• Data Efficiency: With VAST’s advanced compression and deduplication, NCSA achieved significant data reduction, allowing them to provide larger quotas with the same physical storage capacity.
• Streamlined Operations: VAST’s support for multiple fabric types (Ethernet, InfiniBand, Slingshot) and multi-tenancy capabilities allowed NCSA to consolidate its infrastructure. This reduced the number of file systems NCSA had to manage, resulting in fewer maintenance windows and more uptime for its clusters.
• Scalability for Future Growth: With the addition of GPU-optimized clusters like Delta AI, NCSA can now handle larger, more complex AI workloads as VAST ensures GPU resources are fully utilized by minimizing data loading times.

With VAST, NCSA and its users now have access to advanced AI infrastructure that can expedite scientific research across every discipline, supporting their ability to process complex datasets, train AI models, and conduct high-performance simulations across various scientific domains,” said Subramanian Kartik, Global Systems Engineering Lead at VAST Data. “We are proud to be helping these organizations leverage AI to explore new scientific theories and solve complex problems faster and more efficiently.

Additional Resources:

• VAST Data + NCSA Customer Spotlight Page
• VAST for Higher Education & Research