What is the AI Performance ?
AI needs a lot of infrastructure, but how does AI perform after that ? Why does a LLM for instance be able to reply with the most complicated questions within 10 seconds? Is the Infrastructure too much for the performance ? The AI infrastructure could be looked upon as https://www.ibm.com/topics/ai-infrastructure, The compute and storage is not the only AI Infrastructure, but the entire ecosystem. The ecosystem includes as cited from above -
As per definition of AI Infrastructure as cited from above -
"Since?AI infrastructure?is typically?cloud-based, it’s much more scalable and flexible than its?on-premises?IT predecessors. As the?datasets?needed to power?AI applications?become larger and more complex,?AI infrastructure?is designed to scale with them, empowering organizations to increase the resources on an as-needed basis. Flexible?cloud infrastructure?is highly adaptable and can be scaled up or down easily than more traditional IT infrastructure as an enterprise’s requirements change."
It would wrong to think that AI Performance means more compute, though the compute requirements are going to increase in near future, but a need for AI Infrastructure Schedulers to minimize the compute growth exponentially to linear graph.
Summary : AI Performance till now has been judged as an factor exponential growth, but scheduler which run these requests are going to decide whether it can be linear, say 1 Million user need 10 AI Infrastructure and 10 million need 100 AI Infrastructure, AI Infrastructure is simply a compute resource which might be 1 Terraflops in total or a single entity as such.