MemVerge

What are the primitives for Generative AI? If you said NVIDIA GPUs and PyTorch you’d be mostly right in the present day, but what about 5 years from now? Andy’s 2023 shareholder letter lends some clues to what the future looks like. 1. Bottom Layer: the hardware and infrastructure level developer tools Cerebras, Sambanova, AWS’s Inferentia, Trainium, AMD’s MI300, the competition is and will continue to be fierce to gain share vs NVIDIA for cost per token/inference, inference latency, throughput and training performance. 2. Middle Layer: foundation models, orchestration and productivity tools for the development, management and automation of GenAI workflows Ray, PyTorch, Flyte, Kueue, IBM’s WatsonX, AWS’s Bedrock and SageMaker compete to make the ML and AI developer and ops teams more productive and efficient. 3. Top Layer: the application layer, think AI agents, and other end-user facing solutions like ChatGPT and others that leverage open source models like Llama 2, Mistral, and Anthropic’s Claude 3. — This layer is where I think the next Netflix, Uber, Snowflake, and Databricks will be born. The possibilities are endless. AWS released an Amazon Q last year which can help write, debug, test and implement code. Would love to hear comments and reviews on this service from the user community. We have been looking into ways to help HPC code run better on AWS (better recommendations for VMs, better workflows and scripts, better use of the availability inventory of EC2 Spot instances). There are many companies that seem to shy away from using GPUs on the public cloud due to sticker shock, but I’m not so sure that is the real problem or biggest risk. The biggest risk is not executing ROI positive GenAI projects quickly enough to keep up with the competition, regardless of industry vertical. What are you doing at each layer of the GenAI stack to innovate and keep up with competition? #AWS #HPC #AI #ML #NVIDIA #ChatGPT

https://lnkd.in/griNjyyw

In these demo sessions in the AWS booth at SC24, see how the MemVerge solution for AWS Batch and Slurm can be used to enable long running and stateful jobs to survive Spot interruptions and complete on AWS EC2 Spot instances.?Accelerate your scientific research on-time and under budget with MemVerge and AWS.

Your code is complete, your workflow is ready to run and now you’re 2x over your AWS compute budget, what happened!? There’s an almost universal period of time in which this painpoint is acutely felt by a lot of NIH researchers and bioinformatics teams. Here’s a few helpful things you can do: 1. Meet with your AWS team. They are trained to help you via a number of ways to optimize your cloud spend, including referrals to EC2 Spot Ready partners like MemVerge. 2. Check out a really awesome tool called AWS Spot Instance Advisor. There’s a number of EC2 Spot instances that have very low rates of interruption 5% or less. Set up an AWS Batch env and queue that only uses low interruption rate instances. Not sure what the impact of interruptions are when using EC2 Spot? We have a free tool for AWS Batch users called Spot Viewer…coming soon. DM me or Achyutha Harish to get early access. Thank me later. 3. Request more Spot quota. AWS by default gives most accounts very little Spot quota and it might seem like there’s not enough Spot instances to meet your needs. In reality most customers can request thousands of vCPU cores worth of Spot instance quota in 24-48 hrs or less. 4. Use Graviton based EC2. There’s an effort underway to better support the entire nf-core collection of Nextflow pipelines on Graviton. If you’d like to try Graviton and need some help, talk to me. Also ping Brendan Bouffler and James Feasey as I understand there may even be AWS credits in store for testing out workflows. 5. Optimize your storage costs with Health Omics. There’s a benchmark from earlier this year that shows up to 50% cost savings vs S3 intelligent tiering to store your genomics data on Health Omics. If you’re in a region that provides this service and need help using or moving data to Health Omics storage, talk to me. I’ll post the benchmark in the comments. #HPC #AWS #genomics #EC2

The longer your HPC job runs on EC2 Spot… Come by the AWS expo next Tuesday and Wednesday at SC24 to talk to us and cure your Spotophobia!

Is paying $100K for an enterprise license to run Nextflow worth it? Let’s look under the hood of the major platform solutions: - 99% of these vendors run using an AWS managed service backend like AWS Batch. - If using AWS, Batch is the executor used in most Nextflow workflows, and it's free under an Apache license. - Bioinformatics workflows used to be more proprietary. But with the groundswell created by the Broad and the nf-core community, there are now many good open source pipelines. So if I'm on AWS, what am I actually paying for? - GUI or an API to run & monitor workflows. - Guidance to develop better workflows. - Support for when things go wrong. - Advanced features (like SpotSurfing EC2 Spot instances) isn’t offered or will cost extra. __________ There are multiple ways to run Nextflow on AWS without the big software license price tags: - Work with an AWS partner (like MemVerge) who supports multiple Nextflow, WDL, and multiple workflow languages. - Based on gaps or areas that are important to your specific objectives, figure out what vendors might be able to help you accelerate. We partner with Sentieon, Basepair, and TileDB. - Use a platform like DNAnexus, 7Bridges (Velsera), and Seqera. Many pros and cons here. DM me and I'm happy to help you evaluate whether they may (or may not) be good for you based on what I've learned. __________ tl;dr. Don’t buy a platform just to get a GUI. Many companies will design a GUI or build an API for you for a one time fee. SpotSurfer is an example of our vendor-lite approach to running bioinformatics on AWS: - We enable you to use AWS native services like AWS Batch. - We help you run better on Spot and EC2 overall. - We are an AWS select tier services and EC2 Spot Ready partner. If you use a bioinformatics platform solution, I encourage you to share your experiences both good and bad in the comments below. #AWS #HPC #Genomics #Nextflow #WDL

How do you run HPC workloads super cost efficiently on AWS? Especially if you don't have help from an experienced cloud HPC team? This graph is a resource utilization view of a fine-detail mapping style analysis from one of our customers attending ASHG this week. What is happening: - The first few hours of the job is run on a smaller VM (saves money). - The job is also running on EC2 Spot instances (saves more money). - Two-thirds of the way in, the job is about to crash due due to "OOM". - The job auto-migrates to a larger VM to avoid running out of memory. - The job then survives a few Spot preemptions and completes This is all made possible by a game-changing checkpoint restore software technology MemVerge has developed over the course of 6+ years. We call it "SpotSurfer" and "OOM protection" for AWS. If you'd like to learn more, just DM me or sign up for a POC for Memory Machine Cloud via this link: https://lnkd.in/e8Ccye7A.. #AWS #HPC #EC2 #DevOps #FinOps

Flying to Denver to attend ASHG! If you’re interested in using AWS to accelerate your NIH funded research, come meet our team at booth # 486 to learn how we can help! You can also sign up for our Genes ?? after dark event tomorrow evening via the link below. Our EC2 Spot Ready software and finops services make it easier and more cost effective than ever! Accelerate your research and collaborate with others easily with the power of public cloud.

Stop by the CXL Pavilion to see Memverge demos of AI use cases: LLM RAG pipeline using server expansion, Stable Diffusion using CXL shared memory, Global Shared Memory Objects (GISMO), and Dynamic Capacity Device and Hotness Tracking

CUDA is better when placed closer together?? Is your ML AI workload topology aware? If not, you could be wasting a lot of time renting expensive GPUs longer than you need to be. “Many NVIDIA? graphical processing units (GPU)-accelerated HPC applications that use Message Passing Interface (MPI) spend a substantial portion of their runtime in non-uniform GPU-to-GPU communications. These expensive communications prevent users from maximizing performance from their existing hardware.” If you’re a founder, ask your AI team about this. Need help? Talk to us! Read more via the link below…if this was helpful, please like, comment and repost to your network! https://lnkd.in/eEQcmAKU