Vindicator

A while back I posted that we had received an exciting delivery in our London datacentre - our LinuxOne test system from IBM. This post was light on details because of course, our LinuxOne (named "Vindicator") was a little special - it was the one of the first (I am told actually the first but in the Christmas rush I want to be safe!) of the new IBM LinuxOne Express 4s ever shipped to a customer. The actual system was announced yesterday and so I can say a little bit more about what we got.

For readers from an HPC background, it might be a bit unclear what a LinuxOne is. We are very familiar with IBM's POWER based systems from the early 2000s when it made up a significant chunk of the HPC systems worldwide (indeed my first "Tier 1" experience was "HPCx", the UK's national supercomputer resource at the time which was a massive cluster made of IBM pSeries running AIX) and Summit, presently number 7 on the Top 500 is an IBM POWER9 system. So when I mention this system to HPC people, I am asked - is this a POWER system?

It's not.

It's much cooler than that.

The IBM LinuxOne is IBM's "cut down" SKU of the IBM Z mainframes - an architecture that stretches its linage all the way back to System/360, through System/370 and System/390 to today's hyper-modern and secure IBM Z - which is entirely designed to run Linux. You can run the same Linux (the s390x arch) on IBM Z, but you can't run z/OS on the LinuxOne. It's supported by RedHat (of course) and by Canonical, as well as other Linux distros.

We have an R&D budget and I've wanted to do a Linux on Mainframe project for a while, for a number of interconnected reasons:

1. It's hardware we knew little about. I've mentioned elsewhere that there was a time where mainframes and HPC were a lot more related but we have long diverged.
2. While we call ourselves "High Performance Computing", mainframe in enterprise is very much "High Performance Computing", just not in terms of floating point. These are the systems that process the vast majority of all credit card transactions in the world and most financial transactions - they do it with a predictable and low latency for the transactions - an incredible performance feat when you stop to think about the sheer volume of compute required - requiring very powerful I/O and a large number of accelerators to do so.
3. Not all scientific workloads are floating point - for a long time we've had a number of users who are very I/O bound and low in floating point requirements. Mainframes are good at I/O and run Linux! These codes are written in portable or interpreted languages (Java, Python, Perl) - investigating the performance of hardware designed for similar workloads in the enterprise world seems... a very obvious thing to do? I was also distantly aware that the Met Office used Linux on mainframe for something I/O related.
4. There is an increasing focus in security in the HPC world - associated with the rise of the "Trusted Research Environments" (TREs) for performing HPC/ML workloads on various kinds of sensitive data. Both IBM's midrange (IBM I/P) and mainframe (IBM Z) have, due to their heritage, hardware security features that are basically unheard of on commodity x86_64 systems and which people building TREs expend a lot of effort recreating these features in software (and sometimes hardware - we have had some very interesting discussions around things you can do with smart NICs and hypervisors) when they exist already right now in hardware on the IBM platforms. The addition of AI inference acceleration in the latest iteration of the Z architecture just makes this even more attractive.

Last year, IBM announced the rackmount version of the LinuxOne Rockhopper (our London datacentre is pre-racked - another challenge of buying a mainframe!) and we stepped up our efforts to acquire one for testing, working with our partners Logicalis to procure such a system for our R&D programme. It transpired that our timing and budget made the Express 4 a perfect match and as a result we have procured Vindicator, along with an IBM flash storage system to attach to it.

Initially, the system will be used for three projects based around the above points - investigating performance (both "raw performance" and "throughput per watt") for Next Generation Sequencing workloads, how easy it is to integrate the use of the hardware security features into our "new" Linux + Kubernetes based stack, and then some work on the AI performance of the Telum processor to see how much inference we could do in a LinuxOne TRE. After that however, we are keen to work with colleagues around UCL and in the wider university community on projects of interest to you!

This is one of the more personally exciting projects I have gotten to do since we became ARC and we look forward to working with IBM and Logicalis on this.