Another Year in Supercomputing (2024 edition)

The end of this year marks my seventeenth year working in high performance computing and my ninth at the University of Melbourne in this role. When I compare this to previous years there have been some notable changes in the technology and the system I am primarily involved with (Spartan), but also in my own employment activities. Late last year, there was a structural review of our operations at Research Computing Services, as the existing organisational chart was becoming unwieldy and increasingly untenable. I ended up as the team leader for HPC Services and have stepped back somewhat from technical to management of a small but awesome team, along with organisational activities between other service groups (data, cloud) and our very close relationship with the infrastructure group.

Compared to last year (https://levlafayette.com/node/781), Spartan has increased to 7121 accounts and 2361 projects, mainly in engineering, bioinformatics (especially health) economics, mathematics, and more, and has been cited in at least 55 new papers. Machine learning has been a particularly popular area of interest for several years now on the system, which has especially benefited from Spartan's significant investment in GPUs, whose excellent vector computational performance is evident in the system receiving certification as a global supercomputer in November last year, jumping from a position of 453 (for the GPU partitions alone) in November 2023 to 262 in November 2024 (https://top500.org/system/180232/). Directly related to Spartan work, I attended two major conferences in person this year, "Supercomputing Asia" and "eResearch Australasia". For the former, I gave a presentation on the International HPC Certification Forum (https://levlafayette.com/files/2024SCAsia.pdf) and a poster on usage outcomes from training (https://levlafayette.com/files/2024SCAsiaPoster.pdf). For the latter, I gave a presentation on the development of Spartan (https://levlafayette.com/files/2024EResearchAU.pdf) from a small but innovative system to its current supercomputer status.

Training various postgraduate and postdoctoral researchers on how to use the system has been part of my work for more than a decade now, and it took some acceptance on my part several years ago when I realised that I was the most prolific supercomputer educator in the country. This year, several hundred researchers attended the twenty-two workshops that I conducted on Linux knowledge, regular expressions, HPC job submission, high performance and parallel Python, parallel programming (MPI, OpenMP, CUDA), mathematical and statistical programming (R, Octave/MATLAB, etc.), and more. In addition, each year, I am brought in for lectures and assessments for the University's Cluster and Cloud Computing course (https://handbook.unimelb.edu.au/subjects/comp90024/), which also has several hundred students. In addition, this year, I took a leave from the University to travel to the Australian Institute for Marine Science (https://tcpip.dreamwidth.org/376511.html) in Townsville to run a week-long HPC training course for around fifty of the most switched-on (mostly) young researchers I have ever had the pleasure of meeting.

All of this has resulted in an extremely good review by my manager, who really appreciated the initiatives that I have taken within the new structure. These activities will continue, as I am increasingly emphasising the importance of organisational and technical quality assurance to RCS as a whole. A good portion of next year is already organised: I know I will be attending eResearch New Zealand (https://eresearchnz.co.nz/) to deliver a paper on HPC Training for Bioinformatics, eResearch Australasia (https://conference.eresearch.edu.au/) in Brisbane, and I'll be doing lectures for UniMelb's COMP90024 course. In addition, I'll be doing my best to reduce the number of Spartan workshops I run, in preference to more online videos and documentation (we do a lot of the latter already, but it's never enough to satiate demand).

In many ways, I am deeply blessed to have the sort of job that I do. Even if I get a bit grumpy about bureaucracy at times, I love my work. I get to provide supercomputer support to researchers whose discoveries and inventions make real changes to the world we live in with a stunning return on investment of 7:1 over two years (https://www.hpcuserforum.com/ROI/), nearly entirely in the form of positive social externalities. It is computing for medicine, for climatology, for materials, for agriculture, for the environment, rather than computing for social media or games (both of which I use, by the way). It is the sort of computing I was inspired by as a youngster, research-focused and close to the metal. I may add that it is good and secure employment, especially given that technical and knowledge skills are increasingly valuable where the ratio of capital to labour increases. In a nutshell, I am more than happy with how supercomputing is progressing, and I am very happy with this career choice.

https://levlafayette.com/node/789

Image: Gryphon Gallery, University of Melbourne, the launch of the new Spartan high performance computing system 2016. Yours truly emphatically describes the system.