StackHPC develops OpenStack capabilities for research computing use cases. Through extensive experience, they understand HPC and cloud. StackHPC focuses on the needs and the shortcomings of each paradigm. They work with clients to develop solutions that address the gaps in cloud for research computing use cases.
In addition, StackHPC works actively with the OpenStack community to promote the research computing use case. Their CTO, Stig Telfer, is also the co-founder and co-chair of the OpenStack Foundation’s Scientific SIG.
During the OpenInfra Summit of 2022, the CTO of StackHPC, Stig Telfer also held some talks. One of his talks was about how in recent years they have seen increasing divergence from the traditional HPC model, and researchers who are keen to take advantage of new and rapidly developing tools such as Jupyter Notebooks, Dask, Apache Spark and Kubeflow while still maintaining the ability to run existing codes in a traditional batch environment, all without sacrificing performance. The explosion of tools and platforms, coupled with the fact that many of these platforms also need to be customised for each use-case, places a heavy burden on the operators of traditional HPC systems where individual platforms are deployed and maintained by the operator on behalf of users.
During the talk, StackHPC demonstrated how the Azimuth portal is able to reduce time-to-science and operational overhead by providing researchers with self-service access to HPC and machine learning platforms via a simple and intuitive user interface. Azimuth builds on work done at JASMIN, with funding from the IRIS collaboration, to present users with a catalogue of customisable platforms that they can deploy into their cloud allocation. Leveraging cloud-native technologies and automation, these platforms can be deployed on virtual machines or in Kubernetes clusters and are able to take advantage of hardware acceleration such as GPUs or RDMA networking without explicit configuration from the user. The Azimuth portal is in use at several IRIS sites, and is providing platforms for projects including the SKA.
Does that make you interested in the talk? You can view the talk through the following link https://youtu.be/YKdfEU3kV7w
In addition to this talk, we have also interviewed Stig during the OpenInfra summit. Some very interesting things were discussed during this interview, which you can read below.
What is the most exciting thing you heard at the summit?
Some of the very interesting things I have heard and seen has been the maturing of the hardware network acceleration. What that holds in promise is to enable to bring out all the benefits of software defined infrastructure, in particularly, software defined networking without sacrificing on the performance which these things usually bring with them. So. seeing the maturation, new developments, and capabilities of this project, which we have been working with since 2017, is great.
What do you think is the most important news for your organization?
I think it’s just nice to be here. The biggest news is to be able to have a community face to face event. It has been a long time working in hibernation in terms of working from home due to covid. Ultimately a lot of the work we do is with people. I mean, it’s with infrastructure but fundamentally it is a people business. So being able to get out and meet people again has been tremendously important. I think that is the biggest benefit so far, to be able to go to speakers and have those informal discussions with experts across the open infrastructure community. It’s really powerful, we’ve missed that in the last couple of years.
What are the most challenging things you experienced during the past 2 to 3 years?
This happened early on in lockdown. Our business is somewhat tied to the deployment of new systems. So, when there is a huge slippage in lead times for hardware, that is quite frightening. In the first couple of months, we didn’t know where the lockdown was going. However, working in scientific computing and life sciences, suddenly there was a tremendous urgency to do new things, build new systems or to repurpose the existing workloads. So, after a couple of months we found that the life sciences workloads we were working with, suddenly had this explosive need for more help and that really enabled us to weather out the shortage of hardware across the world and across the supply chains. That’s how it has been the last couple of years.
What open source development has been most important for your organization?
Good question. OpenStack, we wouldn’t exist without it. The way the OpenStack community has reacted and adopted the needs and requirements of scientific use cases and research computing are the sort of things our clients are interested in. I think in some ways it has been a reciprocating thing because it has made a great success story for OpenStack as a project. You could say research computing is one of OpenStack’s killer applications now.
What future open source developments you think are important for the future?
I think open source is really dependent on a certain amount of diversity and a broad ecosystem of contributors. It’s really important that the companies that gain benefit from open source are also the ones that contribute back to the open source community. This is something we do at StackHPC, that we are very proud of. I would love to see a great ecosystem of contributors as well as beneficiaries for open source projects. This is also how we manage to encourage people to participate. Not just by using the code but actually by improving it and giving feedback on it. Overall, I think that’s the nut that needs to be cracked, how to maximize that return of contribution back into the project.
It’s a great feeling to be able to contribute back. I think in terms of our business we get a lot of success in terms of the profile this gives us, as contributors and active members of the open source community. It’s a busines model not everyone in enterprise can relate to. It’s been a secret of our success and growth.
That was our interview with Stig Telfer. How did you like it? Do you have any questions for Stig or for Fairbanks? Let us know in the comments.
