Singularity: HPC and containers can now be friends!
Containers and "push button HPC" is all the latest rage, but there is a common difficulty in mixing the two. Typically containers (along side virtual machines) are used to maximize hardware resources by running many distinct operating system images on a single system. Technologies such as KVM, VMWare, Docker, LXC, OpenVC, etc.. all do this very well, so well in fact that people use them for other roles... For example, application and environment portability. While this is a reasonable use-case for many infrastructures, this poses various complexities within a multi-tenant HPC resource (e.g. privilege escalation on network segments containing shared file systems, infrastructure to host the daemon processes owned by root, network segments to mitigate security risks, etc..).
To this end, there is a new project I've been working on which crosses the benefits of Containers (like Docker/LXC) with packaging (such as RPM) to create a lightweight, application specific container solution. Singularity focuses on application portability and compatibility with common workflows and system architectures (such as batch schedulers and high performance file systems) without any need to change your system architecture or risk security holes.
Singularity allows one to run an application that exists in a container just like running a native application!
While it is still in development, things are moving fast! You can get more information as well as watch a demo video on the website: