It’s launch day for Sylabs: Promoting portable high-performance containers for Linux

Today is launch day for Sylabs — a new company focused on promoting Singularity within the enterprise and high-performance computing (HPC) environments and on advancing the fields of artificial intelligence (AI), machine/deep learning, and advanced analytics.

And while it’s launch day for Sylabs, it’s not launch day for the technology it will be promoting. Singularity has already made great strides for HPC and has given Linux itself more prominence in HPC as it has moved more deeply into the areas of scientific and enterprise computing. With its roots at Lawrence Berkeley National Laboratory (Berkeley Lab), Singularity is already providing a platform for a lot of heavy-duty scientific research and is expected to move into many other areas, such as machine learning, and may even change the way some difficult analytical problems are approached.

What is Singularity container technology?

If you’re not familiar with Singularity as a relatively new (late 2015 introduction) container technology, you might be intrigued by some background. It was designed with HPC in mind, is already in its 13th release, and supports millions of containers every day through the Open Science Grid. It’s also open source.

While already the container platform of choice in academia and HPC centers, Singularity’s features will prove increasingly valuable in areas such as machine learning and data science — data-intensive applications that require high-performance resources. And as businesses seek to make better use of increasing data availability, they will need tools that can work with large amounts of data and containers that support and regularize its processing.

Some of the Singularity container features that have appealed to scientific research and system providers include:

• Security — Users can’t escalate their privileges and they can cryptographically sign and validate containers for tighter control (especially important in complex and research environments).
• Single file format — A Singularity container is a single file that contains a complete runtime environment. These containers are highly mobile, security-compliant, and reproducible..
• HPC support — Singularity supports high-performance hardware (e.g., GPUs, Infiniband, and Intel Phi) natively.
• Compatibility — Singularity works with Docker Hub, works with direct host IO (when applicable), and integrates with resource managers, Message Passing Interface (MPI), batch job workflows, etc.

Easy-to-use Singularity containers

Some of the things that appeal to me are the ease with which Singularity containers can be built and moved around as needed. A person can create an environment that does the work they need and then “containerize” it. I also appreciate that they’re easily reproducible and that they support trusted users in trusted environments.

As an aside, it’s encouraging to note that the name “Singularity” has nothing to do with predictions that AI will trigger runaway technological growth or deem mankind a threat and cause unfathomable harm to human civilization (as in last year’s movie by the same name).

Sylabs is investing its development efforts into the public GitHub repository for Singularity. They will also be providing a commercial Singularity Pro version for a modest subscription license. They welcome your questions at their Contact us page.

While primarily aimed at HPC, Singularity is appropriate for many different computing venues, as well, and is available right now for you to download, install, use and get involved. If interested in working with Singularity containers, check out what Sylabs is doing and get some background on Singularity.

Today’s announcement can be found here.