Vaibhav Page, Associate, is a software developer in the Platform Engineering team focused on the Aladdin Data Science platform. He is also the co-author and maintainer of the Open Source project Argo Events.
Unless you have been hiding under a rock, the go-to-tool for building portable applications in containers is Docker. Typically, Docker is the only container tool the majority of us have ever used. This is totally understandable, Docker is clearly considered the de facto container build and runtime tool in the industry. With its relatively low learning curve, Docker facilitates the rapid adoption of an “Infrastructure-as-Code” approach to building and maintaining application environments. Why would you consider using anything else, right? Well, as your adoption increases, you slowly begin to realize that maybe Docker isn’t all “sunshine and lollipops”. Scar tissue begins to form from the dreaded daemon issues, poor debugging options, security, permissions issues with volume mounts, and the list goes on… Does this sound familiar?
Containers themselves do not run Docker, they run on a Linux kernel. Under the hood, the Docker daemon is communicating with this kernel to enable the creation of containers. New disrupters in the container tools space like Buildah facilitate building OCI (Open Container Initiative, the standard for building container images) compliant container images. Buildah allows you to build these OCI compliant images without the need for a container runtime like the Docker daemon. As with many disrupters, Buildah is not fully mature. Given the lack of maturity, many are hesitant about migrating to a new container build tool when there is still a lot of uncertainty. If only you had the option to choose one tool, what would it be?
At BlackRock, we rely extensively on Docker in our CI pipelines. It solves many use-cases for our seasoned developers who have an in-depth knowledge of how to build, maintain, and tune container images. The real challenge comes from the high barrier to entry for our many “Citizen Developers”.
A Citizen Developer is a professional that comes from a non-traditional discipline/background or business domain that is (typically) a self-taught programmer. At BlackRock, these are portfolio managers, research analysts, quants, and operations staff, for example. Given the wide range of experience and programming backgrounds, BlackRock needed a fit-for-purpose “One BlackRock” abstraction that allowed for a low barrier to entry and a simple way to configure and tune container images for everyone while not tightly coupling to a specific technology.
Given all these motivations and the pace at which this space is moving, we decided to develop a framework called OCIBuilder that provides a variety of container build tool avenues like Docker and Buildah along with a declarative build configuration to build OCI compliant images with ease. It offers a seamless integration point with the variety of other open source projects in the container ecosystem to manage the lifecycle of the build process.
We decided to open source the OCIBuilder for several reasons. First, we want to provide a piece of software that solves real problems we faced with container image building at BlackRock to developers around the world. Second, we want to drive adoption of the platform by the broader developer community and get more diverse viewpoints to build a sustainable project.
Below are some of the features of OCIBuilder:
- Supports both Docker and buildah as the container build tool
- Multiple build definitions in the single build configuration
- Templatized build stages
- Environment-specific configuration via overlays
- Parameterize build configuration at runtime
- Lean Distroless image support (application and runtime dependencies only)
- Ansible roles provided build stages
- All typical features like accessing, pulling, and pushing images to and from multiple registries
Our architecture is forward-looking with the ocibuilder library being exposed by either the ocictl cli or through our in-development Kubernetes operator
Getting Started with OCIBuilder
But how do we use it and what does it look like? Here’s a very basic web server in go that we’re going to be building using our shiny new tool
And below is the important stuff – here is our build specification
Going through the above specification we have a single build step to build our go application. Within that step we’re taking advantage of multi-stage builds, with each stage referring to a reusable build template defined above.
We can very simply run a docker build
or a buildah build
We have more documented examples outlining some of our assumed features here.
- Integrate with Grafeas to capture build metadata
- Integrate with container diff tool
- Kubernetes native build operator
Contributions are more than welcome. If you are interested please take a look at our contributing guidelines.