The Containers Ecosystem, the OpenStack Magnum Project, the Open Container Initiative, and You! Megan Kostick Jeffrey Borek Daniel Krook @KostickMegan @JeffBorek @DanielKrook flickr.com/[email protected] What Open Containers and Cloud Native Computing mean to OpenStack
What you will learn today • Introduction to container technology and its open source history • How containerization fits into OpenStack, and in particular Magnum • Introduction to the Linux Foundation collaborative projects on containers – Open Container Initiative – Cloud Native Computing Foundation • How the OCI and CNCF container standardization may affect Magnum 2
Our background is in open source and open standards Megan Kostick • Customer advocate for open technologies adoption (OpenStack, Cloud Foundry, Docker) • Software Engineer, Cloud and Open Source Technologies, IBM • @KostickMegan Daniel Krook • Customer advocate for open technologies adoption (OpenStack, Cloud Foundry, Docker) • Senior Software Engineer, Cloud and Open Source Technologies, IBM • @DanielKrook Jeﬀrey Borek • IBM representative to the OCI & CNCF, Chair of Docker Governance Advisory Board • WW Program Director, Open Technologies and Partnerships, Cloud Computing • @JeﬀBorek 3
Container technology today enables greater density, faster startup, and easier deployment of applications Containers provide isolation for processes sharing compute, networking, and storage resources on a single host system. They are similar to virtualized machine instances but share the host kernel and avoid hardware emulation. Applications can be packaged with all the additional dependencies that they need, above what is provided by the host. This makes them efficient to run, easy to move from host to host, and enable more granular control of applications. There are costs, however... Diagram source: Exploring Opportunities: Containers and OpenStack 4
Containers are not new. Many innovations from many organizations have brought them where we are today FreeBSD Jails expand on Unix chroot to isolate processes 2001 Solaris Zones bring the concept of snapshots VServer Jails 2000 5 2006 cgroups Zones Linux-VServer ports kernel isolation, but requires recompilation Red Hat adds user namespaces, limiting root access in containers 2004 2008 LXC Namespaces Google introduces Process Containers, merged as cgroups Docker provides simple user tools and images. Containers go mainstream 2008 Docker IBM creates LXC, providing user tools for cgroups and namespaces 2013
Several OpenStack projects leverage containers to gain these benefits Nova A Docker hypervisor driver for Nova Compute to treat containers and images as the same type of resource as virtual machines. Kolla Containerizes the OpenStack control services themselves as microservices to simplify the operational experience. Heat A plugin template for orchestrating Docker resources on top of OpenStack resources. Allows access to full Docker API. Murano Magnum Provides an API to manage multitenant Containers-as-a-Service leveraging Heat, Nova, and Neutron. Provides an application catalog of containerized applications that can be deployed to an OpenStack cloud. OpenStack is above all an integration engine, bringing various technologies together through common APIs. Therefore, containers have naturally been plugged into several existing projects and will find their way into other areas as well. 6
The road to a Containers-as-a-Service project in OpenStack 2014 • May 2014: Containers Team Formed – Standardize the container environment – Provide consistency when deploying containers in OpenStack – Remove the risk of betting on a single container strategy • June 2015: Container Networking Subteam Formed – Incorporate a consistent networking strategy for containers • August 2015: OpenStack Silicon Valley – Event focused on containers in OpenStack 2016 7 • October 2015: OpenStack Liberty Release – First production ready release of Magnum
OpenStack Magnum provides APIs and tenant isolation for container orchestration engines • Complete management for containers within OpenStack – Orchestrates the underlying host machines with Heat – Implements multi-tenancy of separate clusters through Keystone – Provides multi-host networking with Neutron • Supports several Container Orchestration Engines (COE) – Docker Swarm – Google Kubernetes – Apache Mesos • Allows direct access to native container APIs – Docker CLI clients can access hosts and containers – The Kubernetes client can also directly manage pods, services, etc. 8
Magnum builds on several other mature OpenStack projects Magnum components Diagram source: Exploring Opportunities: Containers and OpenStack 9
Introducing the Linux Foundation Open Container Initiative (OCI) A single, open container specification: The OCI is a lightweight, open governance structure for the express purpose of creating open industry standards around container formats and runtime Announced June 22, 2015 • Not bound to higher level constructs such as a particular client or orchestration stack • Not tightly associated with any particular commercial vendor or project • Portable across a wide variety of operating systems, hardware, CPU architectures, public clouds, etc. opencontainers.org 10
The OCI aims to meld ecosystems towards an open standard • Users should be able to package their application once and have it work with any container runtime • The standard should fulfill the requirements of the most rigorous security and production environments • The standard should be vendor neutral and developed in the open 11
The OCI governs a container specification and an implementation Docker container runtime implementation: runC (formerly libcontainer) Open Container Format: OCF Spec and implementation updated in concert Community innovation driven into the spec CoreOS runtime implementation: appC (formerly Rocket) Open Container Initiative ecosystem Innovation driven into the spec github.com/opencontainers 12
Introducing the Cloud Native Computing Foundation (CNCF) • Container packaged: In order to improve the overall developer experience, foster code reuse and simplify operations The CNCF plans to create and drive the adoption of a new set of common container technologies, driven and informed by technical merit and end user value, inspired by Internet-scale computing Announced July 21, 2015 • Dynamically managed: Actively scheduled and managed by a central orchestrating process to radically improve machine efficiency • Micro-services oriented: Loosely coupled with dependencies explicitly described through service endpoints for overall agility, maintainability of applications cncf.io 13
CNCF: Supporting companies and initial high level architecture Just as the OCI targets container image portability, the CNCF targets cloud application portability… 14
OCI/CNCF standardization and the implementation of Magnum: What Magnum already brings to the table Standard container environment • Docker Swarm and Kubernetes already manage containers based on runC. • Docker Swarm will work to align its approach with the interoperability goals of the CNCF as Kubernetes has. No container strategy lock-in Adaptable infrastructure 15 • Users can wait to see the results of the foundations’ work. • Docker Swarm and Kubernetes are heavily invested in both the OCI and CNCF, and already in use in Magnum. • Magnum supports COEs, the container strategies themselves. • Supporting COEs allows for continual sync with the latest standards.
OCI/CNCF standardization and the implementation of Magnum: What Magnum is doing now Consistent networking Focus on container evolution 16 • Leveraging Docker’s libnetwork, will provide users with the same experience in and out of OpenStack. • Container networking strategies continue to evolve, and will be an area of foundation focus going forward. • The OpenStack Silicon Valley 2015 conference centered on containers, preparing for what the industry has in store. • Kuryr and other container project design goals will be discussed at sessions at the Summit.
OCI/CNCF standardization and the implementation of Magnum: What Magnum will need to focus on as standards evolve Adapt to foundation standards Contribute to foundation efforts 17 • The OCI and CNCF are continuing to finalize the charter and member agreements. • Being agnostic to container technologies, Magnum can incorporate these decisions with ease. • Past experience and expertise allows Magnum to give insight to both foundations from a production level CaaS perspective.
Summary • Container technology has evolved over the last 15 years with contributions from many organizations. It will continue to do so through the Open Container Initiative and the Cloud Native Computing Foundation. • Containerization is used throughout OpenStack, but the end user facing features provided by Magnum’s Containers-as-a-Service will be the most impacted by standards given the exposure of native APIs and Cloud Orchestration Engines. • The OpenStack Foundation provides governance over Infrastructure-as-a-Service (compute, network, and storage) APIs. The Open Container Initiative and the Cloud Native Computing Foundation will provide governance of container formats and meld orchestration engine technologies. Keep an eye on developments in both of these areas as you formulate your organization's containerization strategy. Please get involved to ensure standards reflect usage scenarios. 18
Online resources and related talks at the Summit The OpenStack Magnum wiki bit.ly/mgm-wiki Exploring Opportunities: Containers and OpenStack whitepaper bit.ly/ctrs-os The Docker and Container Ecosystem TheNewStack publication bit.ly/tns-ctrs Open Containers Initiative web site opencontainers.org Cloud Native Computing Foundation web site cncf.io The history of containers Red Hat EL blog post bit.ly/rh-ctrs Moments in container history Pivotal infographic bit.ly/pvt-ctrs Tuesday 4:40 – 5:20 Connecting the Dots with Neutron: Unifying Network Virtualization Between Containers and VMs Mohammad Banikazemi, Phil Estes Wednesday 2:00 – 2:00 Optimizing and Extending Overlay Networking for Containers Mohammad Banikazemi, Ton Ngo, Baohua Yang 4:40 – 5:20 OpenStack Magnum – Containers-as-a-Service Adrian Otto, PTL of the Magnum project Thursday 9:50 – 10:30 Exploring Magnum and Senlin Integration for Autoscaling Containers Hongbin Lu, Ton Ngo, Julio Ruano, Qiming Teng 4:30 – 5:10 19 Beginners’ Guide to Container Technology and How it Actually Works James Bottomley