Kubernetes:  A Data Management Game Changer

The only sure thing in the data integration and management world is that technology will continually change. While this is good for companies looking for a better way to manage and use the explosion of data now available, it also presents a challenge: How do you know what technology is right for your needs, and will it really move you closer to your business goals?

Every time a new tool is created, there is a lot of buzz about its capabilities, innovative approach to a problem or potential to move data integration or management to the next level. Because information technology infrastructure and business’ reliance on data represent significant investments or company revenue, it’s important to make sure that technology tools added to the network will work as described and enhance rather than disrupt operations.


Kubernetes is a relatively recent technological development that has been the topic of many conversations.
According to the founders of the Kubernetes (“koo-burr-NET-eez”) project, the name comes from a Greek word that means pilot – a good description of a tool that orchestrates the deployment, scaling and management of containerized applications.

Commonly stylized as K8s, Kubernetes was developed by Google to manage a cluster of systems deployed as containers as if it were a single system. In 2014, the container orchestration tool was released as open source software for anyone to use, modify and improve. While there are many different types of container management tools, the vibrancy and size of the K8s online community – users who share experiences with the tool and work together to solve problems – makes it a viable tool for use in many different environments. In addition to the community support for the tool, ­­the amount of time Kubernetes has been widely used means that it has been refined and well-tested.

Prior to Kubernetes, data centers relied on server virtualization to support movement of data among multiple applications. Virtual machines (VM) are good for moving information between shared services, but they do not provide some of the same benefits of containerization, which include flexibility, quicker deployment, easier scaling to reflect increased or decreased needs.

Containers contain pieces of application code and are combined into pods that include other containers that rely on common data file systems. This is an efficient way to share data between applications versus, for example, moving entire file systems from VM to VM.

Kubernetes is the tool that automates the management of these containers and pods. It determines available capacity, automatically scales the number and type of containers required to meet needs, and balances the load between users, applications and containers to maintain consistent, reliable operations.

Setting a minimum number of pods to handle specific tasks and relying on Kubernetes to increase the number of pods to handle a burst of activity means that slowdowns and backlogs are avoided. For example, a company that processes purchase orders on Monday and payments on Friday relies on different applications to pull data to handle those tasks. More pods are required to process purchase orders on Monday, so additional pods are deployed until activity returns to the predetermined “normal” level of activity. Additional pods that pull data from the same files, are deployed on Friday to handle the higher level of activity to process payments. The deployment of additional containers and pods occurs automatically, in real time so that capacity and productivity remain constant.

Automatic deployment of containers, commonly known as self-healing, also occurs if there is a pod failure. Kubernetes recognizes the failure and drop in capacity and deploys another pod to take over the task.  This self-healing capability is crucial for microservices-based architectures.

The simplest description of Kubernetes is that it is software controller. It manages pods of containers to ensure consistent capacity that can handle bursts of activity without slowing processes. K8’s open APIs allow automation driven continuous application integration, testing and deployment practices to be implemented with other community supported tool suites. When updating existing applications, Kubernetes prevents downtime and interruptions to operations by eliminating the “single light switch” approach to updates. New versions can be deployed on a pod by pod basis – also known as rolling updates— to test the software, identify any potential problems, and if necessary, roll back the update to fix the problems. Because Kubernetes is managing deployment of other pods throughout this process, the capacity of the overall application remains the same as new pods are deployed to take the place of pods that are “down.” This approach is a much safer way to update applications because the risk to the business is minimized.

Benefits and advantages

Kubernetes also provides significant benefits for development and implementation of new services or updates. It facilitates a microservices approach to building applications that requires smaller, more agile development teams. Smaller teams of specialized expert developers speeds the creation and deployment of new capabilities because developers focus on one function. To borrow a manufacturing example, the development team can focus on creating a fuel injector versus the entire space shuttle. More importantly, processes can be safely tested in single pods without disrupting overall operations.

Another advantage of Kubernetes and containers is the ability to run any application on any public cloud or any combination of public and private clouds. Because K8s makes your apps more portable and able to move between internal environments, the flexibility to migrate from cloud to cloud when appropriate also provides additional value.

While K8s makes that job of managing containers simpler by automating the function rather than relying on teams of people to monitor and react to needs for capacity, it also presents its own challenges. Despite the active support community for Kubernetes, it is an open-source technology with no formal support structure. This means that a business must have staff who thoroughly understand the technology and can keep up with the continually evolving ecosystem.

With the right staff or partner, Kubernetes is an important factor in the stability, and rapid scalability, accelerated development/deployment cycles of any integration and data management platform. Providing users with on-demand capacity that doesn’t require a slow “buffering” time to access applications and processes not only improves a business’ productivity but also provides an IT infrastructure that can flex and grow in capacity as the organization’s grow and change.

Related Articles