34% of IT Decision Makers reported they are concerned with adopting containers due to a lack of full visibility.1
Containers are best known for their role in simplifying application development, providing a disposable, reusable unit to modularize delivery, and bring consistency to
virtually every development stage. They have demonstrated an ability to move DevOps forward by transforming the way development and infrastructure teams operate, and they have helped these teams move ever closer to continuous delivery. However, managing containers presents an entirely new challenge for most organizations. Containers, by their very nature, rely on shared resources. These may range from operating systems and application files to hosting resources including memory and CPU. When left unchecked, container use can lead to sprawl and may result in resource drain. With hooks into so many different areas, there is a strong incentive to know precisely what these containers are doing, what resources they are consuming, and how they are utilizing the network.
WHY VISIBILITY IS CRITICAL
Containers are small, agile and easy to spin up. As a result, they tend to spread quickly, which can ultimately lead to sprawl. Sprawl exists when containers propagate unfettered, consuming resources and negatively impacting other containers or applications. Since containers work on the concept of shared resources, even small changes can have a widespread impact. By breaking deployments into smaller, more manageable parts, we ultimately increase the number and complexity of items to control. Maintaining a manageable number of containers will prevent mishandling, misconfiguration, and resource fragmentation. Visibility helps to keep container use in check, yet it’s often overlooked in the early stages of container adoption. If your current asset management teams are struggling to keep up with the state of physical machines, patches, and apps, the widespread proliferation of containers will only add to the challenge. Furthermore, unmanaged container use may become a security issue, as it can potentially increase the overall attack surface and make systems more vulnerable to threats.
Since containers are highly mobile and flexible assets, there must be a way to see precisely where the containers are, what exactly they are doing, and on what data they are operating. There is a need for sufficient controls to apply policies and constraints to each container as they spin up, move around, and shut down. And it has become increasingly important to be able to control data movements within virtual environments, including where it can go and what resources it can consume, all while ensuring encryption and compliance controls remain in place.
Yet, with the proper insight into container use, the data provided can help with provisioning, orchestration, version control, and infrastructure management, and becomes an essential part of an ongoing containerization strategy.
SPREADING LIKE WILDFIRE
Container adoption is spreading fast. Docker, a leader in container technology, has documented the rapid growth in the use of its product. “Docker adopters approximately quintuple the average number of running containers they have in production between their first and tenth month of usage.”2 Additionally, based on a sample of 10,000 companies and 185 million containers in real-world use, Docker reports that the average user runs approximately seven bare metal containerized servers simultaneously on each host. This indicates that companies are finding tools like Docker as a lightweight way to share compute resources, not just for providing a versioned runtime environment. In fact, 25% of companies run an average of 14+ containers simultaneously, and 15 percent of hosts monitored are running Docker.3
The following visual further demonstrates just how rapidly Docker is growing in the enterprise:
As these instances of virtualization steadily increase, additional visibility is necessary to gain insight to better manage and control sprawl.
SOLVING THE VISIBILITY CHALLENGE
Containers allow IT to run services in isolation, with high fault tolerances, and with better management over resource provisioning. Containers have an intentionally short lifetime,
averaging about 2.5 days while traditional and cloud-based VMs have an average lifespan of 23 days.3 Because of their short lifetimes and hardware-agnostic behavior, containers
pose new challenges to IT requiring the need for additional infrastructure monitoring.4 The lightweight nature of containers allows them to spin up so frequently and change so rapidly that they become much more difficult to monitor and understand than a physical or even a virtual host.
Since containers live in the loosely monitored space between hosts and applications, traditional application performance and infrastructure monitoring tools lack the visibility needed to keep containers under control. Gaining insight into container use requires a unique set of monitoring tools not typically available to most technology departments. These specific tools are necessary because container resource management applies all the way through the stack, from physical infrastructure to shared files and operating systems — well beyond the reach of common resource monitoring tools. Since containers share resources, and can even limit access to other applications, there is an
increased urgency to understand the exact resources they are consuming. Visibility is not only crucial in monitoring the utilization of physical resources like memory and CPU,
but also extends to application, file, and operating system resources.
HOW VISIBILITY TOOLS WORK
Most tools incorporate a highly visible ‘dashboard’ approach, providing dynamic views into the status and performance of resources, in real-time. These tools come in many different shapes, sizes, and purposes, but most collect traditional metrics such as the utilization of CPU, memory, I/O, and network metrics. However, they also take things a step further by breaking down network traffic by image and container. Thus, in the case of a failure, engineers can immediately see which service is overloaded or causing other services to fail — and they can aggregate these service metrics across any number of hosts. Below are some key insights and capabilities that
visibility tools may provide:
- Monitoring and examining container data to enable IT
operations analytics - Overseeing container performance to ensure containers
are available and issues are fixed quickly, with minimal
effort - Delivering insight on container resource usage, cluster
capacity, and the impact of increasing cluster use for a
specific service - Discovering running containers, services, and nodes
- Adapting instantly to container scaling and updates
- Logging container connections, workloads, and violations
- Gaining better service context and accelerating root-cause
analysis by indexing, searching, and correlating containerbased
data with data from the entire technology stack
WHERE TO BEGIN?
The use of containers is on the rise. Maintaining visibility into container usage and growth will bring adopters closer to their benefits by actively seeking to avoid problems
before they happen. So, what exactly can a company that is new to the use of containers do to ensure proper visibility before diving deep into their use? Here are some suggestions.
- Understand your security requirements
- Document your compliance requirements
- Recognize your host resource requirements
- Map out precisely what your container lifecycle looks
like (or will look like)
From here, selecting the right monitoring tool, or a combination of multiple tools, will help you gain the right level of visibility into your environment as it grows. Unfortunately, there is no one-size-fits-all solution.
Currently, numerous vendors are working on visibility tools, each offering a slightly different feature set. Some tools are resource-focused, some security-focused and some are compliance-focused. Some organizations may need more than one tool to gain the right mix of visibility measures. By determining your organizational needs first, you will better understand what types of insight you need to monitor your containerized environment and achieve the visibility you need.
EMBRACING CONTAINERS
To conclude, it is time to embrace the container. When properly managed, they can increase the speed and ease of application deployment, making it easier to port application stacks across different types of environments. The benefits of working with containers are many, and by transitioning to their use, IT can see value like increased continuous delivery, reduced operational costs, and consistency in code and application deployment. However, it’s important to be vigilant in monitoring container use to avoid sprawl and conflicts when accessing shared resources.
With the right mix of visibility tools, you can best understand container behaviors, prevent resource drain, embrace the benefits of virtualization, all while maintaining a more secure environment. Still have questions? Click the button to contact us or check out our comprehensive web page that covers all aspects of containers.
Sources:
1. IDG Research commissioned by WEI, November 2017.
2. https://www.upwork.com/hiring/for-clients/docker-benefits/
3. https://www.datadoghq.com/docker-adoption/
4. https://www.datadoghq.com/blog/the-docker-monitoring-problem/
