(no subject)
I need to just copy paste this stuff in here before I "accidentally" delete it out of pure frustration.
Container-based OS Virtualization
Author: Liz van Dijk
Category: IT
Product: Container-based OS Virtualization
Thumb:
Summary:
Introduction
When reading about virtualization (or writing about it, for that matter), it’s often easy to get lost in the sea of possible meanings given to the word by various sources. We at Anandtech IT have been trying to provide a clearer view into the different kinds of virtualization and in this article, we'll introduce another one. Previous articles included overviews of both hypervisor-based virtualization and application virtualization. Following that trend, we now present you with container-based OS virtualization, providing you an alternative solution: one quite regularly overlooked, despite its merits. This article is meant to give our readers an idea of what this technology offers, the situations it is applicable in and of course, puts it to the test and provides you with results.
So how does it work?
If you've been keeping up with our IT articles, you've undoubtedly come across Johan’s overview of hypervisor-based virtualization. We'll take a similar approach to its container-based counterpart, giving you a strong introduction into what makes it work and guiding you through the strengths and weaknesses of the technology.
To get started, a clear view of what the system "looks" like is needed. Unlike hypervisors and applications virtualization solutions, the isolation happens inside the OS layer.
This means there is no need to virtualize the access to different hardware components as the OS still has control over all of these. Instead, several environments are running on the same kernel, each with their own processes, libraries, root and users. On one hand, this means that the containers are limited to using the same OS kernel, but on the other it allows for a higher density. Providing abstraction from the kernel also allows environments to run different distributions concurrently on the same machine and kernel.
While there are numerous software packages offering this type of virtualization, we decided to focus on a pair of products that is closely tied together, while still allowing us to go in-depth on its inner workings: OpenVZ and Parallels Virtuozzo. OpenVZ is an open source initiative, backed by Parallels, and can be seen as a testing platform for future developments of its proprietary product: Virtuozzo. Technically, both products are used for the same purpose, but are aimed at different users in the same market. Virtuozzo is a robust product, aimed at corporate customers and comes with a very large feature set, like support for both Windows and Linux platforms. At this point, it is one of the most advanced products in the Container-based OS virtualization market. On the other hand, OpenVZ is freely available, although only for use on Linux-systems, and while still providing the basic functionality one would expect from virtualization, it is probably best-suited for smaller scale environments where Linux is the primary OS in use. Since it is open source, however, OpenVZ allows us to dig deep into its inner workings. Because OpenVZ is in a sense the testing ground for Virtuozzo, we believe gaining insight into the former will provide our readers a solid base to better understand the latter, and even other similar products.
From an administrator's point of view, OpenVZ employs a system similar to Xen's paravirtualization: one of the environments is used as the base for management tools and monitoring. This is what we'll be referring to as the host environment, and while a workload could possibly be added to it, most people would leave this container as-is, to ensure functionality of the management system at times of peak load. From the host environment, we're able to see a complete overview of the resources consumed by the different guests, and run tools like vzctl to make live changes to them.
Moreover, isolation of the containers runs quite deep, ranging from the filesystem, to their own root users and user groups, process trees and inter-process communication objects. This allows a container's root user high levels of customization and tweaking of everything apart from the kernel itself.
Handling of different distributions is taken care of by so-called templates. These are the blueprints for every environment installed on top of the OpenVZ kernel. In reality, they are no more than that distribution's files rounded up in a GZIP-compressed TAR, which will be set up in an isolated part of the filesystem for the container to run in.
So, before continuing on, let's have another quick look at what separates this virtualization technology from the others:
*Unlike with hypervisors, isolation happens from within the kernel, no layers are added to the standard model, only modified.
*Allows for a very high density (tens of environments on a single machine), as long as all environments are able to run on the same kernel.
*A single kernel and centralized management tools allow for comfortable global administration of all environments.
As it stands, despite its obvious merits, we believe container-based OS virtualization is not as well-known and widespread as it could be, so let's have a closer look at the situations this technology is applied in today, and in what fields it can really shine. Keep in mind, in this next part we won't be focusing on OpenVZ specifically, but rather give a general overview of the possibilities offered by containers. Uses of Container-based OS Virtualization
So, when would we generally suggest containers to be a viable solution? Looking at the situations in which the technology is most often applied, we can see the importance of density coming up. Many hosting companies are making use of it in order to provide their customers with a heavily customizable environment in which to run their web servers. Hardware platforms running over 100 environments are no exception here. We need to keep in mind that the total memory and CPU footprint of a container is little more than that of the sum of its processes, so density scales inversely proportional to the “weight” of all containers bundled together. When high density is a priority for whatever reason, containers can provide a solid solution as opposed to a hypervisor-based one, which despite its high flexibility is still quite limited in terms of scalability. Another example where this can come into play is plain service isolation, which could be required for perhaps a DNS.
On the other hand, containers can be very interesting when performance is prioritized. Naturally, as the workload per container is increased, the maximum amount of containers on a single hardware system is decreased, but even then they provide a solid solution. Since all containers' processes have direct access to the kernel, native I/O subsystems remain intact, allowing the full power of the hardware platform to be used, which gives the technology another edge over its "rival". Another fun fact here is that in Virtuozzo (OpenVZ's "bigger brother"), applications can actually share their libraries over several containers. What this means is that the application is loaded into the memory only once, while still allowing different users to use it concurrently, albeit using a small amount of extra memory for their respective data.
Another situation where containers can come in handy is one where many user space environments need to be rolled out in a very short time span. What we're thinking of here are mostly classrooms and hands-on sessions where quick and simple deployment is of the essence. Demos of certain pieces of software often require careful planning and a controlled environment, so applications like VirtualPC or VMWare Server/Workstation are used with "temporary" full-sized virtual machines to make sure everybody is able to run them correctly. Since the creation of a container takes about as long as the actual decompression of its template environment and the startup of its processes, a single server can easily be used for participants or students to log into through SSH or even VNC, giving each user an easily administrated environment, while still allowing them to experiment with high-level permissions.
In this case, it's also positive that everything happens on a single system that is completely accessible to the root user of the "host" container. In this sense, containers reduce both server sprawl and OS sprawl, allowing for high manageability. For example, using a single kernel reduces the amount of time spent patching every individual virtual machine separately. "Broken" containers are easily removed and recreated on the fly, as making use of templates removes the need to go through the steps of installing a fresh OS every time a new environment is needed.
Thinking further along this line brings us to the next point of interest: using containers as a base for VDI purposes. Combining all of the above factors, it is easy to see the technology's merits as a means to bundle desktop environments onto a central server. (needsmoreblahblah)
The big trade-off
There are undeniably a few downsides to this way of approaching virtualization. The first that comes to mind is that when density on a single hardware platform increases, so do its needs for constant availability. Using a single kernel in this case raises the concern on whether the kernel becomes an important single point of failure in the system. Unlike hypervisors, which are in a sense very light-weight and therefore very stable kernels, container-based OS virtualization depends on a slightly modified version of already existing kernels, thus essentially taking over existing weaknesses of both the Windows and Linux platforms.
It has to be noted that a lot of work is put into ensuring kernel stability on all container-based solutions, and unexpected kernel crashes are at least to be deemed "unlikely". Nevertheless, a bit of healthy paranoia when IT is involved is certainly not an obsolete luxury.
Secondly, while a strong feature set is definitely maintained, containers do take away a lot of flexibility from the hypervisor-based solutions. For one, many might find the inability to use different OS's next to each other to be a big handicap. Moreover, while support for live migration does exist on linux systems (both OpenVZ and Virtuozzo have implemented this), it is still missing on Windows implementations.
(Need to find some more stuff to put in here)
Sadly enough, despite all its possibilities, Container-based OS virtualization is most limited, it seems, by its plain invisibility in the current market place. For some reason, it has been quietly pushed into the background by all the current hype surrounding different virtualization technologies, and major software companies fighting for domination of this "gold mine". While it is currently used by most, if not all hosting companies, this type of virtualization seems unable to expand its customer base beyond that. Whether through lack of marketing power or a seemingly less attractive feature list, most businesses seem unaware of its potential. Naturally, it goes without saying that just like hypervisors aren't the perfect "one size fits all" solution, neither are containers, but it is still rather peculiar how a solution as polyvalent could escape the recent surge of interest into virtualization and all its aspects.
(?)
A closer look into OpenVZ's inner workings
Now that we've given a clear overview of containers in general, it is time to move on to the more technical stuff, namely figuring out how they do in fact work. As stated before, most of our research into the subject has gone into OpenVZ, but we feel the actual subjects tackled here do a great job illustrating the challenges faced by any developer of container-based solutions.
To gain a full understanding of how the partitioning of an operating system takes place, it is rather important to grasp the basics of how an OS operates. Since entire books have been written on that subject alone, and it is mostly outside the scope of this article, I'll direct those new to the subject to page 2 of Johan's article on hypervisors and their inner workings.
For now, it is important to know that in our modern day x86 architecture, there are two modes a process can run in, namely a privileged kernel mode and a "secure" user mode. The CPU is able to distinguish between these two modes by each process’ assigned memory addresses, where a 2-bit code makes the difference between kernel space addresses and user space addresses. The reason they are called addresses is because each of them points to a specific piece of memory. These pieces are generally called "pages", they're the smallest unit of memory allocation an operating system can use, and in x86 systems are usually blocks of 4kB each.
The above description introduces two important factors.
Since containers all share the same kernel, kernel mode processes are not isolated, but run outside the environments themselves. Therefore, containers are to be used only for isolation of applications running in the user space.
We raise the issue of memory management. Evidently, a single OS has an elaborate system in place for this, but how is it approached in a partitioned OS? In short, how do we prevent one container from using up all resources needed to support another?
OpenVZ has implemented a system called beancounters for this purpose. In the following pages, we will have a look at how this and other methods are combined to achieve isolation and efficient resource management.
Beancounters
CPU scheduling / IO scheduling / Filesystem
Isolation
Benchmarks
Conclusion
Container-based OS Virtualization
Author: Liz van Dijk
Category: IT
Product: Container-based OS Virtualization
Thumb:
Summary:
Introduction
When reading about virtualization (or writing about it, for that matter), it’s often easy to get lost in the sea of possible meanings given to the word by various sources. We at Anandtech IT have been trying to provide a clearer view into the different kinds of virtualization and in this article, we'll introduce another one. Previous articles included overviews of both hypervisor-based virtualization and application virtualization. Following that trend, we now present you with container-based OS virtualization, providing you an alternative solution: one quite regularly overlooked, despite its merits. This article is meant to give our readers an idea of what this technology offers, the situations it is applicable in and of course, puts it to the test and provides you with results.
So how does it work?
If you've been keeping up with our IT articles, you've undoubtedly come across Johan’s overview of hypervisor-based virtualization. We'll take a similar approach to its container-based counterpart, giving you a strong introduction into what makes it work and guiding you through the strengths and weaknesses of the technology.
To get started, a clear view of what the system "looks" like is needed. Unlike hypervisors and applications virtualization solutions, the isolation happens inside the OS layer.
This means there is no need to virtualize the access to different hardware components as the OS still has control over all of these. Instead, several environments are running on the same kernel, each with their own processes, libraries, root and users. On one hand, this means that the containers are limited to using the same OS kernel, but on the other it allows for a higher density. Providing abstraction from the kernel also allows environments to run different distributions concurrently on the same machine and kernel.
While there are numerous software packages offering this type of virtualization, we decided to focus on a pair of products that is closely tied together, while still allowing us to go in-depth on its inner workings: OpenVZ and Parallels Virtuozzo. OpenVZ is an open source initiative, backed by Parallels, and can be seen as a testing platform for future developments of its proprietary product: Virtuozzo. Technically, both products are used for the same purpose, but are aimed at different users in the same market. Virtuozzo is a robust product, aimed at corporate customers and comes with a very large feature set, like support for both Windows and Linux platforms. At this point, it is one of the most advanced products in the Container-based OS virtualization market. On the other hand, OpenVZ is freely available, although only for use on Linux-systems, and while still providing the basic functionality one would expect from virtualization, it is probably best-suited for smaller scale environments where Linux is the primary OS in use. Since it is open source, however, OpenVZ allows us to dig deep into its inner workings. Because OpenVZ is in a sense the testing ground for Virtuozzo, we believe gaining insight into the former will provide our readers a solid base to better understand the latter, and even other similar products.
From an administrator's point of view, OpenVZ employs a system similar to Xen's paravirtualization: one of the environments is used as the base for management tools and monitoring. This is what we'll be referring to as the host environment, and while a workload could possibly be added to it, most people would leave this container as-is, to ensure functionality of the management system at times of peak load. From the host environment, we're able to see a complete overview of the resources consumed by the different guests, and run tools like vzctl to make live changes to them.
Moreover, isolation of the containers runs quite deep, ranging from the filesystem, to their own root users and user groups, process trees and inter-process communication objects. This allows a container's root user high levels of customization and tweaking of everything apart from the kernel itself.
Handling of different distributions is taken care of by so-called templates. These are the blueprints for every environment installed on top of the OpenVZ kernel. In reality, they are no more than that distribution's files rounded up in a GZIP-compressed TAR, which will be set up in an isolated part of the filesystem for the container to run in.
So, before continuing on, let's have another quick look at what separates this virtualization technology from the others:
*Unlike with hypervisors, isolation happens from within the kernel, no layers are added to the standard model, only modified.
*Allows for a very high density (tens of environments on a single machine), as long as all environments are able to run on the same kernel.
*A single kernel and centralized management tools allow for comfortable global administration of all environments.
As it stands, despite its obvious merits, we believe container-based OS virtualization is not as well-known and widespread as it could be, so let's have a closer look at the situations this technology is applied in today, and in what fields it can really shine. Keep in mind, in this next part we won't be focusing on OpenVZ specifically, but rather give a general overview of the possibilities offered by containers. Uses of Container-based OS Virtualization
So, when would we generally suggest containers to be a viable solution? Looking at the situations in which the technology is most often applied, we can see the importance of density coming up. Many hosting companies are making use of it in order to provide their customers with a heavily customizable environment in which to run their web servers. Hardware platforms running over 100 environments are no exception here. We need to keep in mind that the total memory and CPU footprint of a container is little more than that of the sum of its processes, so density scales inversely proportional to the “weight” of all containers bundled together. When high density is a priority for whatever reason, containers can provide a solid solution as opposed to a hypervisor-based one, which despite its high flexibility is still quite limited in terms of scalability. Another example where this can come into play is plain service isolation, which could be required for perhaps a DNS.
On the other hand, containers can be very interesting when performance is prioritized. Naturally, as the workload per container is increased, the maximum amount of containers on a single hardware system is decreased, but even then they provide a solid solution. Since all containers' processes have direct access to the kernel, native I/O subsystems remain intact, allowing the full power of the hardware platform to be used, which gives the technology another edge over its "rival". Another fun fact here is that in Virtuozzo (OpenVZ's "bigger brother"), applications can actually share their libraries over several containers. What this means is that the application is loaded into the memory only once, while still allowing different users to use it concurrently, albeit using a small amount of extra memory for their respective data.
Another situation where containers can come in handy is one where many user space environments need to be rolled out in a very short time span. What we're thinking of here are mostly classrooms and hands-on sessions where quick and simple deployment is of the essence. Demos of certain pieces of software often require careful planning and a controlled environment, so applications like VirtualPC or VMWare Server/Workstation are used with "temporary" full-sized virtual machines to make sure everybody is able to run them correctly. Since the creation of a container takes about as long as the actual decompression of its template environment and the startup of its processes, a single server can easily be used for participants or students to log into through SSH or even VNC, giving each user an easily administrated environment, while still allowing them to experiment with high-level permissions.
In this case, it's also positive that everything happens on a single system that is completely accessible to the root user of the "host" container. In this sense, containers reduce both server sprawl and OS sprawl, allowing for high manageability. For example, using a single kernel reduces the amount of time spent patching every individual virtual machine separately. "Broken" containers are easily removed and recreated on the fly, as making use of templates removes the need to go through the steps of installing a fresh OS every time a new environment is needed.
Thinking further along this line brings us to the next point of interest: using containers as a base for VDI purposes. Combining all of the above factors, it is easy to see the technology's merits as a means to bundle desktop environments onto a central server. (needsmoreblahblah)
The big trade-off
There are undeniably a few downsides to this way of approaching virtualization. The first that comes to mind is that when density on a single hardware platform increases, so do its needs for constant availability. Using a single kernel in this case raises the concern on whether the kernel becomes an important single point of failure in the system. Unlike hypervisors, which are in a sense very light-weight and therefore very stable kernels, container-based OS virtualization depends on a slightly modified version of already existing kernels, thus essentially taking over existing weaknesses of both the Windows and Linux platforms.
It has to be noted that a lot of work is put into ensuring kernel stability on all container-based solutions, and unexpected kernel crashes are at least to be deemed "unlikely". Nevertheless, a bit of healthy paranoia when IT is involved is certainly not an obsolete luxury.
Secondly, while a strong feature set is definitely maintained, containers do take away a lot of flexibility from the hypervisor-based solutions. For one, many might find the inability to use different OS's next to each other to be a big handicap. Moreover, while support for live migration does exist on linux systems (both OpenVZ and Virtuozzo have implemented this), it is still missing on Windows implementations.
(Need to find some more stuff to put in here)
Sadly enough, despite all its possibilities, Container-based OS virtualization is most limited, it seems, by its plain invisibility in the current market place. For some reason, it has been quietly pushed into the background by all the current hype surrounding different virtualization technologies, and major software companies fighting for domination of this "gold mine". While it is currently used by most, if not all hosting companies, this type of virtualization seems unable to expand its customer base beyond that. Whether through lack of marketing power or a seemingly less attractive feature list, most businesses seem unaware of its potential. Naturally, it goes without saying that just like hypervisors aren't the perfect "one size fits all" solution, neither are containers, but it is still rather peculiar how a solution as polyvalent could escape the recent surge of interest into virtualization and all its aspects.
(?)
A closer look into OpenVZ's inner workings
Now that we've given a clear overview of containers in general, it is time to move on to the more technical stuff, namely figuring out how they do in fact work. As stated before, most of our research into the subject has gone into OpenVZ, but we feel the actual subjects tackled here do a great job illustrating the challenges faced by any developer of container-based solutions.
To gain a full understanding of how the partitioning of an operating system takes place, it is rather important to grasp the basics of how an OS operates. Since entire books have been written on that subject alone, and it is mostly outside the scope of this article, I'll direct those new to the subject to page 2 of Johan's article on hypervisors and their inner workings.
For now, it is important to know that in our modern day x86 architecture, there are two modes a process can run in, namely a privileged kernel mode and a "secure" user mode. The CPU is able to distinguish between these two modes by each process’ assigned memory addresses, where a 2-bit code makes the difference between kernel space addresses and user space addresses. The reason they are called addresses is because each of them points to a specific piece of memory. These pieces are generally called "pages", they're the smallest unit of memory allocation an operating system can use, and in x86 systems are usually blocks of 4kB each.
The above description introduces two important factors.
Since containers all share the same kernel, kernel mode processes are not isolated, but run outside the environments themselves. Therefore, containers are to be used only for isolation of applications running in the user space.
We raise the issue of memory management. Evidently, a single OS has an elaborate system in place for this, but how is it approached in a partitioned OS? In short, how do we prevent one container from using up all resources needed to support another?
OpenVZ has implemented a system called beancounters for this purpose. In the following pages, we will have a look at how this and other methods are combined to achieve isolation and efficient resource management.
Beancounters
CPU scheduling / IO scheduling / Filesystem
Isolation
Benchmarks
Conclusion