Until recently we would try to switch them from full daily backups to full on Mondays and incremental the rest of the week. The problem with this is that this really only works up until about 120GB. After that you are looking at more than 2 tapes. If someone has to switch tapes on a Monday while users are changing files, it is very hard to get a consistent backup. This problem got me thinking, what is a relatively cheap way to get consistent backups of the server, retain more copies of the backups with less pace consumed, and be able to backup to more then two tapes without having backups run on data users are interacting with, which could stress the system, hurt performance, and cause inconsistent backups.

My solution was to purchase a low-end server like an HP ML-110. Put 2 500GB sata drives in the system, and 2 40GB drives with hardware raid 1 for the system OS files. I then installed Solaris 10 and made a ZFS pool with the two 500GB drives mirrored. I shared the ZFS pool with NFS and Samba (NFS for Unix and samba for windows 2003 servers). I have the servers do a full backup to the ZFS pool on Monday, which is about 200GB. The incremental backups the rest of the week is about 50GB total. I set the backups to overwrite the data each week, so as not to use up too much disk space. This brings full weeks backups to about 220GB with compression on the ZFS pool. I then take a snapshot of the entire pool on Saturday and keep that snapshot for 4 weeks. This means that even though I am overwriting the backup files each week. The snapshots only take up as much space as the differences, so for about 380GB of disk space I have 1TB worth of backups on this Solaris box. And it gets even better. I set the tape drive to backup from the Solaris box every day after the other servers dump their backups to it the previous night. Since the data is not live data the backups are consistent and also the backups can be done during the day so someone can insert tape after tape while people are in the building.

This is how ZFS has made life easier for me at one client site. This solution is scalable and priced at one-third the cost of a 400GB auto-loading tape drive. If this solution works out over the next couple of months I fully intend to make this my primary solution for those clients that can’t afford bigger tape drives.

Solaris 10 is a very powerful scalable operating system that feels tightly integrated and highly responsive. However for those of us with a strong Linux or FreeBSD background Solaris can feel a little unfamiliar and even downright confusing. This post is made up of a few things I wish I knew before I started my Solaris 10 adventure. I hope it will make things a little easier for those of you wanting to take the plunge into a true enterprise operating system.

A zone is analogous to a VMware machine but with some distinct advantages as well as some disadvantages. To start explaining zones let’s start by comparing it to VMware since most people are familiar with VMware. VMware is a software program that runs on a host operating system,. This could be Windows, Linux, or a custom kernel in the VMware ESX series. VMware provides a hardware abstraction layer which it uses to create mini virtual computers. The advantages of this setup are as follows: You can create a custom virtual computer designed for the operating system and application you want to run. This means that you have a very high level of customizability which allows you to tailor the virtual machine to provide the best performance for the program you want to run. VMware also supports ? many different operating systems. You could run VMware on a Linux box and then run Windows 2003 in one virtual machine, and a FreeBSD NFS server in another. From the point of view of the guest operating systems, they believe they are on standalone hardware and are unaware of the host OS and other guest OS’s which may be running.

      So this is a good thing and it provides a fair amount of flexibility, but you pay for this flexibility in several key areas. The first area is licensing costs. VMware is not free. While they do have free versions for testing and evaluation, all production software comes with a license. The second area in which you pay is overhead. VMware while a very good virtualization tool has a good amount of overhead. This is because it must present a complete environment to each guest operating system installed. This means that it must schedule CPU time IO and other resources for each of the guest operating systems in real time. This can really be a drag. The other area in which you pay a price is transparency. The cost of the ability to run multiple operating systems on one machine is that each operating system must be segregated to its own virtual hardware emulation layer. What this means is that while VMware has some distinct advantages in lowering the cost of infrastructure it does not lower the cost of human resources and administration.

     Now what does that mean Administration Cost. Simply put, even though you save the cost of purchasing multiple machines by virtualizing them, from the administrative standpoint each operating system must be monitored and maintained as if it was a separate machine. This means from the host operating system you can see that VMware is using %90 of your CPU time, but you don’t know which of the virtual computers is using what. You also do not know what processes are running on each virtual machine or what the load is of each machine. This means that each machine must be logged into individually and administered as if it was the only operating system on the machine.

     So now let’s look at a situation in which VMware might not be the best solution. I think we all agree the VMware is a good choice when you need to have multiple operating systems on one machine. However what if you have a web services company and you have 1 machine running DNS, one machine running DHCP, one machine as a NFS server for file sharing, and 1 server to host your website. Now let’s further assume that you want to virtualize and that as a web services company you are running some type of UNIX/POSIX system. If this is the case you could standardize on Solaris 10 with zones and not only save money on TCO (total cost of ownership) of the hardware itself but also on the IT resources needed. A Solaris zone is a container stored in a folder on a Solaris UNIX machine that can have either a complete copy of the Solaris Operating system, including its own file system. It can also share some common directories ? (read only) with the Solaris machine and be installed with as little as 82mb of disk space. The Solaris zones will boot up as if they were the only Solaris OS on the machine and they are also completely isolated from the host machine which means a web server security breach will not affect the other zones in any way.

     Now what are the advantages of this? In this scenario each zone would be completely secure and segregated from the host Solaris OS. Each zone would have its own kernel with its own drivers and its own processes. Solaris will even leverage its Mainframe heritage too allow you to split up the physical hardware into resource pools. So say you wanted to have 2 cpu’s for the system, 2 cpu’s for oracle 10g, and 2 cpu’s for the web server this could be done without a problem. There are also some other advantages to Solaris zones. You can browse to each zone through the file tree. For example

If the zone is install at /export/zone1 then while the zone is running I can copy files to the “/” or root of that zone by browsing to? “/export/zone1/root/“

It is also important to note that “/etc” “/usr” “/mnt/” and all other folders and file systems can be found under “/export/zone1/root/” as well.

So this is a big advantage files can be copies between zones and the host computer with little trouble. This is also important because while you can browse to that zone from the host computer, the zone cannot browse out of its path. This means great security while providing ease of administration. You can see from this example how easy it would be to browse to “/export/zone1/root/var/admin” ? and view the logs for that system without ever logging in to it.

It is also important to note that each system has its own root account as well as other system and user accounts. Another big advantage of Solaris zones is that any of the system monitoring commands such as iostat vmstat and prstat, will show all processes in each zone and there memory utilization cpu time in the case of vmstat and prstat (prstat is like top for Linux Admins). It will also show disk IO in the case of iostat, and as always? you can filter the input for each zone or display all info with zones separated. So you can see how this saves money and reduces TCO by allowing one login to manage all systems that are virtualized. You can also create zones to experiment or roll out new services in as a testing measure.

     Indeed this website is running in a Solaris zone. It uses Apache2 php5 and Mysql5.? PHP requires the installation of a lot of GNU software which I did not want mucking up my core Solaris system. So I installed it in a zone. Everything on this web server is isolated in that zone which makes keeping my system orderly and tidy an easy job. It also means that any misconfigurations or other errors that may have left my entire system vulnerable in the past is less of a problem. The worst thing that can happen is an attacker neutralizes my web zone, in which case I can restore from a mirror zone, I keep for failover.

Stay tuned for part two in which I will have some good examples of configuring Zones.

END PART 1