I’m excited about today’s launch of Solaris 11 – I’ve been contributing to Solaris for quite a while now, pretty much since 1996, but my involvement in S11 has been the most fun I’ve had in all releases so far.
I’ve talked before about some of the work I’ve done on IPS over the last two years – pkg history, pkgdepend (and here), pkglint and pkgsend and most recently, helping to put together the IPS Developer Guide.
Today, I’m going to talk about the system repository and how I helped.
How zones differ from earlier releases
Zones that use IPS are different than those in Solaris 10, in that they are always full-root: every zone contains its own local copy of each package, they don’t inherit packaged content from the global zone as "sparse" zones did in Solaris 10.
This simplifies a lot of zone-related functionality: for the most part, administrators can treat a zone as if it were a full Solaris instance, albeit a very small one. By default new zones in S11 are tiny. However, packaging with zones is a little more complex, and the system aims to hide that complexity
Some packages in the zone always need to be kept in sync with those packages in the global zone. For example, anything which delivers a kernel module and a userland application that interfaces with it must be kept in sync between the global zone and any non-global zones on the system.
In earlier OpenSolaris releases, after each global-zone update, each non-global zone had to be updated by hand, attaching and detaching each zone. During that detach/attach the ipkg brand scripts determined which packages were now in the global zone, and updated the non-global zone accordingly.
In addition, in OpenSolaris, the packaging system itself didn’t have any way of ensuring that every publisher in the global zone was also available in the non-global zone, making updates difficult if switching publishers.
Zones in Solaris 11
In Solaris 11, zones are now first-class citizens of the packaging system. Each zone is installed as a linked image, connected to the parent image, which is the global zone.
During packaging operations in the global zone, IPS recurses into any non-global zones to ensure that packages which need to be kept in sync between the global and non-global zones are kept in sync.
For this to happen, it’s important for the zone to have access to all of the IPS repositories that are available from the global zone.
This is problematic for a few reasons:
- the zone might not be on the same subnet as the global zone
- the global-zone administrator might not want to distribute SSL keys/certs for the repos to all zone administrators
- the global zone might change its publisher configuration, requiring publisher configuration change in every non-global zone
The System Repository
The system repository, and accompanying zones-proxy services was our solution to the list of problems above.
The SMF Services responsible are:
The first two services run in the global zone, the last one runs in the non-global zones.
With these services, the system repository shares publisher configuration to all non-global zones on the system, and also acts as a conduit to the publishers configured in the global zone. Inside the non-global zone, these proxied global-zone publishers are called system publishers.
When performing packaging operations inside a zone that accesses those publishers, Solaris proxies access through the system repository. While proxying, the system repository also caches any file-content that was
downloaded. If there are lots of zones all downloading the same packaged content, that will be efficiently managed.
If you don’t care about how all this works behind the scenes, then you can stop reading now.
There’s three parts to making all of the above work, apart from the initial linked image functionality that Ed worked on, which was fundamental to all of the system repository work.
- IPS client/repository support
- Zones proxy
- System repository
IPS client/repository support
Brock managed the heavy lifting here. This work involved:
- defining an interchange format that IPS could use to pass publisher configuration between the global and non-global zones
- refreshing the system repository service on every parent image publisher change
- allowing local publisher configuration to merge with system publisher configuration
- ensuring that system-provided publishers could not have their order changed
- allowing an image to be created that has no publishers
- toggling use of the system publisher
The zones proxy client, when started in the non-global zone creates a socket which listens on an inet port on 127.0.0.1. It passes the file descriptor for this socket to the zones proxy daemon via a door call.
The zones proxy daemon then listens for connections on the file descriptor. When the zone proxy daemon receives a connection, it proxies the connection to the system repository.
This allows the zone to access the system repository without any additional networking configuration needed (which I think is pretty neat – nicely done Krister!)
The system repository itself consists of two components:
- A Python program, /usr/lib/pkg.sysrepo
- A custom Apache 2.2 instance
Brock initially prototyped some httpd.conf configurations, and I worked on the code to write them automatically, produce the response that the system repository would use to inform zones of the configured publishers, and also worked out how to proxy access to file-based publishers in the global zone, which was an interesting problem to solve.
When you start the system-repository service in the global zone, pkg.sysrepo(1) determines the enabled, configured publishers then creates a response file served to non-global zones that want to discover the publishers configured in the global zone. It then uses a Mako template from /etc/pkg/sysrepo/sysrepo_httpd.conf.mako to generate an Apache configuration file.
The configuration file describes a basic caching proxy, providing limited access to the URLs of each publisher, as well as allowing URL rewrites to serve any file-based repositories. It uses the SSL keys and certificates from the global zone, and allows proxies access to those from the non-global zone over http.
(remember, data served by the system repository between the zone and non-global zone goes over the zones proxy socket, so http is fine here: access from the proxy to the publisher still goes over https)
The system repository service then starts an Apache instance, and a daemon to keep the proxy cache down to its configured maximum size. More detail on the options available to tune the system repository are in pkg.sysrepo(1) man page.
The practical upshot of all this, is that all zones can access all publishers configured on the global zone, and if that configuration changes, the zones publishers automatically change too. Of course, non-global zones can add their own publishers, but aren’t allowed to change the order, or disable any system
Here’s what the pkg publisher output looks like in a non-global zone:
root@puroto:~# pkg publisher PUBLISHER TYPE STATUS URI solaris (non-sticky, syspub) origin online proxy://http://pkg.oracle.com/solaris11/release/ mypublisher (syspub) origin online http://localhost:1008/mypublisher/89227627f3c003d11b1e4c0b5356a965ef7c9712/ test (syspub) origin online http://localhost:1008/test/eec48b7c8b107bb3ec9b9cf0f119eb3d90b5303e/
and here’s the system repository running in the global zone:
$ ps -fu pkg5srv | grep httpd pkg5srv 206 2334 0 12:02:02 ? 0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht pkg5srv 204 2334 0 12:02:02 ? 0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht pkg5srv 205 2334 0 12:02:02 ? 0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht pkg5srv 939 2334 0 12:46:32 ? 0:00 /usr/apache2/2.2/bin/64/httpd.worker -f /system/volatile/pkg/sysrepo/sysrepo_ht
Personally, I’ve found this capability to be incredibly useful. I work from home, and have a system with an internet-facing non-global zone, and a global zone accessing our corporate VPN. My non-global zone is able to securely access new packages when it needs to (and I get to test my own code at the same time!)
Performing a pkg update from the global zone ensures that all zones are kept in sync, and will update all zones automatically (though, as mentioned in the Zones administration guide, pkg update <list of packages> will simply update the global zone, and ensure that during that update only the packages that cross the kernel/userland boundary are updated in each zone.)
Working on zones and the system repository was a lot of fun – hope you find it useful.