Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

In this guide, we'll describe the features and benefits of SMF, point out some parts of Solaris that have changed significantly, and show how to accomplish typical administration tasks using SMF. A comprehensive guide to SMF and Predictive Self-Healing is available on Sun's BigAdmin website.

The following figure shows the primary components of the SMF framework. When you boot an image, SMF updates the service configuration repository if necessary, reads the repository data, and starts service instances that are enabled. In the following figure, libscf is the library interface that the restarters use to interact with the service configuration repository. Interaction between the service configuration repository and libscf library interfaces is managed by the svc.configd daemon. The svcs, svcprop, svcadm, and svccfg commands are the interface that administrators use to interact with the service configuration repository.

...

Features

The Service Management Facility has improved several aspects of the Solaris administrative model. Some of the most notable updates are:

...

  • Enabling and disabling services

    Prior to Solaris 10, there wasn't a good way to permanently disable a service in Solaris. The typical method used is to rename the relevant rc script to a name that won't get executed, but that change will get overlooked the next time the system is upgraded. Furthermore, inetd-based services are enabled and disabled by a totally different method -- editing a configuration file. Under SMF, both types of services can be configured using the svcadm(1M) command, and the changes will persist if the machine is upgraded. Here's a comparison of how to enable and disable some services:

Old method

SMF Method

mv /etc/rc2.d/S75cron /etc/rc2.d/x.S75cron

svcadm disable system/cron:default

edit /etc/inet/inetd.conf, uncomment the finger line

svcadm enable network/finger:default

...

The last argument to svcadm in these examples is the FMRI of the service.

Note that svcadm should only be used for SMF services -- legacy rc script-controlled services work the same as in past releases.

  • Stopping, starting, and restarting services

    Traditionally, services have been started by an rc script run at boot, run with the argument start. Some rc scripts provide a stop option, and a few also allow restart. In SMF, these tasks are all accomplished with the svcadm(1M) command:

Old method

SMF Method

/etc/init.d/sshd stop

svcadm disable -t network/ssh:default

/etc/init.d/sshd start

svcadm enable -t network/ssh:default

/etc/init.d/sshd stop; /etc/init.d/sshd start

svcadm restart network/ssh:default

kill -HUP `cat /var/run/sshd.pid`

svcadm refresh network/ssh:default

 

The "-t" option to svcadm enable and svcadm disable indicates that the requested action should be temporary -- it will not affect whether the service is started the next time that the system boots. This is in contrast to the "Enabling and disabling services" example, above.

As with the enabling and disabling of services, svcadm should not be used to control rc script-controlled services; they continue to work the same as in past releases.

  • Observing the boot process

    As mentioned in the "Notable Changes" section, the boot process is much quieter by default than in previous releases of Solaris. This was done to reduce the amount of uninformative "chatter" that might obscure any real problems that might occur during boot.

    Some new boot options have been added to control the verbosity of boot. One that you may find particularly useful is "-m verbose", which prints a line of information when each service attempts to start up. This is similar to the default boot mode for some other UNIX-based and UNIX-like operating systems. Verbose boot looks like this:

    Code Block
        {1} ok boot -m verbose
    
        Rebooting with command: boot -m verbose
        Boot device: /pci@1c,600000/scsi@2/disk@0,0:a  File and args: -m verbose
        SunOS Release 5.10 Version Generic 64-bit
        Copyright 1983-2004 Sun Microsystems, Inc.  All rights reserved.
        Use is subject to license terms.
        [ network/pfil:default starting (pfil) ]
        [ network/loopback:default starting (Loopback network interface) ]
        [ system/filesystem/root:default starting (Root filesystem mount) ]
        Oct 18 13:53:02/13: system start time was Mon Oct 18 13:52:57 2004
        [ network/physical:default starting (Physical network interfaces) ]
        [ system/filesystem/usr:default starting (/usr and / mounted read/write) ]
    	( more service messages elided )
        [ system/filesystem/local:default starting (Local filesystem mounts) ]
        [ network/ntp:default starting (network time protocol (NTP)) ]
        [ system/utmp:default starting (utmpx monitoring) ]
        [ system/filesystem/local:default starting (Local filesystem mounts) ]
        [ system/console-login:default starting (Console login) ]
    
        demobox console login: checking ufs filesystems
        /dev/rdsk/c0t0d0s7: is logging.
        Oct 18 13:53:14/50: system/system-log:default starting
        Oct 18 13:53:14/51: network/inetd:default starting
        Oct 18 13:53:14/52: system/cron:default starting
    	( more service messages elided )
    	

    The order of the service start messages may change from boot to boot, because SMF starts services in parallel according to their dependency relationships.

    If a service fails to start successfully, warning messages will be printed in addition to the start message. Here's an example where the NTP service failed to start up:

    Code Block
        [ system/filesystem/local:default starting (Local filesystem mounts) ]
        [ network/ntp:default starting (network time protocol (NTP)) ]
        Oct 25 13:58:42/49 ERROR: svc:/network/ntp:default: 
                 Method "/lib/svc/method/xntp" failed with exit status 96.
        Oct 25 13:58:42 svc.startd[4]: svc:/network/ntp:default: 
                 Method "/lib/svc/method/xntp" failed with exit status 96.
        [ network/ntp:default misconfigured (see 'svcs -x' for details) ]
        [ system/utmp:default starting (utmpx monitoring) ]
    	( more service messages elided )
    	

    The first two error messages would appear during both normal boot and verbose boot; the last one ("network/ntp:default misconfigured ...") would only appear during verbose boot.

  • Discovering what's going wrong

    Solaris has not had a comprehensive place to look for problems with system services. Some solutions exist to help catch and diagnose these problems, ranging from coreadm(1M) logging to site-specific monitoring scripts to comprehensive products such as Sun Cluster. The new svcs(1) command includes an "explain" option ("svcs -x"), which prints out detailed, solution-driven messages about the services that are not running. svcs -x shows when and why the service failed, provides pointers to more information about the problem, and lists what other services are affected by this problem.

    Continuing with the example of the NTP service failing to start up:

    Code Block
        # svcs -x
        svc:/network/ntp:default (Network Time Protocol (NTP).)
         State: maintenance since Mon Oct 18 13:58:42 2004
        Reason: Start method exited with $SMF_EXIT_ERR_CONFIG.
           See: http://sun.com/msg/SMF-8000-KS
           See: ntpq(1M)
           See: ntpdate(1M)
           See: xntpd(1M)
        Impact: 0 services are not running.
    	

    The NTP service has been placed into maintenance mode because the startup script indicated that there was a problem with the service's configuration. Further information about the service failure is available in the service's log file in the /var/svc/log directory (or the /etc/svc/volatile directory). The log file name is based off the short form of the FMRI, with "/"'s replaced by "-"'s. So the log file for the svc:/network/ntp:default service is /var/svc/log/network-ntp:default.log. This log file quickly led to the conclusion that the NTP daemon's configuration file, /etc/inet/ntp.conf, had been removed.

    Another example shows SMF's ability to track dependencies and point out problems relating to disabled services. We use the "-v" option in this example to see the list of impacted services.

    Code Block
        # svcs -x -v
        svc:/application/print/server:default (LP Print Service)
         State: disabled since Mon Oct 18 16:17:27 2004
        Reason: Disabled by an administrator.
           See: http://sun.com/msg/SMF-8000-05
           See: man -M /usr/share/man -s 1M lpsched
        Impact: 1 service is not running:
    	    svc:/application/print/rfc1179:default
    	

    Here, the application/print/server:default service has been explicitly disabled, but another service which depended on it (application/print/rfc1179:default) was not disabled. So the disabling of the first service has kept the second one from running.

  • Observing services

    In earlier versions of Solaris, the only way to see what services were available was to use the ps(1) command and list all the active processes on the system, and then look around for the names of processes that match the names of service applications. Unfortunately, since most systems have many processes, and new services are introduced with each new version of Solaris and when other software packages are added, it's very difficult to track things this way. To further complicate the situation, many modern services are no longer implemented as single processes. Some services are implemented as collections of processes, multi-threaded processes, or both simultaneously.

    The new svcs(1) command makes it much easier to observe the status of a system service. The "-p" option shows all the processes associated with a service:

    Code Block
        % svcs -p network/smtp:sendmail
        STATE          STIME    FMRI
        online         18:20:30 svc:/network/smtp:sendmail
    		   18:20:30      655 sendmail
    		   18:20:30      657 sendmail
    
        % ps -fp 655,657
    	 UID   PID  PPID   C    STIME TTY   TIME CMD
    	root   655     1   0 18:20:30 ?     0:01 /usr/lib/sendmail -bd -q15m
           smmsp   657     1   0 18:20:30 ?     0:00 /usr/lib/sendmail -Ac -q15m
    	

    The "-d" option shows what other services this service depends on, and the "-D" option shows what other services depend on this service:

    Code Block
        % svcs -d network/smtp:sendmail
        STATE          STIME    FMRI
        online         18:20:14 svc:/system/identity:domain
        online         18:20:26 svc:/network/service:default
        online         18:20:27 svc:/system/filesystem/local:default
        online         18:20:27 svc:/milestone/name-services:default
        online         18:20:27 svc:/system/system-log:default
        online         18:20:30 svc:/system/filesystem/autofs:default
        % svcs -D network/smtp:sendmail
        STATE          STIME    FMRI
        online         18:20:32 svc:/milestone/multi-user:default
    	

    We can see that sendmail requires networking, local file systems, name services, the syslog daemon, and the automount daemon to be running before it will run, and sendmail itself must be running before the multi-user milestone can be reached. The service start times (the STIME column) illustrate that these dependencies have been followed.

  • Changing run levels

    SMF has introduced the concept of milestones, which supplant the traditional notion of run levels. Run levels provide a basic description of the set of services running on the machine, traditionally grouped as the services necessary for one user to log in on the machine console (run level S), and for multiple users to log in to the machine (run levels 2 and 3). These system states are represented in SMF as milestones, which are stable services that represent a group of other services. "svcs -d" can be used to see what services must be running before a milestone is reached.

    svcadm(1M) is now the preferred method of setting the system's default run level. This is done with the milestone subcommand and the FMRI of a valid milestone:

Old method

SMF Method

edit /etc/inittab

svcadm milestone -d milestone/single-user:default

...

The "-d" option indicates that the default milestone should be set to the named FMRI. Without "-d", "svcadm milestone" transitions the system to the named milestone immediately.

The boot process has been updated to be aware of milestones. In addition to the traditional "boot -s" (boot into single-user mode), there is now "boot -m milestone=<milestone>", to boot to the named milestone. <milestone> can be "single-user", "multi-user", or "multi-user-server", as well as the special milestones "all" (all enabled services online) and "none" (no services at all). The "none" milestone can be very useful in repairing systems that have failures early in the boot process.

Booting to the single-user milestone (with "-m milestone=single-user") is slightly different than the old "boot -s". When the system is explicitly booted to a milestone, exiting the console administrative shell will not transition the system to multi-user mode, as "boot -s" does. To move to multi-user mode after "boot -m milestone=single-user", use the command "svcadm milestone milestone/multi-user-server:default".

  • Enabling, disabling, and monitoring legacy services

    Services that are started by traditional rc scripts (referred to as legacy services) will generally continue to work as they always have. They will show up in the output of svcs(1), with an FMRI based on the pathname of their rc script, but they can not be controlled by svcadm(1M). They should be stopped and started by running the rc script directly.

    As mentioned in the "Notable Changes" section, rc scripts may not run at exactly the same point in boot as they had in earlier versions of Solaris. In particular, scripts which depend on running before certain Solaris-provided rc scripts may encounter problems. The vast majority of scripts should continue to work without any trouble, though.

  • Adding new services to inetd.conf

    The Internet services daemon, inetd(1M), has been rewritten as part of SMF. It stores all of its configuration data in the SMF database, rather than /etc/inet/inetd.conf, allowing the SMF tools to be used to control and observe inetd-based services. Most inetd-based services that ship with Solaris will no longer have entries in inetd.conf. To provide compatibility for services which haven't converted to SMF, entries can still be added to inetd.conf using the same syntax as always, and the new inetconv(1M) command will convert the new services to SMF services. inetconv should always be run after editing /etc/inet/inetd.conf; it can be run without any arguments.

...