This section will first take you on a tour of the source tree. After that, it will proceed to discuss several of the aspects surrounding the organization of our makefiles and where object files show up.
All sources are found under usr/src. This includes both the sources used to
build the ON consolidation and sources for tools and other peripheral utilities
needed to build but not shipped as part of illumos. Because it includes only ON
sources, it does not contain Java, the windowing system, or packaging and
installation tools. Because of contractual obligations, it may not include all
code from third-party hardware vendors. The usr/src directory has several
subdirectories which are described here.
illumos Developer's Guide
cmdThis directory contains sources for the executable programs and scripts that are part of ON. It includes all the basic commands, daemons, startup scripts, and related data. Most subdirectories are named for the command or commands they provide; however, there are some exceptions listed here.
cmd/AdmMiscellaneous key system files, such as crontabs and data installed in /etc.
cmd/cmd-cryptoBasic cryptographic utilities, such as elfsign and digest.
cmd/cmd-inetNetwork commands and daemons, including the Berkeley r-commands, PPP, telnet, the inetd super-server, and other network-related utilities.
cmd/fs.dUtilities for checking, mounting, unmounting, and analyzing filesystems.
cmd/mdbContains the sources for the modular debugger, mdb(1), the kernel debugger,
kmdb(1), and the majority of the modules.
cmd/netfilesIP port definitions and name service switch configuration files installed into /etc.
cmd/ptoolsUtilities for manipulating and observing processes; these are based on proc(4) and libproc interfaces.
cmd/sgsSoftware Generation System. This directory contains binary utilities, such as
ld(1), ar(1), and mcs(1), and development tools such as lex(1),
yacc(1), and m4(1). Note that this directory also includes several libraries
and headers used by these tools.
commonFiles which are common among cmd, lib, stand, and uts. These typically
include headers and sources to basic libraries used by both the kernel and user
programs.
headUserland header files (kernel headers are in uts/). Note that only libc headers
should be stored here; other libraries should have their headers in their own
subdirectories under lib/.
libLibraries. Most subdirectories are named for the library whose sources they contain or are otherwise self-explanatory.
pkgContains the manifests and package lists for building ips packages from the ON sources.
prototypesSample files showing format and copyright notices.
psmPlatform-specific modules. Currently this contains only OBP and most of the boot code.
standStandalone environment code. This is used for booting; for example, code for reading from UFS and the network is here.
toolsDevelopment tools and sources. See README.tools for more information about each tool; the file should be updated as tools are added or removed.
ucbcmdCommands and daemons installed into /usr/ucb (for SunOS 4.x compatibility).
ucbheadHeader files installed into /usr/ucb (for SunOS 4.x compatibility).
ucblibLibraries installed into /usr/ucb (for SunOS 4.x compatibility).
utsKernel sources are here (UTS == UNIX Time Sharing). There are numerous subdirectories of uts which are of interest:
uts/commonAll platform-independent kernel sources. Nearly all of the illumos kernel is here; only a few small parts are architecture-dependent.
uts/common/confSystem configuration parameters.
uts/common/contractCode to support process contracts. See contract(4) and libcontract(3LIB) for
more information on process contracts.
uts/common/cprCheckPoint-and-Resume support. This implements suspend and resume functionality.
uts/common/cryptoKernel cryptographic framework. See cryptoadm(1M) for more information.
uts/common/ctfKernel code for handling Compact C Type Format data.
uts/common/desImplements the old Data Encryption Standard. This is used by KCF.
uts/common/dispDispatcher, thread handling, and scheduling classes.
uts/common/dtraceCPU-independent dtrace(7D) kernel support.
uts/common/execCode for handling userland binary executable types (a.out, ELF, etc).
uts/common/fsFilesystems.
uts/common/inetIP networking subsystem, including IPv6.
uts/common/ioI/O subsystem. Most of the code in this directory is device drivers (and pseudo-device drivers).
uts/common/ippIP policy framework; includes QoS and other traffic management.
uts/common/kmdbKernel modular debugger driver. See kmdb(1).
uts/common/krtldKernel runtime linker/loader. This is responsible for handling loadable modules and symbol resolution; it is analogous to ld.so.1, and shares code with it.
uts/common/ktliKernel TLI (Transport Layer Interface).
uts/common/netHeader files; most are shipped in /usr/include/net.
uts/common/netinetHeader files; most are shipped in /usr/include/netinet.
uts/common/nfsNetwork File System headers shipped in /usr/include/nfs.
uts/common/osCore operating system implementation. This includes such varied aspects as privileges, zones, timers, the DDI/DKI interfaces, and high-level locking mechanisms.
uts/common/pcmciaPCMCIA I/O subsystem and drivers.
uts/common/rpcRemote Procedure Call subsystem used by services such as NFS and NIS.
uts/common/rpcsvcGenerated RPC header files shipped in /usr/include/rpcsvc.
uts/common/sysHeader files shipped in /usr/include/sys. These same headers are used to build
the kernel as well (if the _KERNEL preprocessor symbol is defined).
uts/common/syscallSystem call implementations. Most syscalls are implemented in files matching
their names. Note that some system calls are implemented in os/ or other
subdirectories instead.
uts/common/vmVirtual memory subsystem.
uts/common/zmodCompression/decompression library.
uts/i86pcArchitecture-dependent files for x86 machines. The architecture-dependent
directories (i86pc, sun, sun4, sun4u) all have a set of subdirectories
similar to common/ above.
uts/intelISA-dependent, architecture-independent files for x86 machines. Note that all architecture-independent source files are built into objects in this hierarchy.
uts/sfmmuCode specific to the SpitFire memory management unit (UltraSPARC).
uts/sparcISA-dependent, architecture-independent files for SPARC machines. Note that all architecture-independent source files are built into objects in this hierarchy.
uts/sunSources common to all Sun implementations. Currently this contains a small
number of device drivers and some headers shipped in /usr/include/sys.
uts/sun4Sources common to all sun4* machine architectures.
uts/sun4uArchitecture-dependent sources for the sun4u architecture. Each system implementation has a subdirectory here.
You may note that we started inside of usr/src. There is just one other
component which is outside of usr/src, the directory exception_lists. The
exception_lists contain several files that relate to exceptions to the
standard checks that we make. This covers things like packaging and cstyle.
Several of these exceptions exist for historical reasons. If you find yourself
making additions to these files, you should think twice about what you're doing.
There are two basic strategies that can be used in the creation of object files and finished binaries (executables and libraries):
(a) place objects in a dedicated directory hierarchy parallel to the sources
(b) place objects in the same directories as the sources they are built from
ON actually uses each of these approaches in different parts of the tree. Strategy (a) must be used for all kernel code and many libraries, is preferred for new code, and will be described in detail here. There are several legacy variations on strategy (a) as well as instances in which strategy (b) is still used; the majority of these are in the cmd hierarchy. The entire uts hierarchy has been converted to strategy (a) as described below, and you will see this same approach used throughout much of the rest of the tree.
First, each platform-independent module has zero or one build directory per
architecture. An architecture in this case can be a machine (sun4u, i86pc)
or a processor architecture (intel, sparc). The path to this location is
always usr/src/uts/<platform>/<module>. The module name in this case is what's
found in /kernel/drv or a similar location, and in the case of device drivers
or STREAMS modules should always match the name of the man page describing that
driver or module.
The only files normally present in this directory for a clean tree are
makefiles. After a build, these directories contain one or more of obj32,
obj64, debug32, and debug64 directories. These directories contain the
object files and finally linked modules that are later installed into the kernel
directory and other locations in the prototype.
'Implementation architecture'-independent modules are produced in individual
directories (one per module) under the 'instruction-set architecture' directory
(i.e.: sparc). Similarly, 'implementation architecture'-dependent modules are
produced in individual directories under the 'implementation architecture'
directory (i.e.: sun4, sun4u, i86pc).
Platform-dependent modules (including 'unix') may be built more than once in different locations. For example, they'll be built under each of the different implementations of a platform. This is more common on sparc than x86.
The sources are not contained in these build directories.
Most libraries and some commands and daemons are built using makefiles very similar to those used to build the kernel. Accordingly, intermediate objects, shared objects, libraries, and executables are built by small makefile fragments and placed in dedicated ISA-specific subdirectories.
Other commands' build systems place objects in the same directories as the sources. See Component Anatomy, Creation, and Modification for more information on how commands are built.
Most of the cmd tree is directly based on the original System V Release 4 source, which uses strategy (b) described above. Since most commands and daemons do not need to provide both 32- and 64-bit versions, or do anything special when building for different architectures, this strategy is adequate and appropriate for most commands and has been applied even to new subdirectories. In situations in which architecture-dependent build options are needed or multiple ISA versions of a program must be delivered, this strategy is unworkable and the more general approach of multiple per-ISA object directories must be used instead. This latter approach is similar to the approach used for kernel modules.
The lib hierarchy is somewhat simpler; nearly all subdirectories must use per-ISA object file locations and makefiles.
A few directories do not appear to follow any rule or pattern, such as
cmd/cmd-inet and cmd/agents. These are primarily historical artifacts of
Sun's internal project organization.
This discussion is intended to provide a step-by-step explanation of what
targets exist and how they are built by the makefiles. I ignore the
platform-specific module architecture because it is unlikely to be of
significant interest except to hardware support engineers. The three main
subtrees of interest are the kernel (uts), commands and daemons (cmd), and
libraries (lib). The next three subsections cover these three subtrees in
turn. There are also a handful of makefiles which apply to all builds:
usr/src/MakefileThis is the top-level makefile. It drives builds for various targets in each
subdirectory. It is aware of the specific targets that need to be built in each
subdirectory in order to perform a complete build, and itself knows how to
create a skeleton proto area for later use by install and install_h targets.
usr/src/Makefile.lintAll linting from the top level is driven by this makefile. It contains long lists of directories known to be lint-clean and contains simple recursive rules for rebuilding each subdirectory's lint target. The actual linting is driven by the lower-level makefiles.
usr/src/Makefile.master
usr/src/Makefile.master.64
These two makefiles contain generic definitions, such as build and installation
tools locations, template macros for compilers, linkers, and other tools to be
used by other makefiles in defining rules, and global definitions such as the
ISA and machine names that apply to this build. Makefile.master.64 contains
definitions specific to 64-bit builds that override the generic definitions.
usr/src/Makefile.msg.targCommon targets for building message catalogues are defined here. Message catalogues provide translations of messages for g11n purposes.
usr/src/Makefile.psmThis makefile defines the installation locations for platform-specific modules.
These are analogous to the other kernel module install locations /kernel and
/usr/kernel.
usr/src/Makefile.psm.targInstallation target definitions for platform-specific modules are defined here. This instructs the build system how to install files into the directories defined by Makefile.psm.
usr/src/TargetdirsThis is a set of definitions for the owner, group, and permissions of each directory that will be created by the installation process. It also contains information about special symbolic links to be installed for some 64-bit library versions.
The driving makefile for any module is located in the leaf directory (build
directory) where the module and its component objects are built. After a make
clobber operation, the makefile should be the only file remaining in that
directory. There are two other types of makefiles in the tree: suffixed and
non-suffixed. Common definitions and rules needed by all leaf makefiles are
contained in the suffixed makefiles; these are included by leaf makefiles.
Non-suffixed makefiles generally invoke multiple lower-level makefiles with the
same target so that many modules can be built with a single make invocation.
uts/Makefile
uts/sparc/Makefile
uts/sun4u/Makefile
uts/intel/Makefile
uts/intel/ia32/Makefile
uts/i86pc/Makefile
These makefiles generally are cognizant of the components made in subdirectories and invoke makefiles in those sub- directories to perform the actual build. Some targets (or pseudo-targets) may be directly built at this level (such as the cscope databases).
uts/Makefile.utsContains common definitions for all possible architectures.
uts/Makefile.targContains common targets for all possible architectures.
uts/common/Makefile.files
uts/sun/Makefile.files
uts/sparc/Makefile.files
uts/sun4/Makefile.files
uts/sun4u/Makefile.files
uts/intel/Makefile.files
uts/intel/ia32/Makefile.files
uts/i86pc/Makefile.files
These makefiles are divided into two sections. The first section defines the object lists which comprise each module. The second section defines the appropriate header search paths and other machine-specific global build parameters.
uts/common/Makefile.rules
uts/sun/Makefile.rules
uts/sparc/Makefile.rules
uts/sun4/Makefile.rules
uts/sun4u/Makefile.rules
uts/intel/Makefile.rules
uts/intel/ia32/Makefile.rules
uts/intel/amd64/Makefile.rules
uts/i86pc/Makefile.rules
The files provide build rules (targets) which allow make to function in a multiple directory environment. Each source tree below the directory containing the makefile has a build rule in the file.
uts/sun4/Makefile.sun4
uts/sun4u/Makefile.sun4u
uts/intel/Makefile.intel
*uts/intel/ia32/Makefile.ia32
uts/i86pc/Makefile.i86pc
These makefiles contain the definitions specific (defaults) to the obvious 'implementation architecture'. These rules can be overridden in specific leaf node makefiles if necessary.
uts/sun4u/unix/Makefile
uts/i86pc/unix/Makefile
Main driving makefile for building unix.
uts/sun4u/MODULE/Makefile (for MODULE in cgsix, cpu, kb, ...)Main driving makefile for building MODULE.
uts/sun4u/genunix/Makefile
uts/i86pc/genunix/Makefile
Main driving makefile for building genunix.
Issuing the command dmake in the uts directory will cause all supported,
modularized kernels and modules to be built.
Issuing the command dmake in a uts/ARCHITECTURE directory (i.e.:
uts/sparc) will cause all supported, "implementation architecture"-independent
modules for ARCHITECTURE to be built.
Issuing the command dmake in a uts/MACHINE directory (i.e.: uts/sun4u)
will cause that kernel and all supported, "implementation architecture"-
dependent modules for MACHINE to be built.
The makefiles are verbosely commented. It is desired that they should stay this way.
Most command and daemon subdirectories follow one of two general layout rules,
depending on where object files will be located (see Parts of a
Command). For ISA-dependent
programs, the layout is similar to that used by the kernel. Programs which do
not need ISA-dependent build behavior use a simplified makefile layout. In the
description here, we use the example of a generic command called foocmd whose
sources are located in usr/src/cmd/foocmd. The makefiles relevant to building
foocmd are:
usr/src/cmd/MakefileTop-level driving makefile for all commands/daemons. This is a simple recursive makefile which is aware of which subdirectories should be built and will cause the given target to be rebuilt in each of them.
usr/src/cmd/Makefile.cmdThis makefile defines the installation directories and rules for installing executables into them.
usr/src/cmd/Makefile.cmd.64Additional definitions specific to 64-bit builds are provided here.
usr/src/cmd/Makefile.cmd.bsmThis specialty makefile is used only by auditstat and dminfo. It provides some
generic boilerplate rules.
usr/src/cmd/Makefile.targBasic target definitions for clobber, lint, and installation.
usr/src/cmd/foocmd/MakefileDriving makefile for foocmd. Normally defines PROG but otherwise contains only
boilerplate definitions and targets. This is almost always copied from another
similar makefile. If foocmd does not require ISA-dependent build behavior, rules
will normally be specified directly, including those for the install target. If
foocmd does require ISA-dependent build behavior, this makefile will instead
define SUBDIRS to include the ISA-specific versions that must be built, and
define targets recursively. This will usually leave the install target
definition for each ISA makefile and cause a link to $(ISAEXEC) to be created.
See section Handling Multiple
Architectures for more
information on platform dependencies and $(ISAEXEC).
usr/src/cmd/foocmd/Makefile.comDefines PROG, OBJS, SRCS, and includes Makefile.cmd. May also contain
additional flags for compilation or linking. This makefile normally defines
targets for all, $(PROG), clean, and lint; this portion is usually
generic and would be copied from another similar makefile.
usr/src/cmd/foocmd/*/MakefileISA-specific makefiles, which may define additional ISA-specific flags or targets, and will generally include its own install target to install the ISA-specific program(s) it builds.
Most library subdirectories follow the same general layout, which is similar to the command layout. Unlike commands, most libraries are built for both 32- and 64-bit architecture variants, so ISA-specific directories will almost always be present. Therefore, the overall build structure for each library is similar to that of the kernel. We'll give an overview of the general makefiles for libraries as well as briefly explain the makefiles for a specific library. Later on in Component Anatomy, Creation, and Modification we'll go into more detail and show examples of building your own library and other components involved.
lib/Makefile.libThis contains the bulk of the macros for building shared objects.
lib/Makefile.lib.64This contains macros for building 64-bit objects, and should be included in Makefiles for 64-bit native ISAs.
lib/Makefile.rootfsThis contains macro overrides for libraries that install into /lib (rather than
/usr/lib).
lib/Makefile.targThis contains rules for building shared objects.
lib/<library>/MakefileThis is the library's top-level Makefile. It should contain rules for building any ISA-independent targets, such as installing header files and building message catalogs, but should defer all other targets to ISA-specific Makefiles.
lib/<library>/Makefile.comThis is the library's common Makefile. It should contain rules and macros which are common to all ISAs. This Makefile should never be built explicitly, but instead should be included (using the make include mechanism) by all of your ISA-specific Makefiles.
lib/<library>/<isa>/MakefileThese are the library's ISA-specific Makefiles, one per ISA (usually sparc and
i386, and sometimes sparcv9 and amd64). These Makefiles should include your
common Makefile and then provide any needed ISA-specific rules and definitions,
perhaps overriding those provided in your common Makefile.
We've done a basic tour of what source code is located where inside of the gate as well as covered how we deal with object files. We've introduced our Makefile system and covered the basic layout for these components. In the next section we'll go through and detail all of the components of commands, libraries, and kernel modules, as well as go through examples of adding new ones.