Overview of illumos Device Drivers

This chapter gives an overview of illumos device drivers. The chapter provides information on the following subjects:

• Device Driver Basics

• Device Driver Entry Points

• Considerations in Device Driver Design

1.1. Device Driver Basics

This section introduces you to device drivers and their entry points on the illumos platform.

1.1.1. What Is a Device Driver?

A device driver is a kernel module that is responsible for managing the low-level I/O operations of a hardware device. Device drivers are written with standard interfaces that the kernel can call to interface with a device. Device drivers can also be software-only, emulating a device that exists only in software, such as RAM disks, buses, and pseudo-terminals.

A device driver contains all the device-specific code necessary to communicate with a device. This code includes a standard set of interfaces to the rest of the system. This interface shields the kernel from device specifics just as the system call interface protects application programs from platform specifics. Application programs and the rest of the kernel need little, if any, device-specific code to address the device. In this way, device drivers make the system more portable and easier to maintain.

When illumos is initialized, devices identify themselves and are organized into the device tree, a hierarchy of devices. In effect, the device tree is a hardware model for the kernel. An individual device driver is represented as a node in the tree with no children. This type of node is referred to as a leaf driver. A driver that provides services to other drivers is called a bus nexus driver and is shown as a node with children. As part of the boot process, physical devices are mapped to drivers in the tree so that the drivers can be located when needed. For more information on how illumos accommodates devices, see illumos Kernel and Device Tree.

Device drivers are classified by how they handle I/O. Device drivers fall into three broad categories:

• Block device drivers – For cases where handling I/O data as asynchronous chunks is appropriate. Typically, block drivers are used to manage devices with physically addressable storage media, such as disks.

• Character device drivers – For devices that perform I/O on a continuous flow of bytes.

  A driver can be both block and character at the same time if you set up two different interfaces to the file system. See Devices as Special Files.

  Included in the character category are drivers that use the STREAMS model (see below), programmed I/O, direct memory access, SCSI buses, USB, and other network I/O.

• STREAMS device drivers – Subset of character drivers that uses the streamio(7I) set of routines for character I/O within the kernel.

1.1.2. What Is a Device Driver Entry Point?

An entry point is a function within a device driver that can be called by an external entity to get access to some driver functionality or to operate a device. Each device driver provides a standard set of functions as entry points. For the complete list of entry points for all driver types, see the Intro(9E) man page. The illumos kernel uses entry points for these general task areas:

• Loading and unloading the driver

• Autoconfiguring the device – Autoconfiguration is the process of loading a device driver's code and static data into memory so that the driver is registered with the system.

• Providing I/O services for the driver

Drivers for different types of devices have different sets of entry points according to the kinds of operations the devices perform. A driver for a memory-mapped character-oriented device, for example, supports a devmap(9E) entry point, while a block driver does not support this entry.

Use a prefix based on the name of your driver to give driver functions unique names. Typically, this prefix is the name of the driver, such as xx_open for the open(9E) routine of driver xx. See Use a Unique Prefix to Avoid Kernel Symbol Collisions for more information. In subsequent examples in this book, xx is used as the driver prefix.

1.2. Device Driver Entry Points

This section provides lists of entry points for the following categories:

• Entry Points Common to All Drivers

• Entry Points for Block Device Drivers

• Entry Points for Character Device Drivers

• Entry Points for STREAMS Device Drivers

• Entry Points for Memory Mapped Devices

• Entry Points for the Generic LAN Device (GLD) Driver

• Entry Points for SCSI HBA Drivers

• Entry Points for PC Card Drivers

1.2.1. Entry Points Common to All Drivers

Some operations can be performed by any type of driver, such as the functions that are required for module loading and for the required autoconfiguration entry points. This section discusses types of entry points that are common to all drivers. The common entry points are listed in Summary of Common Entry Points with links to man pages and other relevant discussions.

Device Access Entry Points

Drivers for character and block devices export the cb_ops(9S) structure, which defines the driver entry points for block device access and character device access. Both types of drivers are required to support the open(9E) and close(9E) entry points. Block drivers are required to support strategy(9E), while character drivers can choose to implement whatever mix of read(9E), write(9E), ioctl(9E), mmap(9E), or devmap(9E) entry points is appropriate for the type of device. Character drivers can also support a polling interface through chpoll(9E). Asynchronous I/O is supported through aread(9E) and awrite(9E) for block drivers and those drivers that can use both block and character file systems.

Loadable Module Entry Points

All drivers are required to implement the loadable module entry points _init(9E), _fini(9E), and _info(9E) to load, unload, and report information about the driver module.

Drivers should allocate and initialize any global resources in _init(9E). Drivers should release their resources in _fini(9E).

In illumos, only the loadable module routines must be visible outside the driver object module. Other routines can have the storage class static.

Autoconfiguration Entry Points

Drivers are required to implement the attach(9E), detach(9E), and getinfo(9E) entry points for device autoconfiguration. Drivers can also implement the optional entry point probe(9E) in cases where devices do not identify themselves during boot-up, such as SCSI target devices. See Driver Autoconfiguration for more information on these routines.

Kernel Statistics Entry Points

The illumos platform provides a rich set of interfaces to maintain and export kernel-level statistics, also known as kstats. Drivers are free to use these interfaces to export driver and device statistics that can be used by user applications to observe the internal state of the driver. Two entry points are provided for working with kernel statistics:

• ks_snapshot(9E) captures kstats at a specific time.

• ks_update(9E) can be used to update kstat data at will. ks_update is useful in situations where a device is set up to track kernel data but extracting that data is time-consuming.

For further information, see the kstat_create(9F) and kstat(9S) man pages. See also Kernel Statistics.

Power Management Entry Point

Drivers for hardware devices that provide Power Management functionality can support the optional power(9E) entry point. See Power Management for details about this entry point.

Summary of Common Entry Points

The following table lists entry points that can be used by all types of drivers.

1.2.2. Entry Points for Block Device Drivers

Devices that support a file system are known as block devices. Drivers written for these devices are known as block device drivers. Block device drivers take a file system request, in the form of a buf(9S) structure, and issue the I/O operations to the disk to transfer the specified block. The main interface to the file system is the strategy(9E) routine. See Drivers for Block Devices for more information.

A block device driver can also provide a character driver interface to enable utility programs to bypass the file system and to access the device directly. This device access is commonly referred to as the raw interface to a block device.

The following table lists additional entry points that can be used by block device drivers. See also Entry Points Common to All Drivers.

1.2.3. Entry Points for Character Device Drivers

Character device drivers normally perform I/O in a byte stream. Examples of devices that use character drivers include tape drives and serial ports. Character device drivers can also provide additional interfaces not present in block drivers, such as I/O control (ioctl) commands, memory mapping, and device polling. See Drivers for Character Devices for more information.

The main task of any device driver is to perform I/O, and many character device drivers do what is called byte-stream or character I/O. The driver transfers data to and from the device without using a specific device address. This type of transfer is in contrast to block device drivers, where part of the file system request identifies a specific location on the device.

The read(9E) and write(9E) entry points handle byte-stream I/O for standard character drivers. See I/O Request Handling for more information.

The following table lists additional entry points that can be used by character device drivers. For other entry points, see Entry Points Common to All Drivers.

1.2.4. Entry Points for STREAMS Device Drivers

STREAMS is a separate programming model for writing a character driver. Devices that receive data asynchronously, such as terminal and network devices, are suited to a STREAMS implementation. STREAMS device drivers must provide the loading and autoconfiguration support described in Driver Autoconfiguration. See the STREAMS Programming Guide for additional information on how to write STREAMS drivers.

The following table lists additional entry points that can be used by STREAMS device drivers. For other entry points, see Entry Points Common to All Drivers and Entry Points for Character Device Drivers.

1.2.5. Entry Points for Memory Mapped Devices

For certain devices, such as frame buffers, providing application programs with direct access to device memory is more efficient than byte-stream I/O. Applications can map device memory into their address spaces using the mmap(2) system call. To support memory mapping, device drivers implement segmap(9E) and devmap(9E) entry points. For information on devmap(9E), see Mapping Device and Kernel Memory. For information on segmap(9E), see Drivers for Character Devices.

Drivers that define the devmap(9E) entry point usually do not define read(9E) and write(9E) entry points, because application programs perform I/O directly to the devices after calling mmap(2).

The following table lists additional entry points that can be used by character device drivers that use the devmap framework to perform memory mapping. For other entry points, see Entry Points Common to All Drivers and Entry Points for Character Device Drivers.

1.2.6. Entry Points for the Generic LAN Device (GLD) Driver

The following table lists additional entry points that can be used by the general LAN driver (GLD). For more information on GLD drivers, see the gld(9E), gld(7D), and gld_mac_info(9S) man pages. For other entry points, see Entry Points Common to All Drivers and Entry Points for Character Device Drivers.

1.2.7. Entry Points for SCSI HBA Drivers

The following table lists additional entry points that can be used by SCSI HBA device drivers. For information on the SCSI HBA transport structure, see scsi_hba_tran(9S). For other entry points, see Entry Points Common to All Drivers and Entry Points for Character Device Drivers.

1.2.8. Entry Points for PC Card Drivers

The following table lists additional entry points that can be used by PC Card device drivers. For other entry points, see Entry Points Common to All Drivers and Entry Points for Character Device Drivers.

1.3. Considerations in Device Driver Design

A device driver must be compatible with illumos, both as a consumer and provider of services. This section discusses the following issues, which should be considered in device driver design:

• DDI/DKI Facilities

• Driver Context

• Returning Errors

• Dynamic Memory Allocation

• Hotplugging

1.3.1. DDI/DKI Facilities

The illumos DDI/DKI interfaces are provided for driver portability. With DDI/DKI, developers can write driver code in a standard fashion without having to worry about hardware or platform differences. This section describes aspects of the DDI/DKI interfaces.

Device IDs

The DDI interfaces enable drivers to provide a persistent, unique identifier for a device. The device ID can be used to identify or locate a device. The ID is independent of the device's name or number (dev_t). Applications can use the functions defined in libdevid(3LIB) to read and manipulate the device IDs registered by the drivers.

Device Properties

The attributes of a device or device driver are specified by properties. A property is a name-value pair. The name is a string that identifies the property with an associated value. Properties can be defined by the FCode of a self-identifying device, by a hardware configuration file (see the driver.conf(4) man page), or by the driver itself using the ddi_prop_update(9F) family of routines.

Interrupt Handling

The DDI/DKI addresses the following aspects of device interrupt handling:

• Registering device interrupts with the system

• Removing device interrupts

• Dispatching interrupts to interrupt handlers

Device interrupt sources are contained in a property called interrupt, which is either provided by the PROM of a self-identifying device, in a hardware configuration file, or by the booting system on the x86 platform.

Callback Functions

Certain DDI mechanisms provide a callback mechanism. DDI functions provide a mechanism for scheduling a callback when a condition is met. Callback functions can be used for the following typical conditions:

• A transfer has completed

• A resource has become available

• A time-out period has expired

Callback functions are somewhat similar to entry points, for example, interrupt handlers. DDI functions that allow callbacks expect the callback function to perform certain tasks. In the case of DMA routines, a callback function must return a value indicating whether the callback function needs to be rescheduled in case of a failure.

Callback functions execute as a separate interrupt thread. Callbacks must handle all the usual multithreading issues.

A driver must cancel all scheduled callback functions before detaching a device.

Software State Management

To assist device driver writers in allocating state structures, the DDI/DKI provides a set of memory management routines called the software state management routines, also known as the soft-state routines. These routines dynamically allocate, retrieve, and destroy memory items of a specified size, and hide the details of list management. An instance number is used to identify the desired memory item. This number is typically the instance number assigned by the system.

Routines are provided for the following tasks:

• Initialize a driver's soft-state list

• Allocate space for an instance of a driver's soft state

• Retrieve a pointer to an instance of a driver's soft state

• Free the memory for an instance of a driver's soft state

• Finish using a driver's soft-state list

See Loadable Driver Interfaces for an example of how to use these routines.

Programmed I/O Device Access

Programmed I/O device access is the act of reading and writing of device registers or device memory by the host CPU. The illumos DDI provides interfaces for mapping a device's registers or memory by the kernel as well as interfaces for reading and writing to device memory from the driver. These interfaces enable drivers to be developed that are platform and bus independent, by automatically managing any difference in device and host endianness as well as by enforcing any memory-store sequence requirements imposed by the device.

Direct Memory Access (DMA)

The illumos platform defines a high-level, architecture-independent model for supporting DMA-capable devices. The illumos DDI shields drivers from platform-specific details. This concept enables a common driver to run on multiple platforms and architectures.

Layered Driver Interfaces

The DDI/DKI provides a group of interfaces referred to as layered device interfaces (LDI). These interfaces enable a device to be accessed from within the illumos kernel. This capability enables developers to write applications that observe kernel device usage. For example, both the prtconf(1M) and fuser(1M) commands use LDI to enable system administrators to track aspects of device usage. The LDI is covered in more detail in Layered Driver Interface (LDI).

1.3.2. Driver Context

The driver context refers to the condition under which a driver is currently operating. The context limits the operations that a driver can perform. The driver context depends on the executing code that is invoked. Driver code executes in four contexts:

• User context. A driver entry point has user context when invoked by a user thread in a synchronous fashion. That is, the user thread waits for the system to return from the entry point that was invoked. For example, the read(9E) entry point of the driver has user context when invoked by a read(2) system call. In this case, the driver has access to the user area for copying data into and out of the user thread.

• Kernel context. A driver function has kernel context when invoked by some part of the kernel. In a block device driver, the strategy(9E) entry point can be called by the pageout daemon to write pages to the device. Because the page daemon has no relation to the current user thread, strategy(9E) has kernel context in this case.

• Interrupt context. Interrupt context is a more restrictive form of kernel context. Interrupt context is invoked as a result of the servicing of an interrupt. Driver interrupt routines operate in interrupt context with an associated interrupt level. Callback routines also operate in an interrupt context. See Interrupt Handlers for more information.

• High-level interrupt context. High-level interrupt context is a more restricted form of interrupt context. If ddi_intr_hilevel(9F) indicates that an interrupt is high level, the driver interrupt handler runs in high-level interrupt context. See Interrupt Handlers for more information.

The manual pages in section 9F document the allowable contexts for each function. For example, in kernel context the driver must not call copyin(9F).

1.3.3. Returning Errors

Device drivers do not usually print messages, except for unexpected errors such as data corruption. Instead, the driver entry points should return error codes so that the application can determine how to handle the error. Use the cmn_err(9F) function to write messages to a system log that can then be displayed on the console.

The format string specifier interpreted by cmn_err(9F) is similar to the printf(3C) format string specifier, with the addition of the format %b, which prints bit fields. The first character of the format string can have a special meaning. Calls to cmn_err(9F) also specify the message level, which indicates the severity label to be printed. See the cmn_err(9F) man page for more details.

The level CE_PANIC has the side effect of crashing the system. This level should be used only if the system is in such an unstable state that to continue would cause more problems. The level can also be used to get a system core dump when debugging. CE_PANIC should not be used in production device drivers.

1.3.4. Dynamic Memory Allocation

Device drivers must be prepared to simultaneously handle all attached devices that the drivers claim to drive. The number of devices that the driver handles should not be limited. All per-device information must be dynamically allocated.

void *kmem_alloc(size_t size, int flag);

The standard kernel memory allocation routine is kmem_alloc(9F). kmem_alloc is similar to the C library routine malloc(3C), with the addition of the flag argument. The flag argument can be either KM_SLEEP or KM_NOSLEEP, indicating whether the caller is willing to block if the requested size is not available. If KM_NOSLEEP is set and memory is not available, kmem_alloc(9F) returns NULL.

kmem_zalloc(9F) is similar to kmem_alloc(9F), but also clears the contents of the allocated memory.

Kernel memory is a limited resource, not pageable, and competes with user applications and the rest of the kernel for physical memory. Drivers that allocate a large amount of kernel memory can cause system performance to degrade.

void kmem_free(void *cp, size_t size);

Memory allocated by kmem_alloc(9F) or by kmem_zalloc(9F) is returned to the system with kmem_free(9F). kmem_free is similar to the C library routine free(3C), with the addition of the size argument. Drivers must keep track of the size of each allocated object in order to call kmem_free(9F) later.

1.3.5. Hotplugging

This manual does not highlight hotplugging information. If you follow the rules and suggestions for writing device drivers given in this book, your driver should be able to handle hotplugging. In particular, make sure that both autoconfiguration (see Driver Autoconfiguration) and detach(9E) work correctly in your driver. In addition, if you are designing a driver that uses power management, you should follow the information given in Power Management. SCSI HBA drivers might need to add a cb_ops structure to their dev_ops structure (see SCSI Host Bus Adapter Drivers) to take advantage of hotplugging capabilities.

Previous versions of the Solaris OS required hotpluggable drivers to include a DT_HOTPLUG property, but this property is no longer required. Driver writers are free, however, to include and use the DT_HOTPLUG property as they see fit.