Device Access: Programmed I/O

illumos provides driver developers with a comprehensive set of interfaces for accessing device memory. These interfaces are designed to shield the driver from platform-specific dependencies by handling mismatches between processor and device endianness as well as enforcing any data order dependencies the device might have. By using these interfaces, you can develop a single-source driver that runs on both the SPARC and x86 processor architectures as well as the various platforms from each respective processor family.

This chapter provides information on the following subjects:

7.1. Device Memory

Devices that support programmed I/O are assigned one or more regions of bus address space that map to addressable regions of the device. These mappings are described as pairs of values in the reg property associated with the device. Each value pair describes a segment of a bus address.

Drivers identify a particular bus address mapping by specifying the register number, or regspec, which is an index into the devices' reg property. The reg property identifies the busaddr and size for the device. Drivers pass the register number when making calls to DDI functions such as ddi_regs_map_setup(9F). Drivers can determine how many mappable regions have been assigned to the device by calling ddi_dev_nregs(9F).

7.1.1. Managing Differences in Device and Host Endianness

The data format of the host can have different endian characteristics than the data format of the device. In such a case, data transferred between the host and device would need to be byte-swapped to conform to the data format requirements of the destination location. Devices with the same endian characteristics of the host require no byte-swapping of the data.

Drivers specify the endian characteristics of the device by setting the appropriate flag in the ddi_device_acc_attr(9S) structure that is passed to ddi_regs_map_setup(9F). The DDI framework then performs any required byte-swapping when the driver calls a ddi_getX routine like ddi_get8(9F) or a ddi_putX routine like ddi_put16(9F) to read or write to device memory.

7.1.2. Managing Data Ordering Requirements

Platforms can reorder loads and stores of data to optimize performance of the platform. Because reordering might not be allowed by certain devices, the driver is required to specify the device's ordering requirements when setting up mappings to the device.

7.1.3. ddi_device_acc_attr Structure

This structure describes the endian and data order requirements of the device. The driver is required to initialize and pass this structure as an argument to ddi_regs_map_setup(9F). 

typedef struct ddi_device_acc_attr {
    ushort_t    devacc_attr_version;
    uchar_t     devacc_attr_endian_flags;
    uchar_t     devacc_attr_dataorder;
} ddi_device_acc_attr_t;
devacc_attr_version

Specifies DDI_DEVICE_ATTR_V0

devacc_attr_endian_flags

Describes the endian characteristics of the device. Specified as a bit value whose possible values are:

  • DDI_NEVERSWAP_ACC – Never swap data

  • DDI_STRUCTURE_BE_ACC – The device data format is big-endian

  • DDI_STRUCTURE_LE_ACC – The device data format is little-endian

devacc_attr_dataorder

Describes the order in which the CPU must reference data as required by the device. Specified as an enumerated value, where data access restrictions are ordered from most strict to least strict.

  • DDI_STRICTORDER_ACC – The host must issue the references in order, as specified by the programmer. This flag is the default behavior.

  • DDI_UNORDERED_OK_ACC – The host is allowed to reorder loads and stores to device memory.

  • DDI_MERGING_OK_ACC – The host is allowed to merge individual stores to consecutive locations. This setting also implies reordering.

  • DDI_LOADCACHING_OK_ACC – The host is allowed to read data from the device until a store occurs.

  • DDI_STORECACHING_OK_ACC – The host is allowed to cache data written to the device. The host can then defer writing the data to the device until a future time.

The system can access data more strictly than the driver specifies in devacc_attr_dataorder. The restriction to the host diminishes while moving from strict data ordering to cache storing in terms of data accesses by the driver.

7.1.4. Mapping Device Memory

Drivers typically map all regions of a device during attach(9E). The driver maps a region of device memory by calling ddi_regs_map_setup(9F), specifying the register number of the region to map, the device access attributes for the region, an offset, and size. The DDI framework sets up the mappings for the device region and returns an opaque handle to the driver. This data access handle is passed as an argument to the ddi_get8(9F) or ddi_put8(9F) family of routines when reading data from or writing data to that region of the device.

The driver verifies that the shape of the device mappings match what the driver is expecting by checking the number of mappings exported by the device. The driver calls ddi_dev_nregs(9F) and then verifies the size of each mapping by calling ddi_dev_regsize(9F).

7.1.5. Mapping Setup Example

The following simple example demonstrates the DDI data access interfaces. This driver is for a fictional little endian device that accepts one character at a time and generates an interrupt when ready for another character. This device implements two register sets: the first is an 8-bit CSR register, and the second is an 8-bit data register. 

#define CSR_REG 0
#define DATA_REG 1
/*
 * Initialize the device access attributes for the register
 * mapping
 */
dev_acc_attr.devacc_attr_version = DDI_DEVICE_ATTR_V0;
dev_acc_attr.devacc_attr_endian_flags = DDI_STRUCTURE_LE_ACC;
dev_acc_attr.devacc_attr_dataorder = DDI_STRICTORDER_ACC;
/*
 * Map in the csr register (register 0)
 */
if (ddi_regs_map_setup(dip, CSR_REG, (caddr_t *)&(pio_p->csr), 0,
  sizeof (Pio_csr), &dev_acc_attr, &pio_p->csr_handle) != DDI_SUCCESS) {
    mutex_destroy(&pio_p->mutex);
    ddi_soft_state_free(pio_softstate, instance);
    return (DDI_FAILURE);
}
/*
 * Map in the data register (register 1)
 */
if (ddi_regs_map_setup(dip, DATA_REG, (caddr_t *)&(pio_p->data), 0,
  sizeof (uchar_t), &dev_acc_attr, &pio_p->data_handle) \
  != DDI_SUCCESS) {
    mutex_destroy(&pio_p->mutex);
    ddi_regs_map_free(&pio_p->csr_handle);
    ddi_soft_state_free(pio_softstate, instance);
    return (DDI_FAILURE);
}
Mapping Setup

7.2. Device Access Functions

Drivers use the ddi_get8(9F) and ddi_put8(9F) family of routines in conjunction with the handle returned by ddi_regs_map_setup(9F) to transfer data to and from a device. The DDI framework automatically handles any byte-swapping that is required to meet the endian format for the host or device, and enforces any store-ordering constraints the device might have.

The DDI provides interfaces for transferring data in 8-bit, 16-bit, 32-bit, and 64-bit quantities, as well as interfaces for transferring multiple values repeatedly. See the man pages for the ddi_get8(9F), ddi_put8(9F), ddi_rep_get8(9F) and ddi_rep_put8(9F) families of routines for a complete listing and description of these interfaces.

The following example builds on Mapping Setup where the driver mapped the device's CSR and data registers. Here, the driver's write(9E) entry point, when called, writes a buffer of data to the device one byte at a time. 

static  int
pio_write(dev_t dev, struct uio *uiop, cred_t *credp)
{
    int  retval;
    int  error = OK;
    Pio *pio_p = ddi_get_soft_state(pio_softstate, getminor(dev));

    if (pio_p == NULL)
        return (ENXIO);
    mutex_enter(&pio_p->mutex);
    /*
     * enable interrupts from the device by setting the Interrupt
     * Enable bit in the devices CSR register
     */
    ddi_put8(pio_p->csr_handle, pio_p->csr,
      (ddi_get8(pio_p->csr_handle, pio_p->csr) | PIO_INTR_ENABLE));

    while (uiop->uio_resid > 0) {
    /*
     * This device issues an IDLE interrupt when it is ready
     * to accept a character; the interrupt can be cleared
     * by setting PIO_INTR_CLEAR.  The interrupt is reasserted
     * after the next character is written or the next time
     * PIO_INTR_ENABLE is toggled on.
     *
     * wait for interrupt (see pio_intr)
     */
        cv_wait(&pio_p->cv, &pio_p->mutex);

    /*
     * get a character from the user's write request
     * fail the write request if any errors are encountered
     */
        if ((retval = uwritec(uiop)) == -1) {
            error = retval;
            break;
        }

    /*
     * pass the character to the device by writing it to
     * the device's data register
     */
        ddi_put8(pio_p->data_handle, pio_p->data, (uchar_t)retval);
    }

    /*
     * disable interrupts by clearing the Interrupt Enable bit
     * in the CSR
     */
    ddi_put8(pio_p->csr_handle, pio_p->csr,
      (ddi_get8(pio_p->csr_handle, pio_p->csr) & ~PIO_INTR_ENABLE));

    mutex_exit(&pio_p->mutex);
    return (error);
}
Mapping Setup: Buffer

7.2.1. Alternate Device Access Interfaces

In addition to implementing all device accesses through the ddi_get8(9F) and ddi_put8(9F) families of interfaces, illumos provides interfaces that are specific to particular bus implementations. While these functions can be more efficient on some platforms, use of these routines can limit the ability of the driver to remain portable across different bus versions of the device.

Memory Space Access

With memory mapped access, device registers appear in memory address space. The ddi_getX family of routines and the ddi_putX family are available for use by drivers as an alternative to the standard device access interfaces.

I/O Space Access

With I/O space access, the device registers appear in I/O space, where each addressable element is called an I/O port. The ddi_io_get8(9F) and ddi_io_put8(9F) routines are available for use by drivers as an alternative to the standard device access interfaces.

PCI Configuration Space Access

To access PCI configuration space without using the normal device access interfaces, a driver is required to map PCI configuration space by calling pci_config_setup(9F) in place of ddi_regs_map_setup(9F). The driver can then call the pci_config_get8(9F) and pci_config_put8(9F) families of interfaces to access PCI configuration space.