Making a Device Driver 64-Bit Ready

This appendix provides information for device driver writers who are converting their device drivers to support the 64-bit kernel. It presents the differences between 32-bit and 64-bit device drivers and describes the steps to convert 32-bit device drivers to 64-bit. This information is specific to regular character and block device drivers only.

This appendix provides information on the following subjects:

Introduction to 64-Bit Driver Design

For drivers that only need support for the 32-bit kernel, existing 32-bit device drivers will continue to work without recompilation. However, most device drivers require some changes to run correctly in the 64-bit kernel, and all device drivers require recompilation to create a 64-bit driver module. The information in this appendix will help you to enable drivers for 32-bit and 64-bit environments to be generated from common source code, thus increasing code portability and reducing the maintenance effort.

Before starting to modify a device driver for the 64-bit environment, you should understand how the 32-bit environment differs from the 64-bit environment. In particular, you must be familiar with the C language data type models ILP32 and LP64. See the following table.

Comparison of ILP32 and LP64 Data Types
C Type	ILP32	LP64
`char`	8	8
`short`	16	16
`int`	32	32
`long`	32	64
`long long`	64	64
`float`	32	32
`double`	64	64
`long double`	96	128
`pointer`	32	64

The driver-specific issues due to the differences between ILP32 and LP64 are the subject of this appendix. More general topics are covered in the Solaris 64-bit Developer’s Guide.

In addition to general code cleanup to support the data model changes for LP64, driver writers have to provide support for both 32-bit and 64-bit applications.

The ioctl(9E), devmap(9E), and mmap(9E) entry points enable data structures to be shared directly between applications and device drivers. If those data structures change size between the 32-bit and 64-bit environments, then the entry points must be modified so that the driver can determine whether the data model of the application is the same as that of the kernel. When the data models differ, data structures can be adjusted. See I/O Control Support for 64-Bit Capable Device Drivers, 32-bit and 64-bit Data Structure Macros, and Associating Kernel Memory With User Mappings.

In many drivers, only a few ioctls need this kind of handling. The other ioctls should work without change as long as these ioctls pass data structures that do not change in size.

General Conversion Steps

The sections below provide information on converting drivers to run in a 64-bit environment. Driver writers might need to perform one or more of the following tasks:

Use fixed-width types for hardware registers.
Use fixed-width common access functions.
Check and extend use of derived types.
Check changed fields within DDI data structures.
Check changed arguments of DDI functions.
Modify the driver entry points that handle user data, where needed.
Check structures that use 64-bit long types on x86 platforms.

These steps are explained in detail below.

After each step is complete, fix all compiler warnings, and use lint to look for other problems. The SC5.0 (or newer) version of lint should be used with -Xarch=v9 and -errchk=longptr64 specified to find 64-bit problems. See the notes on using and interpreting the output of lint in the Solaris 64-bit Developer’s Guide.

Do not ignore compilation warnings during conversion for LP64. Warnings that were safe to ignore previously in the ILP32 environment might now indicate a more serious problem.

After all the steps are complete, compile and test the driver as both a 32-bit and 64-bit module.

Use Fixed-Width Types for Hardware Registers

Many device drivers that manipulate hardware devices use C data structures to describe the layout of the hardware. In the LP64 data model, data structures that use long or unsigned long to define hardware registers are almost certainly incorrect, because long is now a 64-bit quantity. Start by including <sys/inttypes.h>, and update this class of data structure to use int32_t or uint32_t instead of long for 32-bit device data. This approach preserves the binary layout of 32-bit data structures. For example, change:

struct device_regs {
    ulong_t        addr;
    uint_t         count;
};      /* Only works for ILP32 compilation */

to:

struct device_regs {
    uint32_t        addr;
    uint32_t        count;
};      /* Works for any data model */

Use Fixed-Width Common Access Functions

The illumos DDI allows device registers to be accessed by access functions for portability to multiple platforms. Previously, the DDI common access functions specified the size of data in terms of bytes, words, and so on. For example, ddi_getl(9F) is used to access 32-bit quantities. This function is not available in the 64-bit DDI environment, and has been replaced by versions of the function that specify the number of bits to be acted on.

These routines were added to the 32-bit kernel in the Solaris 2.6 operating environment, to enable their early adoption by driver writers. For example, to be portable to both 32-bit and 64-bit kernels, the driver must use ddi_get32(9F) to access 32-bit data rather than ddi_getl(9F).

All common access routines are replaced by their fixed-width equivalents. See the ddi_get8(9F), ddi_put8(9F), ddi_rep_get8(9F), and ddi_rep_put8(9F) man pages for details.

Check and Extend Use of Derived Types

System-derived types, such as size_t, should be used where possible so that the resulting variables make sense when passed between functions. The new derived types uintptr_t or intptr_t should be used as the integral type for pointers.

Fixed-width integer types are useful for representing explicit sizes of binary data structures or hardware registers, while fundamental C language data types, such as int, can still be used for loop counters or file descriptors.

Some system-derived types represent 32-bit quantities on a 32-bit system but represent 64-bit quantities on a 64-bit system. Derived types that change size in this way include: clock_t, daddr_t, dev_t, ino_t, intptr_t, off_t, size_t, ssize_t, time_t, uintptr_t, and timeout_id_t.

When designing drivers that use these derived types, pay particular attention to the use of these types, particularly if the drivers are assigning these values to variables of another derived type, such as a fixed-width type.

Check Changed Fields in DDI Data Structures

The data types of some of the fields within DDI data structures, such as buf(9S), have been changed. Drivers that use these data structures should make sure that these fields are being used appropriately. The data structures and the fields that were changed in a significant way are listed below.

buf Structure Changes

The fields listed below pertain to transfer size, which can now exceed more than 4 Gbytes.

size_t        b_bcount;          /* was type unsigned int */
size_t        b_resid;           /* was type unsigned int */
size_t        b_bufsize;         /* was type long */

ddi_dma_attr

The ddi_dma_attr(9S) structure defines attributes of the DMA engine and the device. Because these attributes specify register sizes, fixed-width data types have been used instead of fundamental types.

ddi_dma_cookie Structure Changes

uint32_t     dmac_address;    /* was type unsigned long */
size_t       dmac_size;       /* was type u_int */

The ddi_dma_cookie(9S) structure contains a 32-bit DMA address, so a fixed-width data type has been used to define the address. The size has been redefined as size_t.

csi_arq_status Structure Changes

uint_t    sts_rqpkt_state;         /* was type u_long */
uint_t    sts_rqpkt_statistics;    /* was type u_long */

These fields in the structure do not need to grow and have been redefined as 32-bit quantities.

scsi_pkt Structure Changes

uint_t      pkt_flags;           /* was type u_long */
int     pkt_time;        /* was type long */
ssize_t     pkt_resid;           /* was type long */
uint_t      pkt_state;           /* was type u_long */
uint_t      pkt_statistics;      /* was type u_long */

Because the pkt_flags, pkt_state, and pkt_statistics fields in the scsi_pkt(9S) structure do not need to grow, these fields have been redefined as 32-bit integers. The data transfer size pkt_resid field does grow and has been redefined as ssize_t.

Check Changed Arguments of DDI Functions

This section describes the DDI function argument data types that have been changed.

getrbuf Argument Changes

struct buf *getrbuf(int sleepflag);

In previous releases, sleepflag was defined as a type long.

drv_getparm Argument Changes

int drv_getparm(unsigned int parm, void *value_p);

In previous releases, value_p was defined as type unsigned long. In the 64-bit kernel, drv_getparm(9F) can fetch both 32-bit and 64-bit quantities. The interface does not define data types of these quantities, and simple programming errors can occur.

The following new routines offer a safer alternative:

clock_t       ddi_get_lbolt(void);
time_t        ddi_get_time(void);
cred_t        *ddi_get_cred(void);
pid_t         ddi_get_pid(void);

Driver writers are strongly urged to use these routines instead of drv_getparm(9F).

delay and timeout Argument Changes

void delay(clock_t ticks);
timeout_id_t timeout(void (*func)(void *), void *arg, clock_t ticks);

The ticks argument to the delay(9F) and timeout(9F) routines has been changed from long to clock_t.

rmallocmap and rmallocmap_wait Argument Changes

struct map *rmallocmap(size_t mapsize);
struct map *rmallocmap_wait(size_t mapsize);

The mapsize argument to the rmallocmap(9F) and rmallocmap_wait(9F) routines has been changed from ulong_t to size_t.

scsi_alloc_consistent_buf Argument Changes

struct buf *scsi_alloc_consistent_buf(struct scsi_address *ap,
    struct buf *bp, size_t datalen, uint_t bflags,
    int (*callback )(caddr_t), caddr_t arg);

In previous releases, datalen was defined as an int and bflags was defined as a ulong.

uiomove Argument Changes

int uiomove(caddr_t address, size_t nbytes,
    enum uio_rw rwflag, uio_t *uio_p);

The nbytes argument was defined as a type long, but because nbytes represents a size in bytes, size_t is more appropriate.

cv_timedwait and cv_timedwait_sig Argument Changes

int cv_timedwait(kcondvar_t *cvp, kmutex_t *mp, clock_t timeout);
int cv_timedwait_sig(kcondvar_t *cvp, kmutex_t *mp,    clock_t timeout);

In previous releases, the timeout argument to the cv_timedwait(9F) and cv_timedwait_sig(9F) routines was defined to be of type long. Because these routines represent time in ticks, clock_t is more appropriate.

ddi_device_copy Argument Changes

int ddi_device_copy(ddi_acc_handle_t src_handle,
    caddr_t src_addr, ssize_t src_advcnt,
    ddi_acc_handle_t dest_handle, caddr_t dest_addr,
    ssize_t dest_advcnt, size_t bytecount, uint_t dev_datasz);

The src_advcnt, dest_advcnt, dev_datasz arguments have changed type. These arguments were previously defined as long, long, and ulong_t respectively.

ddi_device_zero Argument Changes

int ddi_device_zero(ddi_acc_handle_t handle,
    caddr_t dev_addr, size_t bytecount, ssize_t dev_advcnt,
    uint_t dev_datasz):

In previous releases, dev_advcnt was defined as a type long and dev_datasz as a ulong_t.

ddi_dma_mem_alloc Argument Changes

int ddi_dma_mem_alloc(ddi_dma_handle_t handle,
    size_t length, ddi_device_acc_attr_t *accattrp,
    uint_t flags, int (*waitfp)(caddr_t), caddr_t arg,
    caddr_t *kaddrp, size_t *real_length,
    ddi_acc_handle_t *handlep);

In previous releases, length, flags, and real_length were defined with types uint_t, ulong_t, and uint_t *.

Modify Routines That Handle Data Sharing

If a device driver shares data structures that contain longs or pointers with a 32-bit application using ioctl(9E), devmap(9E), or mmap(9E), and the driver is recompiled for a 64-bit kernel, the binary layout of data structures will be incompatible. If a field is currently defined in terms of type long and 64-bit data items are not used, change the data structure to use data types that remain as 32-bit quantities (int and unsigned int). Otherwise, the driver needs to be aware of the different structure shapes for ILP32 and LP64 and determine whether a model mismatch between the application and the kernel has occurred.

To handle potential data model differences, the ioctl, devmap, and mmap driver entry points, which interact directly with user applications, need to be written to determine whether the argument came from an application using the same data model as the kernel.

Data Sharing in ioctl

To determine whether a model mismatch exists between the application and the driver, the driver uses the FMODELS mask to determine the model type from the ioctl mode argument. The following values are OR-ed into mode to identify the application data model:

FLP64 – Application uses the LP64 data model
FILP32 – Application uses the ILP32 data model

The code examples in I/O Control Support for 64-Bit Capable Device Drivers show how this situation can be handled using ddi_model_convert_from(9F).

Data Sharing in devmap

To enable a 64-bit driver and a 32-bit application to share memory, the binary layout generated by the 64-bit driver must be the same as the layout consumed by the 32-bit application. The mapped memory being exported to the application might need to contain data-model-dependent data structures.

Few memory-mapped devices face this problem because the device registers do not change size when the kernel data model changes. However, some pseudo-devices that export mappings to the user address space might want to export different data structures to ILP32 or LP64 applications. To determine whether a data model mismatch has occurred, devmap(9E) uses the model parameter to describe the data model expected by the application. The model parameter is set to one of the following values:

DDI_MODEL_ILP32 – The application uses the ILP32 data model
DDI_MODEL_LP64 – The application uses the LP64 data model

The model parameter can be passed untranslated to the ddi_model_convert_from(9F) routine or to STRUCT_INIT. See 32-bit and 64-bit Data Structure Macros.

Data Sharing in mmap

Because mmap(9E) does not have a parameter that can be used to pass data model information, the driver's mmap(9E) entry point can be written to use the new DDI function ddi_model_convert_from(9F). This function returns one of the following values to indicate the application's data type model:

DDI_MODEL_ILP32 – Application expects the ILP32 data model
DDI_MODEL_ILP64 – Application expects the LP64 data model
DDI_FAILURE – Function was not called from mmap(9E)

As with ioctl and devmap, the model bits can be passed to ddi_model_convert_from(9F) to determine whether data conversion is necessary, or the model can be handed to STRUCT_INIT.

Alternatively, migrate the device driver to support the devmap(9E) entry point.

Check Structures with 64-bit Long Data Types on x86-Based Platforms

You should carefully check structures that use 64-bit long types, such as uint64_t, on the x86 platforms. The alignment and size can differ between compilation in 32-bit mode versus a 64-bit mode. Consider the following example.

#include &lt;studio>
#include &ltsys>

struct myTestStructure {
        uint32_t        my1stInteger;
        uint64_t        my2ndInteger;
};

main()
{
        struct myTestStructure a;

        printf("sizeof myTestStructure is: %d\n", sizeof(a));
        printf("offset to my2ndInteger is: %d\n", (uintptr_t)&a.bar - (uintptr_t)&a);
}

On a 32-bit system, this example displays the following results:

sizeof myTestStructure is: 12
offset to my2ndInteger is: 4

Conversely, on a 64-bit system, this example displays the following results:

sizeof myTestStructure is: 16
offset to my2ndInteger is: 8

Thus, the 32-bit application and the 64-bit application view the structure differently. As a result, trying to make the same structure work in both a 32-bit and 64-bit environment can cause problems. This situation occurs often, particularly in situations where structures are passed into and out of the kernel through ioctl calls.

Well Known ioctl Interfaces

Many ioctl(9E) operations are common to a class of device drivers. For example, most disk drivers implement many of the dkio(7I) family of ioctls. Many of these interfaces copy in or copy out data structures from the kernel, and some of these data structures have changed size in the LP64 data model. The following section lists the ioctlsthat now require explicit conversion in 64-bit driver ioctl routines for the dkio, fdio(7I), fbio(7I), cdio(7I), and mtio(7I) families of ioctls.

`ioctl` command	Affected data structure	Reference
`DKIOCGAPART` `DKIOCSAPART`	`dk_map` `dk_allmap`	dkio(7I)
`DKIOGVTOC` `DKIOSVTOC`	`partition` `vtoc`	dkio(7I)
`FBIOPUTCMAP` `FBIOGETCMAP`	`fbcmap`	fbio(7I)
`FBIOPUTCMAPI` `FBIOGETCMAPI`	`fbcmap_i`	fbio(7I)
`FBIOGCURSOR` `FBIOSCURSOR`	`fbcursor`	fbio(7I)
`CDROMREADMODE1` `CDROMREADMODE2`	`cdrom_read`	cdio(7I)
`CDROMCDDA`	`cdrom_cdda`	cdio(7I)
`CDROMCDXA`	`cdrom_cdxa`	cdio(7I)
`CDROMSUBCODE`	`cdrom_subcode`	cdio(7I)
`FDIOCMD`	`fd_cmd`	fdio(7I)
`FDRAW`	`fd_raw`	fdio(7I)
`MTIOCTOP`	`mtop`	mtio(7I)
`MTIOCGET`	`mtget`	mtio(7I)
`MTIOCGETDRIVETYPE`	`mtdrivetype_request`	mtio(7I)
`USCSICMD`	`uscsi_cmd`	scsi_free_consistent_buf(9F)

Device Sizes

The nblocks property is exported by each slice of a block device driver. This property contains the number of 512-byte blocks that each slice of the device can support. The nblocks property is defined as a signed 32-bit quantity, which limits the maximum size of a slice to 1 Tbyte.

Disk devices that provide more than 1 Tbyte of storage per disk must define the Nblocks property, which should still contain the number of 512 byte blocks that the device can support. However, Nblocks is a signed 64-bit quantity, which removes any practical limit on disk space.

The nblocks property is now deprecated. All disk devices should provide the Nblocks property.