MPO Observability Tools

This chapter describes the tools that are available to use the MPO functionality that is available in the illumos operating system.

This chapter discusses the following topics:

2.1. The pmadvise utility

The pmadvise utility applies rules to a process that define how that process uses memory. The pmadvise utility applies the rules, called advice, to the process with the madvise(3C) tool. This tool can apply advice to a specific subrange of locations in memory at a specific time. By contrast, the madv.so.1(1) tool applies the advice throughout the execution of the target program to all segments of a specified type.

The pmadvise utility has the following options:

-f

This option takes control of the target process. This option overrides the control of any other process. See the proc(1) manual page.

-o

This option specifies the advice to apply to the target process. Specify the advice in this format: 

private=advice
shared=advice
heap=advice
stack=advice
address:length=advice

The value of the advice term can be one of the following: 

normal
random
sequential
willneed
dontneed
free
access_lwp
access_many
access_default

You can specify an address and length to specify the subrange where the advice applies. Specify the address in hexadecimal notation and the length in bytes.

If you do not specify the length and the starting address refers to the start of a segment, the pmadvise utility applies the advice to that segment. You can qualify the length by adding the letters K, M, G, T, P, or E to specify kilobytes, megabytes, gigabytes, terabytes, or exabytes, respectively.

-v

This option prints verbose output in the style of the pmap(1) tool that shows the value and locations of the advice rules currently in force.

The pmadvise tool attempts to process all legal options. When the pmadvise tool attempts to process an option that specifies an illegal address range, the tool prints an error message and skips that option. When the pmadvise tool finds a syntax error, it quits without processing any options and prints a usage message.

When the advice for a specific region conflicts with the advice for a more general region, the advice for the more specific region takes precedence. Advice that specifies a particular address range has precedence over advice for the heap and stack regions, and advice for the heap and stack regions has precedence over advice for private and shared memory.

The advice rules in each of the following groups are mutually exclusive from other advice rules within the same group: 

MADV_NORMAL, MADV_RANDOM, MADV_SEQUENTIAL
MADV_WILLNEED, MADV_DONTNEED, MADV_FREE   
MADV_ACCESS_DEFAULT, MADV_ACCESS_LWP, MADV_ACCESS_MANY

2.2. The plgrp tool

The plgrp utility can display or set the home lgroup and lgroup affinities for one or more processes, threads, or lightweight processes (LWPs). The system assigns a home lgroup to each thread on creation. When the system allocates a CPU or memory resource to a thread, it searches the lgroup hierarchy from the thread's home lgroup for the nearest available resources to the thread's home.

The system chooses a home lgroup for each thread. The thread's affinity for its home lgroup is initially set to none, or no affinity. When a thread sets an affinity for an lgroup in its processor set that is higher than the thread's affinity for its home lgroup, the system moves the thread to that lgroup. The system does not move threads that are bound to a CPU. The system rehomes a thread to the lgroup in its processor set that has the highest affinity when the thread's affinity for its home lgroup is removed (set to none).

For a full description of the different levels of lgroup affinity and their semantics, see the lgrp_affinity_set(3LGRP) manual page.

The plgrp tool supports the following options:

-a lgroup list

This option displays the affinities of the processes or threads that you specify for the lgroups in the list.

-Algroup list/none|weak|strong[,...]

This option sets the affinity of the processes or threads that you specify for the lgroups in the list. You can use a comma separated list of lgroup/affinity assignments to set several affinities at once.

-F

This option takes control of the target process. This option overrides the control of any other process. See the proc(1) manual page.

-h

This option returns the home lgroup of the processes or threads that you specify. This is the default behavior of the plgrp tool when you do not specify any options.

-H lgroup list

This option sets the home lgroup of the processes or threads that you specify. This option sets a strong affinity for the listed lgroup. If you specify more than one lgroup, the plgrp utility will attempt to home the threads to the lgroups in a round robin fashion.

2.2.1. Specifying Lgroups

The value of the lgroup list variable is a comma separated list of one or more of the following attributes:

The all keyword represents all of the lgroup IDs in the system. The root keyword represents the ID of the root lgroup. The leaves keyword represents the IDs of all of the leaf lgroups. A leaf lgroup is an lgroup that does not have any children.

2.2.2. Specifying Process and Thread Arguments

The plgrp utility takes one or more space-separated processes or threads as arguments. You can specify processes and threads in a the same syntax that the proc(1) tools use. You can specify a process ID as an integer, with the syntax pid or /proc/pid. You can use shell expansions with the /proc/pid syntax. When you give a process ID alone, the arguments to the plgrp utility include all of the threads of that process.

You can specify a thread explicitly by specifying the process ID and thread ID with the syntax pid/lwpid. You can specify multiple threads of a process by defining ranges with can be selected at once by using the - character to define a range, or with a comma-separated list. To specify threads 1, 2, 7, 8, and 9 of a process whose process ID is pid, use the syntax pid/1,2,7-9.

2.3. The lgrpinfo Tool

The lgrpinfo tool prints information about the lgroup hierarchy, contents, and characteristics. The lgrpinfo tool is a Perl script that requires the Solaris::Lgrp module. This tool uses the liblgrp(3LIB) API to get the information from the system and displays it in the human-readable form.

The lgrpinfo tool prints general information about all of the lgroups in the system when you call it without any arguments. When you pass lgroup IDs to the lgrpinfo tool at the command line, the tool returns information about the lgroups that you specify. You can specify lgroups with their lgroup IDs or with one of the following keywords:

all

This keyword specifies all lgroups and is the default behavior.

root

This keyword specifies the root lgroup.

leaves

This keyword specifies all of the leaf lgroups. A leaf lgroup is an lgroup that has no children in the lgroup hierarchy.

intermediate

This keyword specifies all of the intermediate lgroups. An intermediate lgroup is an lgroup that has a parent and children.

When the lgrpinfo tool receives an invalid lgroup ID, the tool prints a message with the invalid ID and continues processing any other lgroups that are passed in the command line. When the lgrpinfo tool finds no valid lgroups in the arguments, it exits with a status of 2.

2.3.1. Options for the lgrpinfo Tool

When you call the lgrpinfo tool without any arguments, the tool's behavior is equivalent to using the options -celmrt all. The valid options for the lgrpinfo tool are:

-a

This option prints the topology, CPU, memory, load and latency information for the specified lgroup IDs. This option combines the behaviors of the -tcemrlL options, unless you also specify the -T option. When you specify the -T option, the behavior of the -a option does not include the behavior of the -t option.

-c

This option prints the CPU information.

-C

This option replaces each lgroup in the list with its children. You cannot combine this option with the -P or -T options. When you do not specify any arguments, the tool applies this option to all lgroups.

-e

This option prints lgroup load averages for leaf lgroups.

-G

This option prints the OS view of the lgroup hierarchy. The tool's default behavior displays the caller's view of the lgroup hierarchy. The caller's view only includes the resources that the caller can use. See the lgrp_init(3LGRP) manual page for more details on the OS and caller's view.

-h

This option prints the help message for the tool.

-I

This option prints only IDs that match the IDs you specify. You can combine this option with the -c, -G, -C, or -P options. When you specify the -c option, the tool prints the list of CPUs that are in all of the matching lgroups. When you do not specify the -c option, the tool displays the IDs for the matching lgroups. When you do not specify any arguments, the tool applies this option to all lgroups.

-l

This option prints information about lgroup latencies. The latency value given for each lgroup is defined by the operating system and is platform-specific. It can only be used for relative comparison of lgroups on the running system. It does not necessarily represent the actual latency between hardware devices and may not be applicable across platforms.

-L

This option prints the lgroup latency table. This table shows the relative latency from each lgroup to each of the other lgroups.

-m

This option prints memory information. The tool reports memory sizes in the units that give a size result in the integer range from 0 to 1023. You can override this behavior by using the -u option. The tool will only display fractional results for values smaller than 10.

-P

This option replaces each lgroup in the list with its parent or parents. You cannot combine this option with the -C or -T options. When you do not specify any arguments, the tool applies this option to all lgroups.

-r

This option prints information about lgroup resources. When you specify the -T option, the tool displays information about the resources of the intermediate lgroups only.

-t

This option prints information about lgroup topology.

-T

This option prints the lgroup topology of a system graphically, as a tree. You can only use this option with the -a, -c, -e, -G, -l, -L, -m, -r, and -u options. To restrict the output to intermediate lgroups, use the -r option. Omit the -t option when you combine the -T option with the -a option. This option does not print information for the root lgroup unless it is the only lgroup.

-uunits

This option specifies memory units. The value of the units argument can be b, k, m, g, t, p, or e for bytes, kilobytes, megabytes, gigabytes, terabytes, petabytes, or exabytes, respectively.

2.4. The Solaris::lgrp Module

This Perl module provides a Perl interface to the lgroup APIs that are in liblgrp. This interface provides a way to traverse the lgroup hierarchy, discover its contents and characteristics, and set a thread's affinity for an lgroup. The module gives access to various constants and functions defined in the lgrp_user.h header file. The module provides the procedural interface and the object interface to the library.

The default behavior of this module does not export anything. You can use the following tags to selectively import the constants and functions that are defined in this module:

:LGRP_CONSTANTS

LGRP_AFF_NONE, LGRP_AFF_STRONG, LGRP_AFF_WEAK, LGRP_CONTENT_DIRECT, LGRP_CONTENT_HIERARCHY, LGRP_MEM_SZ_FREE, LGRP_MEM_SZ_INSTALLED, LGRP_VER_CURRENT, LGRP_VER_NONE, LGRP_VIEW_CALLER, LGRP_VIEW_OS, LGRP_NONE, LGRP_RSRC_CPU, LGRP_RSRC_MEM, LGRP_CONTENT_ALL, LGRP_LAT_CPU_TO_MEM

:PROC_CONSTANTS

P_PID, P_LWPID, P_MYID

:CONSTANTS

:LGRP_CONSTANTS, :PROC_CONSTANTS

:FUNCTIONS

lgrp_affinity_get, lgrp_affinity_set, lgrp_children, lgrp_cookie_stale, lgrp_cpus, lgrp_fini, lgrp_home, lgrp_init, lgrp_latency, lgrp_latency_cookie, lgrp_mem_size, lgrp_nlgrps, lgrp_parents, lgrp_root, lgrp_version, lgrp_view, lgrp_resources, lgrp_lgrps, lgrp_leaves, lgrp_isleaf, lgrp_lgrps, lgrp_leaves.

:ALL

:CONSTANTS, :FUNCTIONS

The Perl module has the following methods:

You can export constants with the :CONSTANTS or :ALL tags. You can use any of the constants in the following list in Perl programs.

When an underlying library function fails, the functions in this module return either undef or an empty list. The module can use the following error codes:

EINVAL

The value supplied is not valid.

ENOMEM

There was not enough system memory to complete an operation.

ESRCH

The specified process or thread was not found.

EPERM

The effective user of the calling process does not have the appropriate privileges, and its real or effective user ID does not match the real or effective user ID of one of the threads.