Introduction
Testing entry and wakeup from low-power states is crucial when developing a SuperSpeed USB device. However forcing a device into these states in a controlled manner can be somewhat tricky, as the operating system is designed to do so automatically, based upon a variety of parameters and system settings. On this page, the methods for testing low-power states on Linux are discussed, with the assumption that no driver has been loaded on behalf of the device under test.
More information on low-power states can be found on this page, as well a separate page discussing the invocation into a low-power state and a page on exiting from such.
There are several players that may affect the behavior of the power management subsystem of a device. For example, a Linux driver may have the disable_hub_initiated_lpm flag set in its driver declaration (the usb_driver struct), which makes the host refrain from initiating U1 or U2 states. Likewise, the host puts a device in U3 (suspend) after a period of inactivity only if a driver for that device is loaded, and that driver has set the supports_autosuspend flag in its driver declaration. And even then, odds are that autosuspend is turned off globally, leaving the U3 exclusively for the system suspend state, because many devices don’t wake up properly from the U3 state.
Only host-initiated low-power state transition is discussed here: Even though the USB spec allows the device to request transition into U1 or U2, it’s assumed (and to some extent taken care of) that the device never does that.
In real life, the most common setting for an operational device is to enable U1 with automatic transition into U2, and that U3 is avoided, except for if the host is put in suspended mode. Staying in U1 for long periods (i.e. the automatic transition to U2 is disabled) is probably a rare choice, judging by poor behavior observed in this mode by mainstream hardware (LFPS.Ping detection in particular).
Testing the U3 state (device suspend)
The natural way for putting a device in U3 is to make Linux autosuspend due to inactivity. The time required for this timeout to occur is easily controlled (shown below).
However as of Linux kernel 4.2 (and most likely further on), the OS is reluctant to put a device in the U3 state unless the host itself is put in suspend mode. Autosuspend is often disabled globally by default.
Also, if a driver is loaded on behalf of a USB device, autosuspend will not kick in unless the driver has declared explicitly that it supports autosuspend (with a “supports_autosuspend” entry in its usb_driver definition structure). This requires the device to maintain a reference count for the device, with calls to kernel API functions such as usb_autopm_get_interface(), in order to prevent autosuspend when unsuitable. Most drivers simply don’t bother.
So bottom line: If a driver is loaded, autosuspend is most likely inactive.
Turning on autosuspend
Change directory:
$ cd /sys/bus/usb/devices
There are a lot of symbolic links to directories there, one for each USB device (including hubs). It may be difficult to tell which belongs to which device. One way is to tell by the messages in the kernel log as the device is plugged in, for example,
usb 2-5: new SuperSpeed USB device number 105 using xhci_hcd
Alternatively, scan the directories: Possibly without root privileges (and from the /sys/bus/usb/devices directory), go
$ for i in * ; do [ -e $i/idVendor ] && echo $(cat $i/{busnum,devnum,idVendor,idProduct}) ' ===> ' $i ; done
This prints out the bus number, bus address, vendor and product IDs of each device on the bus, along with the name of the symbolic link in /sys/bus/usb/devices for the relevant device. Comparing with the output of “lsusb”, it’s easy to spot the directory to select.
Change to the directory of the relevant device, e.g. to /sys/bus/usb/devices/2-5.
Enable autosuspend: As root, go
# echo auto > power/control
Disable autosuspend (wakes up the device immediately if it was in U3):
# echo on > power/control
If the device was idling prior to enabling autosuspend, it goes into U3 immediately. These two commands effectively move the device in and out of suspend mode for an idling device.
There is however no way to force a device into suspend directly. “auto” and “on” are the only two options for power/control.
The timeout for autosuspend is given in power/autosuspend_delay_ms:
$ cat power/autosuspend_delay_ms 2000
For example, to change it to 200 ms (requires root):
# echo 200 > power/autosuspend_delay_ms
Important: As said above, if a driver is loaded on behalf of a USB device, autosuspend will most likely not kick in regardless of the manipulations of power/control.
To check whether the device is in U3 state,
# cat power/runtime_status suspended
If the device isn’t suspended (i.e. in states U0, U1 or U2), the answer will be “active”. Note that even though U1 and U2 are low-power modes, these are not reflected in runtime_status (this has been verified with hardware that reported its power state).
Controlling entry into U1 and U2
Host-initiated invocation into U1 and U2 is done directly by the USB hub to which the device is connected, in response to lack of communication on the link to that device. Note that an xHCI USB controller is a root hub, and is treated as any USB hub here.
As the timeouts for invoking the U1 and U2 states are measured in microseconds, and waking up from them quickly is crucial for efficient communication, the software doesn’t control these transitions directly (neither kernel nor user space). Rather, the driver of the hub enables or disables such initiations into U1 and/or U2 by the hub, and sets the timeout periods for invoking each. This boils down to setting the PORT_U1_TIMEOUT and PORT_U2_TIMEOUT attributes for each port of the hub. Sections 10.4.2.1 and 10.14.2.10 in the USB 3.0 spec detail the meaning of their values, and this page also discusses these.
The goal is hence to set these two attributes for the device under test, controlling if and when invocations into U1 and U2 take place. However during normal system operation, it’s completely down to the hub’s Linux driver to decide how to set up these two attributes, depending on several parameters. As of Linux 4.2, there is no exposed interface for setting them directly.
To overcome this, the source of a Linux kernel module (”timeouter”), which is intended for testing, is listed at the end of this page. It presents itself as the driver for the device to test by matching its Vendor / Product IDs, and configures these two attributes for this device.
This is however slightly trickier than so: These two attributes are set on the hub to which the device is connected, not on the device itself. The way it’s normally done is by sending two SET_FEATURE SETUP commands to the hub to which the device is connected, one for each of these two attributes.
Accordingly, the kernel module, which is loaded on behalf of the device to test, sends SETUP commands to the hub, setting the values of PORT_U1_TIMEOUT and PORT_U2_TIMEOUT for the port that the device is connected to. Practically speaking, this is easier than it might sound, as the Linux kernel API is kind enough to supply the handle to the hub (”parent”) of the device, as well as its port number. But surely it was nobody’s intention that the driver of a device would send commands to the hub this way, so this is definitely not a technique recommended for a production system. This is also the reason for a bit of Tom & Jerry played with the hub’s driver.
The textbook way to do this would have been to change the hub’s driver, adding sysfs entries for these two attributes. This requires editing code that goes into the kernel image, and then compiling the entire kernel. Inserting a kernel module is so much easier, and it’s stable enough for testing purposes.
And once again, the root hub is a USB hub for all purposes. The kernel module correctly alters these two attributes whether the device is connected directly to one of the motherboard’s USB ports or through a series of USB hubs.
Principle of operation
The timeouter kernel module, which is listed below, is loaded with two parameters, e.g.
# insmod timeouter.ko u1_timeout=120 u2_timeout=15
These are the values to assigned to PORT_U1_TIMEOUT and PORT_U2_TIMEOUT. Once again, refer to this page for a brief definition of these two.
As with any Linux USB driver, the Vendor ID and Product ID is given in the timeouter_id_table, which is declared near the end of the code. These are currently set to 0x1234 and 0x5678 respectively, and must be changed to the IDs of the USB device to test prior to compiling the module.
Important: Only one device matching the IDs is allowed on the host. The kernel module uses global variables for simplicity, and will therefore fail colossally if it’s assigned to more than one USB device.
When loaded into the kernel with insmod, the module’s timeouter_probe() function is called if there’s a matching USB device attached to the host. Same thing happens if the module is already loaded, and such device is plugged in. This function merely queues another function, timeout_manipulate(), for execution 200 ms later as a kernel worker thread.
timeout_manipulate() executes the following:
- Prints two messages to the kernel log (use e.g. “dmesg” to see those), supplying identification of the device targeted as well as the hub to which the timeout setting commands will be sent. There is no reason this should turn out wrong, but it may help in case of doubt.
- Disables the U1_ENABLE and U2_ENABLE features of the targeted device. This tells the device not to initiate transitions to any low-power state. Note that a malfunctioning device could ignore this setting, and possibly initiate a low-power state successfully despite this setting. As these feature disable commands are sent to the device and not to the hub, the hub has no particular reason to refuse such illegal initiation. Still, a device needs to be very poorly designed to ignore this.
- Set the PORT_U1_TIMEOUT and PORT_U2_TIMEOUT parameters for the hub’s port, to which the device is connected. As mentioned above, this is a slight abuse of the parent hub’s handle in the kernel API, as each driver is supposed to deal with the device it’s assigned to, and not with others.
The reason for the 200 ms delay between the call to timeouter_probe() and the execution of these tasks, is that the hub’s driver typically sets the timeout parameters immediately after timeouter_probe() finishes. This 200 ms delay puts the actions of timeout_manipulate() after this, so the user-requested timeout settings become the final word.
Monitoring transitions into U1/U2
One possibility for monitoring the port’s state (not chosen here) is by virtue of a Get Port Status command to the hub (see section 10.14.2.6 of the USB 3.0 spec), which supplies the state of the link among others. However this gives only the hub’s side of the picture, and supplies no information on how and why the link reached the current state.
In particular, note that the device is allowed to refuse a transition into U1 or U2 indefinitely, unless the Force_LinkPM_Accept feature has been set. The link can hence stay in U0 even if the timeout parameters have been set up for going into U1 and/or U2 because of such refusals.
For this reasons are others, proper testing of power state transitions requires additional monitoring tools, preferably capturing the link layer traffic of power management packets (which are invisible to a USB packet sniffer on the host). It’s also important to monitor the device’s power states as a function of time, in particular if the U1 to U2 transition is enabled.
The tools for capturing this information are chosen depending on the project’s nature. For example, if the USB is implemented on an FPGA, this information is easiest obtained by capturing information from the FPGA logic itself.
A simple way to wake up the device from a low-power state (possibly momentarily) is to issue an
# lsusb -v
command (as root), as it queries the device for some configuration information. This traffic requires a wakeup of the device.
Using the module
This is worth repeating: Only a single device with matching Vendor / Product IDs may be connected to the host at a time.
Compile the kernel module listed below (possibly with “make”, using the Makefile also listed below). Then insert the module into the kernel with
# insmod timeouter.ko u1_timeout=100 u2_timeout=20
The two parameters are the PORT_U1_TIMEOUT and PORT_U2_TIMEOUT parameters that are set on hub’s port for the device. Section 10.4.2.1 in the USB 3.0 spec defined their meaning from the hub’s point of view.
The kernel module assigns a default of zero for both parameters, if not given in the insmod command.
A typical output in the kernel log when the module is loaded:
[ 1047.798260] usbcore: registered new interface driver timeouter [ 1047.996356] Manipulating 1234:5678, bus,dev,port=002,006,2 [ 1047.996514] Sending commands to hub: 05e3:0626, bus,dev=002,003
Note that if the module is already loaded, the manipulation takes place immediately. When the module is rmmod’ed, the timeout parameters may revert to defaults (depends on the hub, including root hubs).
Also note that the device will not enter U3 as long as the module is loaded, as it’s doesn’t set the “supports_autosuspend” flag. Nether does it supply methods for suspend and resume, so the module will cause a kernel oops if the device is suspended.
Module listing
The listing of the timeouter.c kernel module is given below. Replace the Vendor / Product IDs in USB_DEVICE to match the device under test.
It’s a hack for the sake of lab testing, and a bad example for coding practice: It uses a global variable for the USB device’s handle, and will therefore fail if there’s more than one USB device. Neither does it conform to kernel coding practices (indentation, comments and other issues).
And it sends commands to a USB device (its parent hub) that it’s not assigned to.
So except for testing U1/U2 transitions, leave it alone.
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/gfp.h>
#include <linux/uaccess.h>
#include <linux/workqueue.h>
#include <linux/usb.h>
// Usage: insmod timeouter.ko u1_timeout=120 u2_timeout=15
MODULE_LICENSE("GPL");
static int u1_timeout = 0;
static int u2_timeout = 0;
module_param(u1_timeout, int, 0);
module_param(u2_timeout, int, 0);
static const char driver_name[] = "timeouter";
static struct workqueue_struct *my_wq;
struct delayed_work timeouter_workitem;
struct usb_device *usbdev;
static void timeout_manipulate(struct work_struct *work) {
struct usb_device *parent = usbdev->parent;
int rc;
pr_warn("Manipulating %04x:%04x, bus,dev,port=%03d,%03d,%d\n",
usbdev->descriptor.idVendor,
usbdev->descriptor.idProduct,
usbdev->bus->busnum,
usbdev->devnum,
usbdev->portnum);
pr_warn("Sending commands to hub: %04x:%04x, bus,dev=%03d,%03d\n",
parent->descriptor.idVendor,
parent->descriptor.idProduct,
parent->bus->busnum,
parent->devnum);
rc = usb_control_msg(usbdev,
usb_sndctrlpipe(usbdev, 0),
1, // bRequest = CLEAR_FEATURE
0, // bmRequestType = Device
48, // wValue -> U1_ENABLE
0, // wIndex
NULL, 0, // data, length of data
100 // Timeout in ms
);
if (rc < 0)
pr_err("Failed to clear U1_ENABLE, error %d\n", rc);
rc = usb_control_msg(usbdev,
usb_sndctrlpipe(usbdev, 0),
1, // bRequest = CLEAR_FEATURE
0, // bmRequestType = Device
49, // wValue -> U2_ENABLE
0, // wIndex
NULL, 0, // data, length of data
100 // Timeout in ms
);
if (rc < 0)
pr_err("Failed to clear U2_ENABLE, error %d\n", rc);
rc = usb_control_msg(parent,
usb_sndctrlpipe(parent, 0),
3, // bRequest = SET_FEATURE
0x23, // bmRequestType = Class, Other
23, // wValue -> PORT_U1_TIMEOUT
usbdev->portnum | u1_timeout << 8, // wIndex
NULL, 0, // data, length of data
100 // Timeout in ms
);
if (rc < 0)
pr_err("Failed to set PORT_U1_TIMEOUT, error %d\n", rc);
rc = usb_control_msg(parent,
usb_sndctrlpipe(parent, 0),
3, // bRequest = SET_FEATURE
0x23, // bmRequestType = Class, Other
24, // wValue -> PORT_U2_TIMEOUT
usbdev->portnum | u2_timeout << 8, // wIndex
NULL, 0, // data, length of data
100 // Timeout in ms
);
if (rc < 0)
pr_err("Failed to set PORT_U2_TIMEOUT, error %d\n", rc);
}
static int timeouter_probe(struct usb_interface *intf, const struct usb_device_id *id)
{
struct device *dev = &intf->dev;
usbdev = interface_to_usbdev(intf);
my_wq = alloc_workqueue("timeouter", 0, 0);
if (!my_wq) {
dev_err(dev, "Failed to allocate workqueue\n");
return -ENOMEM;
}
INIT_DELAYED_WORK(&timeouter_workitem, timeout_manipulate);
queue_delayed_work(my_wq, &timeouter_workitem, 200);
return 0;
}
static void timeouter_disconnect(struct usb_interface *intf) {
flush_workqueue(my_wq);
destroy_workqueue(my_wq);
}
static const struct usb_device_id timeouter_id_table[] = {
{ USB_DEVICE(0x1234, 0x5678) },
{ }
};
MODULE_DEVICE_TABLE(usb, timeouter_id_table);
static struct usb_driver timeouter_driver = {
.name = driver_name,
.probe = timeouter_probe,
.disconnect = timeouter_disconnect,
.id_table = timeouter_id_table,
};
module_usb_driver(timeouter_driver);
Makefile
For convenience, a sample Makefile is given below
ifneq ($(KERNELRELEASE),) obj-m := timeouter.o else ifeq ($(TARGET),) TARGET := $(shell uname -r) endif PWD := $(shell pwd) KDIR := /lib/modules/$(TARGET)/build default: timeouter.c @echo $(TARGET) > module.target $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules clean: @rm -f *.ko *.o modules.order Module.symvers *.mod.? .timeouter*.* *~ @rm -rf .tmp_versions module.target .cache.mk endif
