Introduction

This page discusses USB SuperSpeed Transfers, Bursts and Short Packets, in an attempt to shed light on the relationships between these, and higher level terms, such as the xHCI Transfer Descriptor, Microsoft Windows’ IRP and Linux’ URB.

This is not an attempt to cover all aspects, so the respective standards should be referred to for the exact details (in particular the USB 3.0 and xHCI specifications).

SuperSpeed Streams are discussed on another page.

Transfers

The USB data flow is not based upon a continuous stream (like general file I/O and TCP/IP), not even regarding bulk endpoints: Chunks of data are transmitted in either direction in what is called “transfers”. Those who have written software for a USB device (a kernel driver or e.g. a libusb based user-space program) are familiar with this term: It's some kind of work item, that is executed by data being sent to or from the USB device.

Curiously, the USB 3.0 doesn't define this term accurately. Rather, it refers to the USB 2.0 spec (see citations at the bottom of this page), indicating that the same semantics are retained. The USB 2.0 spec relies on a Microsoft Windows term, IRP, in relation to transfers. The IRP is a Windows kernel data structure, that is created for system calls requesting I/O operations. Among others, it contains the address to the data buffer and the requested data length. In Linux, there is a parallel data structure for USB requests, called a URB.

The USB 2.0 spec makes a 1:1 connection between a transfer and an IRP on the operation system, in the context of defining the response to short packets arriving from the device. As just mentioned, the USB 3.0 spec merely references to this relationship.

A more appealing definition for a transfer relates to the “Transfer Descriptor” (TD), defined in the xHCI specification as the data structure used by the low-level software to request USB operations from the USB controller. These TDs are queued by the software (the xHCI driver) for each endpoint, each TD containing, among others, the number of bytes required for transmission (”TD Transfer Size”). The fulfillment of such TD is hence the execution of a USB transfer.

The length of a transfer

For an OUT (host-to-device) transfer, the number of bytes actually transferred is controlled by the host, and is hence always equal to the number required (unless something kind of error occurs). However for an IN (device-to-host) transfer, the exact amount of data is controlled by the device: It may signal the end of transfer by issuing a short packet (i.e. with less than 1024 bytes for USB 3.0). This is not considered an error by neither the USB 3.0 spec nor the eHCI spec (see for example eHCI 1.2 spec section 4.10.1.1.1), but just an early termination of the transaction with less data than expected.

If the number of bytes expected in an IN transaction isn’t a multiple of 1024, the last packet must be less than full (i.e. a short packet is imminent). The host’s USB 3.0 controller has no way to inform the device how many bytes it expects, but only limit the number of packets it’s allowed to send at any given moment. Therefore, the device may very well fill the last packet beyond what was expected by the host, causing a “Babble Error” per eHCI spec 4.10.2.4. It’s therefore generally wise to set the length in a TD to a multiple of 1024, even if less data is actually required, unless the device is guaranteed to produce the exactly correct amount of data.

SuperSpeed bursts

The major difference between USB 3.0 and earlier USB revisions is that there are separate physical wires for data in each direction, so packets can be transmitted in both directions simultaneously. In particular, acknowledgment packets can be transmitted at the same time as data continues to be transmitted in the opposite direction.

In order to facilitate the physical bandwidth efficiently, bursts of packets are allowed in USB 3.0. This allows more than one data packet for an endpoint to be transmitted before it’s acknowledged. The mechanism is implemented with the NumP field in ERDY and ACK packets, and with the EOB flag in Data Header Packets.

The essence of the NumP field is a permission to transmit data packets without receiving an ACK for the previous ones. It indicates the number of packets that the receiver of data packets is ready to immediately accept at the time that the ERDY or ACK packet that carries it is dispatched. NumP can be zero to temporarily stop the packet flow.

In more detail:

  • The ACK packet’s NumP field is the number of data packets, for the relevant endpoint, that the receiver is ready to receive after the data packet that the said ACK packet acknowledges.
  • ERDY is sent only by devices to indicate a readiness to receive data packets, as part of the flow control (see below). The meaning of its NumP field is the same as for an ACK packet, but ERDY doesn’t acknowledge anything.

Note that the NumP fields mentioned here are carried in ACK / ERDY packets sent by the side receiving the data packets. The data packet sender’s say on bursts is embodied in the EOB flag in data packet headers. When this flag is set, the sender indicates it has no more data packets to send at that moment. and also humbly asks the host not to request more data from that endpoint until further notice (i.e. with an ERDY packet, see below).

Even though NumP can’t possibly exceed the number of 16 (and is often limited even further), a burst can be infinite in principle, if the physical bus turnaround is quick enough: There might be a everlasting margin of allowance to send further data packets, hence the burst can go on until the end of the transfer, no matter how long. Nevertheless, when interpreting an ACK packet, its receiver must take the data packets it has sent since that packet into account, as the NumP field in the ACK packet relates to the data packet it acknowledges, and not the situation at the transmitting side.

It’s crucial to note that there’s no necessary connection between bursts and transfers. The termination of a burst (i.e. a flow control condition) doesn’t indicate an end of transfer, and an end of transfer doesn’t necessarily end a burst. The only point where they are related is when the device sends a short packet, after which it’s not allowed to continue sending data packets until receiving an ACK packet for that last short packet with a non-zero NumP. The rationale is that the device can’t know if there’s an outstanding TD for that endpoint after the transfer it has just terminated, hence it needs to get permission from the host’s USB controller to continue.

Note however that even though a short packet is de facto the last packet it a burst, it doesn’t put the endpoint in a flow control condition, unless that last packet had the EOB flag set. In particular, a burst of an OUT endpoint may continue immediately after a short packet if the host has another TD queued.

Bulk endpoint’s flow control

The USB 3.0 spec, like all the others before it, lets the host control of all data flow. This goes along with the TD queuing mechanism defined in the xHCI spec, so there’s data exchange with an endpoint only if there are TDs queued. In other words, USB traffic takes place only when there are outstanding requests by the software for it (and specifically allocated data buffers for its data).

Unlike USB 2.0, there is no polling of endpoints, but instead the host sends data at will to the device (OUT endpoint), or requests data by sending an ACK packet with a nonzero value in the NumP field (IN endpoint). Such ACK packet can initiate a burst or be an intermediate response in an ongoing one. The device may refuse to communication by responding with an NRDY packet (in both directions).

To avoid wasting bandwidth on refused attempts by the host, a flow control mechanism requires the device to inform the host on its readiness to send or receive data (see section 8.10.1 in the spec): Both IN and OUT endpoints go into flow condition by sending an NRDY packet, and return to active state (i.e. exit flow control condition), by sending an ERDY packet.

Endpoints may also enter the flow control condition efficiently by terminating an ongoing burst gracefully: An IN endpoint may set the EOB bit in a Data Packet Header on the last packet it transmit, thereby signaling it’s the last packet it has. An OUT endpoint responds with an ACK with the NumP field set to zero. In both cases, the flow condition is invoked without any extra handshake traffic. An ERDY packet from the device is required to get into active state again.

Note that while the meaning of the NumP field in an ERDY packet on an OUT endpoint is the same as for an ACK packet (the number of packets the device is ready to receive for that endpoint), the significance of NumP in an ERDY packet for an IN endpoint is somewhat less clear. According to Table 8-14 in the USB 3.0 spec, it should contain the number of data packets that the device has to send. It further says it’s possibly for informative purposes only, and yet it’s somewhat unclear why this is required; after all, the host’s USB controller will know when the data is exhausted by virtue of the EOB field. Possibly it helps with bandwidth efficiency or even preventing the deadlock situation discussed below, by preferring endpoints with a larger NumP.

Also note that the host has the prerogative of forgetting the flow control state, and attempt communication at any time. The device simply responds with an NRDY in that case. Inefficient, but legit. On the other hand, the device is not allowed to send ERDY packets if it isn’t in a flow control condition (USB 3.0 spec section 8.10.1). So the host is allowed to forget its flow control condition and try its way by sending packets that may turn out to be a waste of bandwidth, but the device doesn’t have this freedom.

Looking at the overall picture, the device sort-of controls when data is transmitted by virtue of flow control, but the host has the final say: No data is transmitted anyhow if there’s no queued TD, and the host is allowed to ignore the flow control state (even though this is probably uncommon).

Short packets

A short packet marks the end of transfer, but not necessarily a flow condition: Per section 8.10.3, a short packet is always the last in a burst, and indicates the end of transfer. However by itself, it doesn’t request a flow control condition, so the host may issue another ACK (i.e. DATA IN request) packet.

After sending a short packet, the device must stop sending packets for that endpoint until it has received an ACK for that packet. A nonzero NumP of that ACK packet indicates that the host is ready to receive more packets (i.e. that there another TD queued for the same endpoint). If the NumP is zero, there is no more TD queued, in which case no packets should be sent.

An EOB requests a flow condition. Per section 8.10.1, to issue a flow control condition, a DP with EOB set to one or an NRDY packet is sent by the device. The host is not likely to issue an ACK packet for that endpoint until an ERDY packet arrives from the device, but it may.

Interleaving between bulk endpoints transactions

Per section 8.1 in the USB 3.0 spec: The host may interleave packets of BULK OUT transactions, but not BULK IN. The said section clearly states that an IN transaction can’t be initiated until the previous one has been terminated: “… host shall not initiate another IN bus transaction to any endpoint until the host receives all DPs or an NRDY or a STALL TP or the transaction times out for the current ACK TP sent”.

This opens for a deadlock situation, if a BULK IN endpoint responds to a request from the host with less data than required to complete a transfer, but terminating it with an EOB rather than by virtue of a Short Packet. This prevents the host from using the data channel for starting BULK IN transfers from other endpoints, and the current endpoint is in a flow control condition. This deadlock is easily avoided by proper design of the device / endpoint relations, in particular if the endpoint always sends a Short Packet when it runs out of data (this may even be a zero length packet if the last data packet was 1024 bytes long).

Alternatively, the USB controller could make use of its prerogative to attempt communication even if the device has signaled a flow control condition. Accordingly, it may attempt to send another ACK packet to the device, for the sake of provoking it to respond with NRDY, which allows it to go on with other endpoints per the citation above.

This knot is untied in later USB spec revisions, but only for devices of these later USB revisions among themselves: Simultaneous IN Transactions are allowed in the USB 3.2 spec, but not to a SuperSpeed (USB 3.0) bus instance. In USB 3.2’s section 8.1.1, “receives a DP with EOB flag set” is listed as an condition not appearing in USB 3.0 for starting a BULK IN transaction to another endpoint. In other words, USB 3.0 requires the end or absence of a BULK IN transaction for moving on to another, but on USB 3.2 it’s allowed between bursts.

It appears like someone forgot to mention the EOB flag in the list of excuses to switch to another endpoint in the USB 3.0 spec.

References in the USB 3.0 / 2.0 specs on Short Packets and transfers

These are the places in the two specifications from which the definition a transfer can be derived, through its relation with Short Packets.

USB 3.0, section 8.10.3 (“Short Packets”) says that SuperSpeed retains the semantics of short packet behavior that USB 2.0 supports, but it doesn’t say exactly what a “transfer” means, possibly avoiding Microsoft Windows terms (IRPs in particular):

SuperSpeed retains the semantics of short packet behavior that USB 2.0 supports. When the host or a device receives a DP with the Data Length field shorter than the maximum packet size for that endpoint it shall deem that that transfer is complete.

USB 2.0, section 5.3.2 (“Pipes”) clarifies this:

If there are no IRPs pending or in progress for a pipe, the pipe is idle and the Host Controller will take no action with regard to the pipe; i.e., the endpoint for such a pipe will not see any bus transactions directed to it. The only time bus activity is present for a pipe is when IRPs are pending for that pipe.

and further down:

An IRP may require multiple data payloads to move the client data over the bus. The data payloads for such a multiple data payload IRP are expected to be of the maximum packet size until the last data payload that contains the remainder of the overall IRP. See the description of each transfer type for more details. For such an IRP, short packets (i.e., less than maximum-sized data payloads) on input that do not completely fill an IRP data buffer can have one of two possible meanings, depending upon the expectations of a client:

  • A client can expect a variable-sized amount of data in an IRP. In this case, a short packet that does not fill an IRP data buffer can be used simply as an in-band delimiter to indicate “end of unit of data.” The IRP should be retired without error and the Host Controller should advance to the next IRP.
  • A client can expect a specific-sized amount of data. In this case, a short packet that does not fill an IRP data buffer is an indication of an error. The IRP should be retired, the pipe should be stalled, and any pending IRPs associated with the pipe should also be retired.

As suggested above, the term “IRP” is would probably have been best replaced with an xHCI Transfer Descriptor (”TD”).