HTTPS SSH

System Requirements

The USB REVue toolset was developed and tested under Ubuntu 10.04 and 11.04. Some aspects work in 10.04 and some are known to not work. For other distros, YMMV, depending on whether the minimum requirements are met.

  • tcpdump (corresponding with libpcap)
  • libpcap 1.1.0 (1.0.0 is known to not work)
    • 1.0.0 uses the read(2) interface to usbmon, which does not result in a complete capture.
  • python-pcapy
  • python-qt4
  • python-gflags
  • python-qwt5-qt4
  • usbmon

In addition, for usbreplay, ensure that pyusb-1.0.0 is inserted into the python tree. For example, on some linux systems, a good location would be: /usr/lib/python2.7. Pyusb, for example, allows imports of usb.core and usb.util.

Viewer

The viewer allows the user to view, edit, annotate, and analyze a USB packet stream.

  1. USAGE

To view a pcap file:

$ ./usbview.py < file.pcap

To view a live stream, and dump to a file:

$ sudo tcpdump -i usbmon0u -w - -U | ./usbview.py -p > outfile.pcap

The -p option enables 'passthru' -- all incoming packets will be dumped to output as they arrive.

  1. FILTERING

The viewer provides filtering of displayed and captured packets using user- provided Python expressions. All fields of the usbmon_packet struct are available (see section "THE USB PACKET" below).

The capture filter instructs the viewer to ignore any incoming packets for which the filter expression returns False. The display filter expression is used to determine which packets are visible in the view. Use the capture filter to remove packets you will never be interested in; use the display filter to focus on the packets you are interested in right now.

A typical capture filter might look like

devnum == 3 and xfer_type != isochronous

This will ignore traffic to all devices but device 3, and will remove any isochronous packets from the captured stream.

Examples of display filter expressions:

  • Show traffic from endpoint 0x81 (mind the hex!)
    epnum == 0x81
  • Show setup packets
    setup
  • Show setup packets with a bmRequestType type of 'vendor'
    setup.bmRequestTypeType == 'vendor'
  • Show packets with a data payload
    data
  • Show errors
    event_type == 'E'
  • Show callbacks from interrupt endpoints
    event_type == 'C' and xfer_type == interrupt
  1. ANNOTATIONS

At the bottom of the window is a text field for inserting annotations into a live packet stream. To use this, just type something into it and press enter. Note that there is currently no way to preserve annotations across sessions.

  1. EDITING

Double-clicking existing packet data allows it to be edited. The editor only accepts hexadecimal characters as input, and does not allow the length of the data to be changed.

  1. DUMPING

Selected packets can be dumped to standard output, either for writing packets to a pcap file or for interaction with other tools. To do this, select one or more packets in the list, right click, and choose 'Dump selected'.

Note that dumping will do nothing if standard output is not redirected or piped on the command line.

  1. CODE GENERATION

The viewer supports limited code translation of packets. A selection of packets can be translated into the libusb C code that would replicate them by choosing 'Copy as libusb code' from the context menu.

An example of generated code for an incoming control transfer:

if ((err = libusb_control_transfer(handle, 0xa1, 0xfe, 0x0, 0x0, data, 1, TIMEOUT)) < 0)
handle_error(err);

Note that this assumes the existence of a number of variables and constants:

  • 'handle' is an open libusb device handle
  • 'err' is an integer
  • 'data' is a character array large enough to contain the data (in this case, it need only be one byte long)
  • 'TIMEOUT' is an integer
  • 'handle_error' is a function or macro that deals with an error code

Outgoing packets use string literals for the data, and expect 'n_transferred' to be defined as an integer. Code formatting and names are not currently configurable.

Note that any reference to device and bus number are lost in the translation. If you attempt to generate code for a selection that includes multiple devices, you will receive a warning saying so. (The code is still generated, just in case you know what you're doing.)

  1. OPTIONS

'Passthru' dumps incoming packets as soon as they're received. It is disabled by default, but can be enabled using the -p command-line option.

'Autoscroll' ensures that the most recently captured packet is visible in the display. It is disabled by default.

Both options can be toggled from the context menu. Note that these are only particularly useful for live captures, not for viewing prerecorded dumps.

Modifier

The purpose of the Modifier is to alter packets programmatically. It allows the user to apply a function to every packet in the input stream and writes the modifier packets as output.

  1. GETTING STARTED

Like other tools in the USB REVue toolkit, the Modifier reads and writes via a pcap-formatted stream or file. The Modifier reads from standard input and writes to standard output. To connect an input source and output destination, use a pipeline. For example, to read from a pcap file called "foo.pcap" and write to a file called "bar.pcap", do:

$ cat foo.pcap | ./modifier.py > bar.pcap

To read a pcap stream directly from USB bus 7, do:

$ sudo tcpdump -i usbmon7u -w - -U | ./usbmodify.py[ | ...]

Note that if no output destination is given, all packets will be silently lost.

  1. THE USB PACKET

USB REVue uses a USB packet class based on the usbmon_packet C struct described in the usbmon documentation. Each packet contains the following typed attributes (usbmon_packet name given in parentheses where it differs from USB REVue):

u64 urb (id) unsigned char event_type (type) unsigned char xfer_type unsigned char epnum unsigned char devnum u16 busnum char flag_setup char flag_data s64 ts_sec s32 ts_usec int status unsigned int length unsigned int len_cap unsigned char setup[SETUP_LEN] -- only for Control S-type unsigned int error_count -- only for ISO unsigned int numdesc -- only for ISO int interval int start_frame unsigned int xfer_flags unsigned int ndesc unsigned char data[...]

The setup array, if present, contains the following packed fields, accessed by referencing setup.[name]:

bmRequestType bRequest wValue wIndex wLength

Alternatively, you can access the transfer direction, type, and recipient subfields of the bmRequestType bitmap with the following:

bmRequestTypeDirection
(possible values: 'host_to_device', 'device_to_host')
bmRequestTypeType
(possible values: 'standard', 'class_', 'vendor', 'reserved')
bmRequestTypeRecipient
(possible values: 'device', 'interface', 'endpoint', 'other')

The data array may be of variable size, depending on the specific packet. For more information about these attributes, see the usbmon documentation (probably supplied with your Linux kernel documentation, or available at http://www.mjmwired.net/kernel/Documentation/usb/usbmon.txt).

Since the resulting packet must still be valid for encoding, any modified attribute values must still be of the respective type indicated above. For example, changing 'status' to a floating-point number will result in an error.

  1. ALTERING PACKETS

The Modifier requires that the user specify at the command line the way(s) in which incoming USB packets will be modified. There are three methods for altering USB packets, ranging from most restrictive to most flexible: simple statements, an external routine, and an external Python module.

With any of the above methods, to display the details of each modified packet on the fly, use the --verbose flag.

3a. With Simple Statements (--exp)

Simple statements are designed to easily modify the data payload of a stream of USB packets. Other packet attributes may be modified as well, but this behavior is not supported. To use simple statements, use the --exp flag followed by one or more quoted and comma-separated statements:

$ ./usbmodify.py --exp "data[0] = data[1] + data[2]","data[3]= data[4] ^ data[5]"

Data payload byte offsets are referenced as "data[0], data[1], ...". Arithmetic operators (+, -, *, /), logical operators (and, or, not), and bitwise operators (^, &, |, !) are supported. For logical xor, use "bool(a) ^ bool(b)".

The statement(s) will only be applied to a USB packet if the packet's data payload contains every byte offset referenced in ALL statements. For example, given the two statements "data[0] = ! data[1]" and "data[2] = data[3] | data[4]", a packet must have at least 5 data bytes for either of these statements to be applied to it.

3b. With an External Routine (--routine)

The user can use an external routine to specify more complex modifications. Any packet attribute may be referenced and/or altered. The external routine must be written as a sequence of one or more Python statements. To use an external routine, use the --routine flag followed by the name of the file containing the routine. For example, the following routine is saved as "mod_routine":

if len(data) >= 8:
data[7] = data[0] or data[1]
elif epnum == 0 and not status:
status = 1

The routine can be applied to all incoming packets by doing:

$ ./usbmodify.py --routine mod_routine

Note that, unlike simple statements passed at the command line, there is no checking done on the existence of attributes or byte offsets before the routine is applied. Be sure to examine each packet (e.g. using the xfer_type attribute or Python's len()) where necessary.

For more information about Python statements, see the Python documentation (http://docs.python.org).

3c. With a Python Module (--mod)

A user-supplied Python module is the most flexible way to modify a USB stream. Instead of supplying a routine to be applied to each packet automatically, the user-supplied module can add or remove packets from the stream and save information about previous packets.

The module must be saved with a .py extension in the same directory as usbmodify.py. To use a user-supplied module, use the --mod flag followed by the name of the module (without the .py extension).

The module must implement a function with the following interface:

modify(packet_gen, commit_func)

The function must take two arguments, both of them functions. The first argument ("packet_gen") is a Python generator that supplies individual USB packets from the incoming pcap stream/file. The second function ("commit_func") is used to pass a packet to the output stream.

To access the USB packet stream, use the generator passed as the first argument:

for packet in packet_gen():
...

To pass a single packet to the output stream, use the function passed as the second argument:

commit_func(packet)

Note that any packet destined for output, whether modified or not, must be passed to the commit function. To remove or ignore a packet, simply don't pass it to the commit function.

Additionally, at any time you can create a new packet by importing Packet from usbrevue:

new_packet = Packet()

You can then assign values to the new packet's attributes manually.

As a simple example, the following module output all packets with epnum 1 or 2, modifies the data of packets with epnum 2, and ignores all other packets:

"""mymod.py"""

def modifier(generator, commit):

for packet in generator():

if packet.epnum == 1:
commit(packet)
elif packet.epnum == 2:
if len(packet.data) >= 4:
packet.data[3] = packet.data[0] | packet.data[1]

commit(packet)

else:
continue

To run this module, do:

$ ./usbmodify.py --mod mymod

Grapher

The Grapher allows the user to plot values on a two-dimensional Cartesian plane based on the data payload of an incoming stream of USB packets.

  1. USAGE

To graph from a pcap file:

$ ./usbgraph.py < file.pcap

To graph from a live stream:

$ sudo tcpdump -i usbmon0u -w - -U | ./usbgraph.py

Note that the Grapher has no dumping or output functionality.

  1. INTERFACE

The Grapher provides a graphical user interface for setting up and displaying data and their plots. The main window is composed of the following components (from the top left and moving clockwise):

  • A table of all incoming byte offset values
  • 'Byte Values' - A plot of selected byte offset values over time
  • 'Plot Width' - Control how many packets are displayed in the
    plot's x-axis
  • 'Clamp Y Axis' - Manually adjust the maximum and minimum values
    of the plot's y-axis
  • 'Custom Byte Expressions' - Define custom value(s) based on one
    or more byte offsets to be plotted
  1. PLOTTING BYTES

Each incoming USB packet data payload is listed as a single row in the table. The data bytes are displayed by offset. To graph the value of a byte offset over time, click the checkbox in the first row of that offset column in the table.

'Autoscroll', available by right-clicking in the table, ensures that the most recent packet is visible in the display. You may need to disable autoscroll in order to click a byte offset checkbox.

The user can display up to 31 byte curves at one time, including simple byte offsets and custom byte expressions (below).

3.1 Custom Byte Expressions

The 'Custom Byte Expressions' feature allows the user to manipulate one or more byte offsets in order to define custom value(s) to plot. Use the following notation to refer to existing byte offsets:

  • byte offset 0: [0]
  • byte offset 1: [1] ...

Any valid Python expression, using bitwise, arithetic, or logical operators, can be used to modify byte offset values. For example, to left-shift byte offset 0 by 8 bits and add byte offset 1, enter:

([0] << 8) + [1]

Or equivalently:

([0] * 256) + [1]

Separate multiple expressions with commas. To plot the square of byte offset 0 and the sum of byte offsets 2 and 3:

[0] ** 2, [2] + [3]
  1. PLOT OPTIONS

The user can adjust the length of the x-axis and the maximum and minimum values of the y-axis.

The most recent packets are always displayed in the plot. To adjust how many previous packets are displayed, use the 'Plot Width' slider.

The y-axis will automatically adjust to encompass the minimum and maximum values of all active bytes in the given range of packets. To manually adjust the maximum and minimum values of the y-axis, use the 'Clamp Y Axis' feature. Note that if min/max values are given manually, both a minimum and maximum must be specified.

Replayer

The purpose of the Replayer is to replay a packet stream to a usb device. The packet stream may or may not have been modified by the modifier or other modules as described elsewhere in this document.

  1. GETTING STARTED

Like other tools in the USB REVue toolkit, the Replayer reads a pcap-formatted stream or file. Unlike other tools, it does not output a pcap stream. Instead, the output is usb packets replayed to a usb device. Thus, the Replayer reads from standard input and writes to a usb device.

Command line help may be found by executing:

$ sudo ./usbreplay.py -h
OR

$ sudo ./usbreplay.py --help

To determine vendor and product id's of USB devices on your system, you may execute the command, 'lsusb'.

To connect an input source and output destination, you may use a pipeline. For example, to read from a pcap file called "usbmon.pcap" and replay to a usb device with vendor id 0x1111 and product id 0x2222, you may do the following:

$ cat usbmon.pcap | sudo usbreplay.py --vid 0x1111 --pid 0x2222

Another method using command line arguments, is to send a dev.pcap file for a keyboard device on interface 3 as follows. Note that This particular device has vendor id 0x413c and product id 0x2105:

$ sudo ./usbreplay.py -v 0x413c -p 0x2105 -i 3 -f dev.pcap

1.1 A Replayer Use Case

1.1.1 Use case 1: Toggling capslock and numlock leds

When capslock or numlock is toggled, a corresponding led light on the keyboard is also toggled. This use case shows how to generate a pcap file containing commands to toggle the keyboard capslock and numlock leds.

To replay the capslock and numlock leds, I did the following:

  1. Executed 'lsusb' to list the usb devices along with their vendor and product ids. This produced the following output.

    Bus 007 Device 009: ID 413c:2003 Dell Computer Corp. Keyboard Bus 007 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub Bus 006 Device 044: ID 0461:4d22 Primax Electronics, Ltd Bus 006 Device 040: ID 413c:2105 Dell Computer Corp. Model L100 Keyboard Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

  2. I then determined the bus used for the keyboard was bus 6 for device 40.

  3. To capture traffic on bus 6 I executed the following command. sudo tcpdump -i usbmon6u -w keyboard.pcap

  4. I then toggled the capslock and numlock keys again and again so that usb traffic from the keyboard could be captured into the keyboard.pcap file by usbmon. When I was through, I hit CNTRL-C to abort usbmon.

  5. Note that the usbmon command captures all traffic on bus 6, including keyboard traffic. Therefore, to eliminate all but the keyboard traffic, I piped the keyboard.pcap file into the Viewer so I could filter out all traffic on bus 6 except for the keyboard device which according to lsusb is device 40.

    I also used 2 keyboards, since once the replayer starts it disables the kernel driver for the keyboard device being replayed.

    The command I executed was therefore: cat keyboard.pcap | ./usbview.py | sudo ./usbreplay.py --vid 0x413c --pic 0x2105

  6. Once the Viewer came up, I filtered out everything except device number 40 as follows by inserting the following command into the Display filter: devnum==40

  7. I then selected some packets and replayed them by using the 'Dump selected' menu item. After playing around with replaying selected packets, I then replayed everything so I could see the led visual toggle pattern I had originally created. I did this by selecting all packets in the Viewer, and then using the 'Dump selected' menu item to dump all selected packets into the Replayer to be replayed en mass to the usb device.

  8. As all packets were replayed, the capslock and numlock leds toggled on and off according to the pattern I originally created. Success!

  9. To get the keyboard back, I then closed the Viewer and used the second keyboard to CTRL-C out of the Replayer, since the keyboard being replayed had its kernel driver disabled. Note that CTRL-C'ing out of the replayer will re-enable the kernel driver.

  1. THE USB DESCRIPTORS

Note that much of the information below is derived from 'USB in a NutShell'.

The Replayer needs at a minumum a device designated by its vendor id and product id. All other arguments may then be either derived from this device if it exists, or given as command line arguments. If no vendor or product id is given, then vendor and products id defaults are used, which only work for one specific computer. Therefore, it is recommended to always specify the vendor and product id for your device.

You may execute lsusb at any time to see a listing of USB devices on your system.

2.1 Device descriptor

A basic usb class hierarchy is shown below. Note that a device descriptor, as shown below at the top of the hierarchy, includes all sub-descriptors and therefore represents an entire device. This, of course, means that any USB device can only have one device descriptor.

device

/

configuration configuration

/

interface interface ...

/

endpoint endpoint ... ...

2.2 Configuration descriptor

Each USB device can have more than one configuration, although most USB devices tend to only have one configuration. The configuration descriptor specifies values for each configuration the device has, such as amount of power a configuration uses, if device is self or bus powered and number of interfaces it has. Since very few devices have more than 1 configuration, the cfg index will usually be empty and we will simply select the only active configuration, which happens to be at index 0. Note that only 1 configuration may be enabled at a time.

2.3 Interface descriptor

The interface descriptor represents a group for performing a specific function on the bus.

For example, for an all-in-one printer, interface 1 may represent and describe the endpoints of the fax function, interface 2 may describe the endpoints of the copy function, and interface 3 may describe the endpoints of the printer function. Note that more than one interface may be enabled at a time, and that the altsetting function allows the interface to dynamically change interface settings.

2.3 Endpoint descriptor

Endpoint descriptors are used to describe any endpoints other than endpoint zero. Endpoint zero is the control endpoint, is the only required endpoint and is configured first.

USB devices are host-centric, so with respect to the host, an endpoint is viewed as either a source or sink for data. For example, if you send a packet to your device at endpoint 1, the data flows out from host to device, ending up in the endpoint 1 buffer. At that point, your firmware decides when to read the data. Alternatively, if the device has data to send to the host, it cannot initiate this transaction since the bus is controlled by the host and not the device. It can however write to the endpoint 1 IN buffer. The host then can send an IN packet at any time requesting that data. This explains why the Replayer requires polling threads, since it never knows when data is being presented from the device to the host.

Note that, as previously stated, all devices must support endpoint zero. This is necessary since all control and status requests are sent through endpoint zero.

  1. CAVEATS

It is possible that due to resolution issues, replayer timing will not be exact. This may or may not result in issues.

Note that to abort the replayer, it is necessary to hit CNTL-C.

Statisfier

The purpose of the statsifier is to tell some basic statistics about the usb data. This data includes minimum and maximum values (if they apply), the number of packets vs. the number of packets matched (if matching applies) and the frequency of occurance of packets (if it applies.)

1. GENERAL USE Like other tools in the USB REVue toolkit, the Statisfier reads. The Statisfier reads from standard input. Unlike other tools the Statisfier must be able to write to standard output (in its current form.) To connect an input source and output destination, use a pipeline. For example, to read from a pcap file called "foo.pcap" do:

$ cat foo.pcap | ./modifier.py

To read a pcap stream directly from USB bus 7, do:

$ sudo tcpdump -i usbmon7u -w - -U | ./usbmodify.py[ | ...]

The statisfier will post the information desired to standard output.

2. USE CASES There are two primary ways you'll likely use this tool. One is with a comparison and thus find how often fields meet certain criteria. To do this you'd use an expression akin to:

$ cat foo.pcap | ./modifier.py --exp "data[0] >= data[1]+1"

Another is simply to check on a set of data. If this is the case:

$ cat foo.pcap | ./modifier.py --exp "data[0]"

Currently only one expression can be successfully used. This is to be fixed. If more than one expression is entered then the number of packets will be incorrect when displayed and the only reporting will be on the final expression. This is (obviously) a known error.

Note: A later version of the Statisfier should be written to instead post its results (and update them) in realtime to a separate window and instead use standard output in the same format as the rest of the tools.

That said, when this goes into the effect, the currently deprecated "--verbose" option should be uncommented in the code and be used to post to standard output while a GUI window or some other alternate option takes care of the output from the program.