1. Bespoke Silicon Group
  2. BSG
  3. bsg_pci




This is a full PCIe stack for sending/receiving data between an Xilinx-Virtex6 board (ML605) and a host-PC with a Linux kernel 2.6.X. The stack is composed of a hardware implementation, Linux kernel driver and host-side software.

This project perform a trivial-loopback between the Virtex6-FPGA in the ML605 board and the host-PC. Loopback-sample projects are well suited as starting point for more complex projects. For more information on how to modify this loopback please see the FAQ section.

See http://bjump.org for more details.

Build instructions

Build hardware using Xilinx-ISE with:

make build_hw

Download the bitstream with:

make download

Build Linux kernel driver with:

make build_driver

Build host-side software with:

make build_host

Source directory structure

  • Hardware implementation directory src/hw
    • Xilinx Impact script src/hw/impact/ml605.impact
    • Xilinx IP-cores directory src/hw/ngc
    • Xilinx ISE TCL flow src/hw/tcl/ise.tcl
    • Xilinx User Constraint File src/hw/ucf/top.ucf
    • Verilog code directory src/v
      • Asynchronous FIFO module code src/v/asfifo
      • Miscellaneous module code src/v/misc
      • PCI module code src/v/pci
      • Top-project module code src/v/top.v
  • Linux kernel driver src/sw/driver
    • PCI character device code src/sw/driver/bsg_pci_char.c
    • PCI hardware interface code src/sw/driver/bsg_pci_hw.c
    • PCI init and exit kernel function code src/sw/driver/bsg_pci_module.c
    • PCI prototypes and macros src/sw/driver/bsg_pci.h
  • Host-side software src/sw/host
    • Host-side function code src/sw/host/bsg_pci_host.cc
    • Host-side function prototypes src/sw/host/bsg_pci_host.h
    • Host-side main program src/sw/host/main.cc

Output directory structure

  • Xilinx-ISE output out/ise
  • Linux kernel driver out/driver/bsg_pci.ko
  • Host-side software out/host/bsg_pci_loopback


Hardware Implementation

  • Where the loopback happen?
    • The loopback is done in the FPGA.

  • Where should I start hacking the loopback?
    • Open src/hw/v/top.v and look for the module pcie_inst.

  • How can I add verilog source code to the project?
    • You can place source code any place you want. However, you could keep using the same directory structure. Open src/hw/tcl/ise.tcl and add the path for the files as: xfile add /path/to/file.

  • Where I specify the Xilinx license environment variable?
    • Open the Makefile in the main directory and modify XILINX_LM_LICENSE_FILE and XILINXD_LICENSE_FILE variables for your machine.

  • Where I specify the Xilinx binaries environment variables?
    • The only Xilinx programs used in this project are: xtclsh and impact. Open the Makefile in the main directory and modify XTCLSH and IMPACT_BIN for your machine.

  • Which Xilinx version was used for building the project?
    • 14.7

  • What is the revision number for the XC6VLX240T hosted in the ML605?
    • Version 2

  • How can I get the revision number of my ML605?
    • You can get the IDCODE using Xilinx-Impact and the version will show up

Linux Kernel driver

  • How can I load the kernel driver?
    • As superuser you can run insmod /path/to/bsg_pci.ko

  • How can I get logs after loading the kernel driver?
    • You can run dmesg

  • What output should I expect after the module is loaded?

    [bsg_pci debug] Initializing device data structures

    [bsg_pci] Reg: 0x2810, MaxPayload: 128, MaxRead: 512

    PCI: Setting latency timer of device 0000:05:00.0 to 64

    [bsg_pci debug] I/O Address: 0xfd9ff000 + 0x800

    [bsg_pci debug] Using IRQ 18

    [bsg_pci debug] Mapped Region: 0xffffc200100bc000 + 0x800

    [bsg_pci debug] Read the channel number from the hardware. Channel_number=0x00000002

    [bsg_pci debug] Read the channel number from the hardware. Channel_number=0x00000002

    [bsg_pci debug] Added Char Device 250, 0

    [bsg_pci debug] Before class_create

    [bsg_pci debug] After class_create

    [bsg_pci debug] After device_create

    [bsg_pci debug] Device Ready for Use

    [bsg_pci] Driver Registered

    [bsg_pci] Using PCIE ID: 10ee:0008

Host-side software

  • What output should I expect after running the host program a.k.a bsg_pci_loopback?

    BSG PCI status reg: 0x0000ffff

    Channel Number = 0x00000002

    Linux to FPGA status regs for channel 0 is = 0x000001fd

    Linux to FPGA status regs for channel 1 is = 0x000001fd

    BSG PCI status reg: 0x0000ffff

    BSG PCI device is ready to work!

    Finished to write on channel 0

    Received data = 0x00010002

    Received data = 0x00030004

    Received data = 0x00050006

    Received data = 0x00070008

    Received data = 0x0009000a

    Received data = 0x000b000c

    Received data = 0x000d000e

    Received data = 0x000f0000

    Finished Channel 0 test

    Finished to write on channel 1

    Received data = 0x10011002

    Received data = 0x10031004

    Received data = 0x10051006

    Received data = 0x10071008

    Received data = 0x1009100a

    Received data = 0x100b100c

    Received data = 0x100d100e

    Received data = 0x100f1000

    Finished Channel 1 test