Zipversa Demo Part 1

Overview

The goal of the ZipVersa project is to demonstrate how a completely open source tool chain can be used to compile HDL designs, including a RISCV core, to a Lattice ECP5-5G Versa FPGA board.

In Part 1 of the demo, we will manually walk through the process and describe how to get example projects/simulations up and running.

In Part 2 of the demo, we will write proper scripts for doing the steps in Part 1.

OS

Fedora Release 30

Board

Lattice ECP-5G Versa Board with Micron Flash (support for Macronix flash is under development)

Hardware Setup

The ZipVersa design currently supports UDP based networking using Ethernet Port 1 of the Lattice board (i.e. the port closer to the USB cable). There are two major limitations to this support: 1) no DHCP support i.e. requires a static IP to be assigned for the FPGA, and 2) no Tri-Speed Ethernet MAC i.e. it can only operate at 1Gbps. If you have a router with 1Gbps LAN ports, simply connect the board and host machine to it, set up the DNS, and you should be good to go.

In my case, the router had 100Mbps LAN ports. Since only the FPGA has a hard requirement for a 1Gbps link, I simply cascaded the router with a switch (instead of buying a new router) as shown in the figure below. Now, the FPGA and host talk to the switch over a 1Gbps link, while the link to the router is 100Mbps.

alt text

Link status can be easily verified by running

./netstat

from zipversa/sw/host once the design has been loaded.

Before You Begin

Verify Jumper Placement

From Project Trellis: “If your Versa board is new, you will need to change J50 to bypass the iSPclock. Rearrange the jumpers to connect pins 1-2 and 3-5 (leaving one jumper spare).” See page 20 of the user guide.

Verify Flash Device

The flash controller is configured to run in QUAD I/O XIP mode and uses commands specific to the Micron N25Q128A flash device. While the Macronix flash device has similar commands, it does not support XIP mode. Therefore, the design currently only works with Micron N25Q128A flash devices (or a device with XIP support and the same commands as Micron)

Verify Network Connectivity

Verify IP addresses of the testbed, as well as MAC address of the board. Ensure that the networking hardware, including cables, support Gigabit ethernet.

Verify Host Connectivity

Ensure that your machine can see the FPGA via the USB cable. Running lsusb should print out something along the lines of:

Bus 001 Device 003: ID 0403:6010 Future Technology Devices International, Ltd FT2232C/D/H Dual UART/FIFO IC

Demo Walkthrough

IP and MAC Addresses

Next we add in our custom network parameters for the FPGA static IP4 address (DEVIP), host machine IP4 address (HOSTIP), gateway IP4 address (ROUTERIP), FPGA MAC address (DEVMAC) and the subnet mask (MASK).

If you are running this project using QEMU/KVM, the ROUTERIP is the physical gateway and HOSTIP is the virtual IP4 address for the VM.

DEVMAC is currently taken as is from the original ZipVersa project.

DEVIP=(192 168 8 10)
HOSTIP=(192 168 8 100)
ROUTERIP=(192 168 8 1)
DEVMAC="a25345b6fb5e"
MASK=("FF" "FF" "FF" "00")

Select TTY Device

Next we select the FPGA tty device. This will be the serial port used to communicate with the FPGA once the ZipVersa base design has been loaded. In my case, this was ttyUSB1.

UBP=$(ls /dev/ | grep "ttyUSB1")

YOSYS

Now that we are all set with specifying our testbed specific parameters, let’s get to setting up the environment. First up is Yosys, an open source synthesis tool. It compiles the HDL source files and generates a netlist (JSON format in our case) for the target device. The design is optimized using the Berkley ABC optimizer.

dnf -y groupinstall "Development Tools" "Development Libraries"
dnf -y install cmake clang bison flex mercurial gperf tcl-devel libftdi-devel python-xdot graphviz
git clone https://github.com/YosysHQ/yosys.git
cd yosys
make -j$(nproc)
make install
cd ..

Boost.Python 3

Next we install Boost.Python 3, which is needed by Project Trellis.

dnf -y install boost-python3-devel

Project Trellis

Project Trellis is the database containing the reverse engineered low-level layout of the Lattice ECP5 boards. This database is what allowed open source Place and Route tools, such as Nextpnr (below), to generate the FPGA bitstream from a Yosys netlist output. Installing Project Trellis requires cloning two separate repositories as shown below. The target installation directory can be specified using -DCMAKE_INSTALL_PREFIX when running cmake.

git clone --recursive https://github.com/SymbiFlow/prjtrellis
cd prjtrellis
rm -rf database
git clone https://github.com/SymbiFlow/prjtrellis-db database
cd libtrellis
cmake -DCMAKE_INSTALL_PREFIX=/usr .
make
make install
cd ../..

Nextpnr

As mentioned above, Nextpnr is an open source Place&Route tool. It maps the logical layout and connectivity of the synthesized design to an actual set of Look Up Tables (LUTs) and wires/switch-fabric in the FPGA. Place & Route is more time consuming than synthesis, and a harder problem to open source since the physical layout of the FPGA boards is typically proprietary.

Using the -DARCH flag, we configure it for the ECP5 board. If Project Trellis was installed in a custom folder, then modify -DTRELLIS_ROOT to specify this location.

dnf -y install eigen3-devel qt5-devel
git clone https://github.com/YosysHQ/nextpnr.git
cd nextpnr
cmake -DARCH=ecp5 -DTRELLIS_ROOT=/usr/share/trellis
make -j$(nproc)
make install
cd ..

OpenOCD

Finally, we install OpenOCD which allows the bitstream, generated above, to be downloaded onto the FPGA board. For this demo, we build OpenOCD from source using the commands below. Once OpenOCD is set up, one can start to deploy custom designs for the FPGA. Project Trellis has a few example you could try.

tar -xf openocd-0.10.0.tar.bz2
cd openocd-0.10.0
./configure  --enable-ft2232_libftdi --enable-libusb0 --disable-werror
make
make install
cd ..

The rest of this demo is going to set up dependencies for the ZipVersa project and run an example design.

RISCV GNU Toolchain

The example designs in Project Trellis use assembly language to code for the PicoRV RISC-V core. As designs get more sophisticated, a C/C++ compiler is needed for the RISC-V. Enter the RISCV GNU toolchain. We install the toolchain for a 32 bit integer instruction set architecture (--with-arch=rv32i) and a programming model with 32 bit data types ints/long/pointers (--with-abi=ilp32).

dnf -y install autoconf automake libmpc-devel mpfr-devel gmp-devel gawk  bison flex texinfo patchutils gcc gcc-c++ zlib-devel expat-devel
git clone --recursive https://github.com/riscv/riscv-gnu-toolchain
cd  riscv-gnu-toolchain
./configure --prefix=/opt/riscv --with-arch=rv32i --with-abi=ilp32
make        
export PATH="$PATH:/opt/riscv/bin"
cd ..

Simulation Tools

Verilator is an open source RTL simulation tool while Gtkwave allows us to plot the signal waveforms. Of these, Verilator is required to build the ZipVersa project.

dnf -y install verilator
dnf -y install gtkwave

AutoFPGA

“The goal of AutoFPGA is to be able to run it with a list of peripheral definition files, given on the command line, and to thus be able to generate (or update?) the various board definition files.”

git clone https://github.com/ZipCPU/autofpga
cd autofpga/
make
export PATH="$PATH:$PWD/sw"
cd ..

ELF Utils

Libelf is needed to build the applications in sw/host.

git clone git://sourceware.org/git/elfutils.git
cd elfutils
autoreconf -i -f
./configure --enable-maintainer-mode --disable-debuginfod
make
make check
make install
cd ..

NCURSES

Needed to build zipdbg in sw/host.

dnf -y install ncurses-devel

ZipVersa

Now that we have set up our environment, we can now get to building the actual ZipVersa project.

Clone Git

git clone https://github.com/asanaullah/zipversa
cd zipversa/

Update Network Addresses

Using the sed command, update the network parameters specified above.

sed -i "58d" sw/rv32/etcnet.h
sed -i "58i #define	DEFAULTMAC	0x${DEVMAC}ul" sw/rv32/etcnet.h
sed -i "66d" sw/rv32/etcnet.h
sed -i "66i #define	DEFAULTIP	IPADDR(${DEVIP[0]},${DEVIP[1]},${DEVIP[2]},${DEVIP[3]})" sw/rv32/etcnet.h
sed -i "72d" sw/rv32/etcnet.h
sed -i "72i #define	LCLNETMASK 0x${MASK[0]}${MASK[1]}${MASK[2]}${MASK[3]}" sw/rv32/etcnet.h
sed -i "77d" sw/rv32/etcnet.h
sed -i "77i #define	DEFAULT_ROUTERIP	IPADDR(${ROUTERIP[0]},${ROUTERIP[1]},${ROUTERIP[2]},${ROUTERIP[3]})" sw/rv32/etcnet.h
sed -i "148d" sw/host/udpsocket.cpp
sed -i "148i 	getaddrinfo(\"${HOSTIP[0]}.${HOSTIP[1]}.${HOSTIP[2]}.${HOSTIP[3]}\", portstr, &hints, &res);" sw/host/udpsocket.cpp
sed -i "208d" sw/host/testfft.cpp
sed -i "208i 	UDPSOCKET *skt = new UDPSOCKET(\"${DEVIP[0]}.${DEVIP[1]}.${DEVIP[2]}.${DEVIP[3]}\");" sw/host/testfft.cpp

Build Project

make

Program Board

Make sure that the ecp5-versa.cfg matches your board. If not, find the appropriate one in /usr/share/trellis/misc/openocd and link to that.

openocd -f ecp5-versa.cfg -c "transport select jtag; init; svf rtl/zipversa.svf; exit"

Start UART Connection

While we could have started this as a background process in the same terminal, it throws out a lot of garbage values which make it difficult to read the actual board responses. gnome-terminal wasn’t working for me so I ran it using xterm instead. Note that unless netuart is run, the board will not respond.

dnf -y install xterm
cd sw/host
xterm -hold  -e ./netuart /dev/$UBP&

Load FFT design for PicoRV

This checks the board flash memory to see if it matches the FFT program. If so, the command completes. Otherwise, sector by sector, the memory is erased and the FFT program is written.

./zipload ../rv32/fftmain

Note that once the flash memory has been programmed, the design will start executing. It will send out ARP packets to determine MAC addresses of the router. This does not mean that the actual FFT has started executing. Doing so requires running the host application (shown below). The other two example designs (pingtest, gettysburg) do not require a host application.

Also note that it is likely that there will be a couple of failed attempts to get the MAC address; this is fine. If, however, the xterm window opened earlier continues to print that the ARP-lookup failed, double check the network parameters specified in the beginning and run ./netstat to verify that the link is 1000Mbps.

Run FFT Application

Running ./testfft causes the board to send out ARP-packets again, this time trying to get the MAC address of the host machine. Then the testfft application sends the board four impulse functions, which computes the FFT, and returns the transforms back to the host.

./testfft

The figure below shows an example result of loading the FFT program onto the board and running the host application. Note that the errors seen are due to a mismatch in the operating frequencies of the RISCV core and the network controller. This should not significantly impact the observed waveforms in the next step. \

alt text

Display results returned by FPGA

Finally, we plot the waveforms for the four returned transforms.

dnf -y install octave
octave ./chkfftresults.m 

Other Example Designs

Execute from zipversa/sw/host. Ensure that netuart is running.

Gettysburg: ./zipload ../rv32/gettysburg This will print out the Gettysburg address on the netuart terminal.
Ping Test: ./zipload ../rv32/pingtest This will ping the router and print the results on the netuart terminal.

Simulation

From zipversa/sim/verilated/README.md: “To run the simulation , first kill any netuarts that might be running, and then run main_tb. main_tb may also be given an argument, which is the name of any (ELF) program to run within the CPU within. This program will then be loaded into design memory, and the design will begin as though it were already loaded at startup. For example, main_tb ../../sw/rv32/fftsimtest will run a simulated-based test of the internal FFT. A -d flag may also be used to generate a .vcd trace file as well for debugging purposes. Do be aware, this trace faile can become quite large. (I usually kill the simulation before it gets to 20GB.)”