SIFIVE FREEDOM

Verilog RTL, RISC-V, and more

SIFIVE FREEDOM


I. Hardware

SiFive freedom demo on VC707 FPGA board is using the U540 core with ISA of rv64gc.

Original repo. Modified repo

To build on VC707 FPGA, you need Xilinx Vivado design software. Install it in Initial Setup: II.h).

I. a) Build

Git clone:

$ git clone https://github.com/thuchoang90/freedom.git	
$ cd freedom/
$ git checkout keystone-fpga			#commit 76dd152a on 5-Nov-2019

this will 'submodule update' without download the toolchain again, and also patch the fpga-shells submodule afterward
$ . update.sh

(i) Make

Before ‘make’, remember to set the environment (see I. b) Other Utilities) simply by: $ . setenv.sh

The format for ‘make’ command is:

This is for creating .v files:
$ make [CONFIG] [MODEL] [BOOTROM_DIR] [BUILD_DIR] -f Makefile.vc707-u500devkit verilog -j`nproc`

This is for Vivado build:
$ make [CONFIG] [MODEL] [BOOTROM_DIR] [BUILD_DIR] -f Makefile.vc707-u500devkit mcs -j`nproc`

The [CONFIG] option is for selecting the frequency:

CONFIG=DevKitU500FPGADesign_WithDevKit50MHz (default value)
       DevKitU500FPGADesign_WithDevKit100MHz
       DevKitU500FPGADesign_WithDevKit125MHz
       DevKitU500FPGADesign_WithDevKit150MHz

The [MODEL] option is for selecting the PCIe option:

with PCIe build (default value):
MODEL=VC707PCIeShell

without PCIe build:
MODEL=VC707BaseShell

The [BOOTROM_DIR] option is to specify the bootrom directory that you want to use:

using the sifive's sdboot (default value):
BOOTROM_DIR=`pwd`/bootrom/sdboot

using the keystone's zsfl+fsbl boot:
BOOTROM_DIR=`pwd`/bootrom/freedom-u540-c000-bootloader

The [BUILD_DIR] option is to specify the build directory:

BUILD_DIR=`pwd`/builds/<name>
default value: BUILD_DIR=`pwd`/builds/vc707-u500devkit

Example:

To build with PCIe, 125MHz, using sifive's sdboot:
$ make CONFIG=DevKitU500FPGADesign_WithDevKit125MHz BUILD_DIR=`pwd`/builds/vc707-u500devkit-withpcie-sdboot -f Makefile.vc707-u500devkit verilog -j`nproc`
$ make CONFIG=DevKitU500FPGADesign_WithDevKit125MHz BUILD_DIR=`pwd`/builds/vc707-u500devkit-withpcie-sdboot -f Makefile.vc707-u500devkit mcs -j`nproc`

To build without PCIe, 150MHz, using sifive's sdboot:
$ make CONFIG=DevKitU500FPGADesign_WithDevKit150MHz MODEL=VC707BaseShell BUILD_DIR=`pwd`/builds/vc707-u500devkit-nopcie-sdboot -f Makefile.vc707-u500devkit verilog -j`nproc`
$ make CONFIG=DevKitU500FPGADesign_WithDevKit150MHz MODEL=VC707BaseShell BUILD_DIR=`pwd`/builds/vc707-u500devkit-nopcie-sdboot -f Makefile.vc707-u500devkit mcs -j`nproc`

To build with PCIe, 125MHz, using keystone's zsbl-fsbl boot:
$ make CONFIG=DevKitU500FPGADesign_WithDevKit125MHz BOOTROM_DIR=`pwd`/bootrom/freedom-u540-c000-bootloader BUILD_DIR=`pwd`/builds/vc707-u500devkit-withpcie-keystoneboot -f Makefile.vc707-u500devkit verilog -j`nproc`
$ make CONFIG=DevKitU500FPGADesign_WithDevKit125MHz BOOTROM_DIR=`pwd`/bootrom/freedom-u540-c000-bootloader BUILD_DIR=`pwd`/builds/vc707-u500devkit-withpcie-keystoneboot -f Makefile.vc707-u500devkit mcs -j`nproc`

To build without PCIe, 150MHz, using keystone's zsbl-fsbl boot:
$ make CONFIG=DevKitU500FPGADesign_WithDevKit150MHz MODEL=VC707BaseShell BOOTROM_DIR=`pwd`/bootrom/freedom-u540-c000-bootloader BUILD_DIR=`pwd`/builds/vc707-u500devkit-nopcie-keystoneboot -f Makefile.vc707-u500devkit verilog -j`nproc`
$ make CONFIG=DevKitU500FPGADesign_WithDevKit150MHz MODEL=VC707BaseShell BOOTROM_DIR=`pwd`/bootrom/freedom-u540-c000-bootloader BUILD_DIR=`pwd`/builds/vc707-u500devkit-nopcie-keystoneboot -f Makefile.vc707-u500devkit mcs -j`nproc`

To clean and build again:

(ii) Notes

1: Guide for program & debug on VC707 can be found here.

2: Internal coding structure of the SiFive Freedom folder can be found here.

3: The maximum frequency for the VC707 board with PCIE is 125MHz, and without PCIE is 150MHz.

4: Sometime the make mcs end with timing error and did not continue to generate the final mcs files for the flash programming, but still, it did generate the .bit file. Then, we can manually generate the .mcs from the .bit:

cd to the build folder
$ cd builds/<name>/obj/

for VC707PCIeShell:
$ vivado -nojournal -mode batch -source ../../../fpga-shells/xilinx/common/tcl/write_cfgmem.tcl -tclargs vc707 VC707PCIeShell.mcs VC707PCIeShell.bit

for VC707BaseShell:
$ vivado -nojournal -mode batch -source ../../../fpga-shells/xilinx/common/tcl/write_cfgmem.tcl -tclargs vc707 VC707BaseShell.mcs VC707BaseShell.bit

5: Built files are under **builds//obj/**. The important built files are:

VC707Shell.v							the verilog source code
VC707Shell.mcs and VC707Shell.prm		the two files for flash programming
VC707Shell.bit						the bitstream file for direct programming

I. b) Other utilities

To quickly set build environment:

$ vi setenv.sh
and change the correct corresponding paths in your machine

Then from this point forward, you can auto set your environment by simply:
$ . setenv.sh

If you want to change the ROM size:

BootRom size is declared in the file:
	rocket-chip/src/main/scala/devices/tilelink/MaskROM.scala

On the line 12:
	case class MaskROMParams(address: BigInt, name: String, depth: Int = 2048, width: Int = 32)

Change the <depth> value to change the size.

If you want to change the modules’ addresses, number of CPU cores, etc:

The declarations are in the file:
	src/main/scala/unleashed/DevKitConfigs.scala

But remember to change the addresses on the software as well. These files:
	bootrom/sdboot/include/platform.h
	bootrom/sdboot/linker/memory.lds
	bootrom/freedom-u540-c000-bootloader/sifive/platform.h
	bootrom/freedom-u540-c000-bootloader/memory_vc707.lds

TODO: script to auto update ROM size, modules’ addresses, and number of CPU to software

I. c) Use with Idea IntelliJ

Guide to install Idea IntelliJ is in Initial Setup: II.e).

To import the freedom folder to the Idea IntelliJ tool:

java -jar /home/ubuntu/project/freedom/rocket-chip/sbt-launch.jar ++2.12.4 "runMain freechips.rocketchip.system.Generator /home/ubuntu/project/freedom/builds/chip uec.nedo.chip ChipShell uec.nedo.chip ChipDesignTop"
(the content may be differ on your machine)

In the first time 'make', it usually appears near after this line:
	[success] Total time: xxx s, completed xxx, xxx, xx:xx:xx
In the NOT first time 'make', it usually appears right away

To debug with the Idea IntelliJ tool:


II. Software

II. a) Without KeyStone

Original repo. Modified repo.

TODO: currently, this way uses the prebuilt toolchain of gcc 7. In the future, this need to be upgraded to the current mainstream toolchain of sifive-freedom, gcc 9.2. Then after that, we can avoid to download the prebuilt toolchain again.

(i) Git clone

$ git clone https://github.com/mcd500/freedom-u-sdk.git
$ cd freedom-u-sdk
$ git checkout linux_u500vc707devkit_config		#commit 29fe529f on 12-Apr-2019
$ git submodule update --init --recursive

Note: sometimes the riscv-gnu-toolchain submodule fails to clone. If that happened, we have to git clone it manually:

$ rm -rf riscv-gnu-toolchain		#first, remove the corrupted folder
$ git clone https://github.com/riscv/riscv-gnu-toolchain.git	#then clone a new
$ cd riscv-gnu-toolchain/
$ git checkout b4dae89
$ git submodule update --init --recursive
$ cd ../						#get back to the top folder of freedom-u-sdk

(ii) Make the bbl.bin

$ unset RISCV				#make sure that there is nothing on the RISCV variable

export your toolchains to the PATH (both the elf & linux toolchains)
$ export PATH=/opt/riscv64gc/bin/:$PATH

then 'make' with PCIE:
$ make -j`nproc` BOARD=vc707devkit

or 'make' without PCIE:
$ make -j`nproc` BOARD=vc707devkit_nopci

(iii) Write SD card & boot on

$ sudo dd if=work/bbl.bin of=/dev/sdX bs=4096 conv=fsync
where X is the number of the USB device

Put in the SD card to the board, program the board, then wait for the board to boot on. Communicate with the board via UART:

$ sudo minicom -b 115200 -D /dev/ttyUSBx
where x is the number of connected USB-UART
Login by the id of 'root' and the password of 'sifive'.

II. b) SD card with KeyStone

Ref link for the KeyStone project.

(i) Partition the SD card

Use the gptfdisk tool to create 4 partitions in the SD card.

If you don’t have the gptfdisk tool, then do the followings to install it:

$ git clone https://github.com/tmagik/gptfdisk.git	#branch master commit 3d6a1587 on 9-Dec-2018
$ cd gptfdisk/
$ make -j`nproc`

To use gptfdisk:

$ cd gptfdisk/	#go to gptfdisk folder
$ sudo ./gdisk /dev/sd?	#point to the sd-card

Some commands:
$ p	# print partitions information
$ d	# delete partition
$ n	# create new partition
$ w	# write partition

The partitions after created:

Number	Start (sector)	End (sector)	Size			Code	Name
1		2048		67583		32.0 MiB		5202	SiFive bare-metal (...
2		264192		15759326	7.4 GiB		8300	Linux filesystem
4		67584		67839		128.0 KiB	5201	SiFive FSBL (first-...

(ii) Prepare BBL & FSBL

For the bbl.bin:

Follow the instruction to make the bbl.bin of the Keystone & Keystone demo.

The bbl.bin is for the 1st partition of the SD card:

$ cd <keystone folder>				#cd to your keystone folder
$ sudo dd if=hifive-work/bbl.bin of=/dev/sdX1 bs=4096 conv=fsync
where the X1 is the 1st partition of the USB device

For the fsbl.bin:

After the hardware make (section I. a) above), there is a vc707fsbl.bin file inside the folder bootrom/freedom-u540-c000-bootloader/. That file is for the 4th partition of the SD card:

$ cd <your freedom folder>
$ cd bootrom/freedom-u540-c000-bootloader/
$ sudo dd if=vc707fsbl.bin of=/dev/sdX4 bs=4096 conv=fsync
where the X4 is the 4th partition of the USB device

(iii) Boot on & run the test

Finally, put in the SD card to the board, program the board, then wait for the board to boot on. Communicate with the board via UART:

$ sudo minicom -b 115200 -D /dev/ttyUSBx
where x is the number of connected USB-UART
Login by the id of 'root' and the password of 'sifive'.

To run the keystone test:

$ insmod keystone-driver.ko		#install driver
$ time ./tests/tests.ke 			#okay if the 'Attestation report SIGNATURE is valid' is printed
$ cd keystone-demo/			#go to the keystone-demo test
$ ./enclave-host.riscv &			#run host in localhost
$ ./trusted_client.riscv localhost	#connect to localhost and test
okay if the 'Attestation signature and enclave hash are valid' is printed

BOTTOM PAGE

Back Next
Keystone (using CMake) for RV64 Build VexRiscv 32-bit MCU