Skip to content

Latest commit

 

History

History
379 lines (304 loc) · 18.2 KB

README.org

File metadata and controls

379 lines (304 loc) · 18.2 KB
Raw

GAPS TOF Firmware

Table of Contents

Organization

readout-firmware/
  ├── Top      : Top level project configuration files for HOG
  ├
  ├── common   : Files common to MT and RB design
  ├── bd       : Block diagram files
  ├── dma      : Files for DMA driver
  ├── drs      : Files for DRS4 control
  ├── ltb      : Files for Local Trigger Board
  ├── trg      : Files for Master Trigger Board
  ├── ip       : Vivado IP cores
  ├
  ├── regmap   : Register XML to VHDL tools
  ├── trigger  : Master Trigger Design
  ├── util     : Helper scripts for build system
  ├── xdc      : Xilinx XDC constraint files
  ├── Hog      : Submodule containing the HOG project
  ├── Projects : Auto-generated directory containing the Vivado projects
  └── plnx     : PetaLinux source files and docs

Software dependencies

  • Xilinx tools
    • Vivado 2020.1
  • Build system
    • git
    • make
    • python3.6+
  • Optional
    • emacs

Register Access

Firmware registers mapped to a Wishbone-like A32/D32 interface, which is accessed through an AXI⟷Wishbone Bridge

From the Zynq, it appears as a memory-mapped into an address space, starting at Base Address=0x8000_0000 (with a maximum address of 0x8000_FFFF, giving an effective address space of 16 bits or 65536 registers).

An interactive client can read and write from registers, by calling reg_interface.py

A python library rw_reg.py allows for register access by name

Address Table

The address table is defined in a “templated” XML file: registers.xml

A convenient document describing the address table can be seen at:

To update the address table in the project, make edits directly to rb_registers.xml or mt_registers.xml, then build using

make reg

Building the Firmware

Special note for 2020.1 and hardware generation

Vivado 2020.1 cannot be used with Hog to export hardware .xsa out of the box due to a bug Xilinx shipped in that version. Implement either of two work arounds below.

  1. Stock 2020.1: ensure bitstream successfully generated. Open BD or Implemented Design: File->Export->Export Hardware (Platform type=Fixed), next-> Check include bitstream, next->set file name/path->finish.
  2. Fix 2020.1 with Xilinx “tactical patch”. See https://www.xilinx.com/support/answers/75210.html. If using this option, no further steps are required when using the Hog build system.

Build instructions

This firmware is using the HOG framework as a build system:

Clone project recursively to pull all HOG scripts

git clone --recursive https://gitlab.com/ucla-gaps-tof/firmware.git

The firmware can then be built with:

make readout_board_zx3
make readout_board_zx2
make trigger_board

a list of Make targets will be displayed by typing:

make

To use any of these commands you need to make sure that Vivado is in your path, i.e. if you type Vivado into the terminal it should open Vivado.

To make sure the environment variables always set, modify your .bashrc to contain a similar command as below, depending on where you installed Vivado.

source /opt/Xilinx/Vivado/2020.1/settings64.sh

The scripts use only very primitive bash and otherwise use the Vivado TCL shell, so there should be no external dependencies.

Best practice is for released builds the entire repo should be cloned from scratch and built from the clean cloned repository to ensure that no files are missing, the build directory is clean, and so on.

DRS Data Flow

./drs/data-flow.svg

RB Dataformat

FieldLenDescription
HEAD[15:0]0xAAAA
STATUS[15:0][0] = empty event fragment
[1] = drs was busy (lost trigger)
[2] = locked
[3] = locked (past second)
[15:4] = 12 bit FPGA temperature
LEN[15:0]length of packet in 2 byte words
ROI[15:0]size of region of interest
DNA[15:0]Zynq7000 Device DNA bits [63:48] ^ [47:32] ^ [31:16] ^ [15:0]
RSVD0[15:0]Reserved
RAT_HOUSEKEEPING[31:0]Software Defined
FW_HASH[15:0]First 16 bits of Git Hash
ID[15:0][15:8] = readout board ID
[7:6] = reserved
[5:0] = MTB link id
CH_MASK[15:0][8:0] = Channel Enable Mask ‘1’=ON
[15:9] reserved
EVENT_CNT[31:0]Event ID Received From Trigger
DTAP[15:0]DTAP Frequency in 100Hz
DRS_TEMP[15:0]DRS temperature, written by software
TIMESTAMP[47:0]# of 33MHz clocks elapsed since resync
PAYLOAD0 to XXXX wordsHEADER[15:0] = Channel ID
----- begin block data -----
DATA[13:0] = ADC data
DATA[14] = Cell Sync Err
DATA[15] = Channel Sync Err
----- end block: len = ROI words -----
TRAILER[31:0] = crc32
STOP CELL[15:0]Stop cell of the DRS
CRC32[31:0]Packet CRC (excluding Trailer)
TAIL[15:0]0x5555

Master Trigger to RB Data Format

FieldLenDescription
START[0:0]‘1’ to start
TRIGGER[0:0]‘1’ initiates a trigger; ‘0’ for an event fragment
CH_MASK[7:0]bitfield set to ‘1’ to readout a channel
EVENT_ID[31:0]Event ID
CMD[1:0]3=resync
CRC8[7:0]lfsr(7:0)=1+x^{2+x}4+x^{6+x}7+x^8

Local Trigger Data Format

https://gaps1.astro.ucla.edu/wiki/gaps/index.php?title=Local_Trigger_Board_Operation

The LT to MT link consists of 2 LVDS pairs per MT.

This can be optionally expanded to 3 pairs with an unstuffed isolator, in case of bandwidth requirements.

Each MT pair transmits at 200 Mbps, on an asynchronous clock.

The data format on each pair consists of a start bit, followed by the data payload.

The entire data packet consists of 16 bits of payload (8 channels of low, medium, and high threshold).

Since it is divided into 2 links, this means 8 bits / link + 2 start bits per trigger.

//      | no hit| thr0 | thr1 | thr2
//----------------------------------
// bit0 |    0  |  0   |  1   |  1
// bit1 |    0  |  1   |  0   |  1

//LINK0  = START bit +paddles bit 0 (9 bits total)
//LINK1 = START bit +paddles bit 1 (9 bits total)

Master Trigger DAQ Data Format

FieldLenDescription
HEADER[31:0]0xAAAA_AAAA
EVENT_CNT[31:0]Event counter
TIMESTAMP[31:0]Internal timestamp at the time of trigger (1 unit = 10 ns)
TIU_TIMESTAMP[31:0]Timestamp at the edge of the TIU GPS (1 unit = 10 ns)
TIU_GPS[47:0]Second received from the TIU (format?)
TRIG_SOURCE[15:0]Bitmask showing all trigger sources
0-3: reserved
4: track umb central trigger
5: gaps trigger
6: any trigger
7: forced trigger
8: track trigger
9: central track trigger
10: configurable trigger
11: track_umb_central_is_global
12: read_all_channels
13: track_central_is_global
14: track_trigger_is_global
15: any_hit_trigger_is_global
RB_MASK_0[31:0]RBs [31:0] are triggered
RB_MASK_1[31:0]RBs [49:32] are triggered
BOARD_MASK[31:0]25 bits indicating boards which local trigger boards are read out
HITS–Variable sized, 16 bits / board * n_boards
PAD[15:0]Optional, only here if the # of LTBs read is odd
CRC[31:0]CRC32, same polynomial as the RB
TRAILER[31:0]0x5555_5555

Master Trigger External IO

SignalAssignmentDescription
TIU BusyEXT_IN0LVDS IN: Busy acknowledgment from the TIU. Trigger should be deasserted only after busy is received.
TIU TimecodeEXT_IN1LVDS IN: Asynchronous serial input containing the GPS timestamp.
TIU Event IDEXT_OUT0LVDS OUT: Asynchronous serial output containing the event ID
TIU TriggerEXT_OUT1LVDS OUT: Trigger output from the MT to TIU. Asynchronous level which should not be deasserted until the BUSY is received back from the TIU
PinFunction
ext_io(0)ext_trigger input
ext_io(1)–
ext_io(2)SDA
ext_io(3)SCL
ext_io(5)hk_ext_clk;
ext_io(6);hk_ext_miso
ext_io(7)hk_ext_mosi;
ext_io(8)hk_ext_cs_n(0);
ext_io(9)hk_ext_cs_n(1);
ext_io(10)trigger mirror
ext_io(12)–
ext_io(13)–

Trigger Latency

ItemDelay (ns)Source
LTB Analog Frontend (AD8014)0.5estimate
LTB Analog Frontend (ADCMP601)3.5datasheet
LTB Input Routing1.5estimate
LTB FPGA Input Delay3.6Vivado timing report
LTB FPGA Firmware50measurement
LTB FPGA Output Delay4.2Vivado timing report
LTB Output Routing1estimate
LTB Differential Buffer (DS90LV027A)1.5datasheet
Cable (10 ft)13measurement
DSI Isolator (ADN4654)4datasheet
DSI to MTB FPGA Routing1.5estimate
MTB FPGA Input Delay2timing report
MTB FPGA LTB Deserialization459 bits @ 200 MHz
MTB FPGA Firmware60measurement (ILA)
MTB FPGA Output Delay3.9timing report
MTB FPGA to Output Routing1estimate
MTB Output Buffer (DS90LV031ATMTC)2datasheet
Total223.2

Gitlab runner registration

Some simple instructions for registering a Gitlab runner

  1. Install gitlab-runner
  2. Execute gitlab-runner register
  3. At the prompt of “Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):”, enter: https://gitlab.com/
  4. At the prompt of “Please enter the gitlab-ci token for this runner:”, enter the token that you get from Settings -> CI/CD -> Runners –> Set up a specific Runner manually.
  5. At the prompt of “Please enter the gitlab-ci description for this runner:”, give it a name:
  6. At the prompt of “Please enter the gitlab-ci tags for this runner (comma separated):”, enter hog
  7. At the prompt of: “Please enter the executor: docker+machine, docker-ssh+machine, kubernetes, parallels, virtualbox, docker-ssh, shell, ssh, custom, docker:”, enter: shell

Now you can simply start the runner (gitlab-runner run). Make sure Vivado is in the path.

Updating mapping

Note: this requires an installation of babashka https://github.com/babashka/babashka

  1. clone the firmware
  2. update the mapping
cd trigger/src/trg
make
  1. Commit the updated files, push to devel of the repo
git add mapping.csv rb_map.vhd trigger.vhd
git commit -m "mtb: Update link mapping"
  1. Open a merge request from devel -> master

Updating trigger

If you send me a new version of the trigger definitions it is trivial to recompile firmware with whatever you want. The current definitions I got from you are here: https://gitlab.com/ucla-gaps-tof/firmware/-/blob/develop/trigger/src/trg/generate_triggers.bb For reference in case I am not available sometime and you need things in a rush the to update the new firmware is:

  1. clone the firmware
  2. edit generate_triggers.bb to your liking
  3. run Make in the same directory. You need awk and babashka installed
  4. Commit the updated files, push to devel of the repo
  5. Open a merge request from devel -> master