CONNECT Note (2) - Setup

Run CONNECT testbench

Thank Siddharth for providing the notes!

1. Clone the repository from Crossroads GitHub

2. To build a basic double ring network with 4 ports, 256 bit flits, 2 virtual channels, and peek flow control (not credit-based, although we will want credit based later on), do python gen_network.py -t double_ring -w 256 -n 4 -v 2 -d 8 --router_type=vc --peek_flow_control --use_virtual_links --gen_rtl

3. cd into build (it will make that directory for you when you do --gen_rtl). This contains a bunch of .v files that make up the generated network. Create a new file inside build called connect_parameters.v and fill it like so (NUM_USER_SEND_PORTS/NUM_USER_RECV_PORTS: how many ports did you give (the -n param); NUM_VCS: how many virtual channels (the -v param); FLIT_DATA_WIDTH: width of the network (the -w param)

   `define NUM_USER_SEND_PORTS 4
   `define NUM_USER_RECV_PORTS 4
   `define NUM_VCS 2
   `define FLIT_DATA_WIDTH 256
   

4. Inside build, make a file called run_modelsim.sh and paste the script from below inside it, then do chmod +x run_modelsim.sh (A Short Intro to ModelSim Verilog Simulator)

   vlib work
   vlog *.sv *.v
   vsim -c -do "run -all" CONNECT_testbench_sample_peek
   

5. Inside build, ./run_modelsim.sh and see modelsim compile and run and eventually two flits injected into the network and two flits ejected from it

Top Module & Interface

The following module can be seen as the top module under testbench with peek flow control.

module mkNetworkRealSimple(NetworkSimple);

  String name = "Simple Network";

  // Vector of input and output interfaces to connect network clients
  Vector#(NumUserSendPorts, InPortSimple)  send_ports_ifaces;
  Vector#(NumUserRecvPorts, OutPortSimple) recv_ports_ifaces;
  Vector#(NumUserRecvPorts, RecvPortInfo)  recv_ports_info_ifaces;
  
  function get_rt( Integer i );
  `ifdef USE_VOQ_ROUTER
      return mkVOQRouterSimple(i); 
  `elsif USE_IQ_ROUTER
      return mkIQRouterSimple(i); 
  `else
      return mkRouterSimple(i);
  `endif
  endfunction
  
  function get_port_info_ifc( Integer id );
    let recv_port_info_ifc =
      interface RecvPortInfo
        method UserRecvPortID_t getRecvPortID;
          return fromInteger(id);
	      endmethod
      endinterface;
    return recv_port_info_ifc;
  endfunction

  // Declare router and traffic source interfaces
  Vector#(NumRouters, RouterSimple)      routers <- genWithM( get_rt );
  Vector#(NumLinks, ConnectPorts)        links;

  interface send_ports  = send_ports_ifaces;
  interface recv_ports  = recv_ports_ifaces;
  interface recv_ports_info = recv_ports_info_ifaces; 

endmodule

The interface of this module is in NetworkTypes.bsv.

interface NetworkSimple;
  interface Vector#(NumUserSendPorts, InPortSimple)     send_ports;
  interface Vector#(NumUserRecvPorts, OutPortSimple)    recv_ports;
  interface Vector#(NumUserRecvPorts, RecvPortInfo)  recv_ports_info; // Used by clients for obtaining response address
endinterface

Internal definitions are in NetworkExternalTypes.bsv.

////////////////////////////////////////////////
// Simpler InPort and OutPort interfaces
//   - Routers only exchange notFull signals, instead of credits
// Implemented by routers and traffic sources                      
////////////////////////////////////////////////                   
interface InPortSimple;                             
  (* always_ready *) method Action putFlit(Maybe#(Flit_t) flit_in);
  (* always_ready *) method ActionValue#(Vector#(NumVCs, Bool)) getNonFullVCs;
endinterface

interface OutPortSimple;
  (* always_ready *) method ActionValue#(Maybe#(Flit_t)) getFlit();
  (* always_ready *) method Action putNonFullVCs(Vector#(NumVCs, Bool) nonFullVCs);
endinterface

// Used by clients to obtain response address
interface RecvPortInfo;
  (* always_ready *) method UserRecvPortID_t getRecvPortID;
endinterface

Definition of Flit_t.

typedef struct {
  Bool              is_tail;  // only required for multi-flit packets
  UserRecvPortID_t  dst;
  VC_t              vc;  
  FlitData_t        data;     // payload of flit
} Flit_t
  deriving(Bits, Eq);

Implement AXI Interface

We develop AXI4 standard protocol interface with SystemVerilog.

1. Global constants are defined in AXI4Package.sv and encapsulated in axi4_pkg package.

2. The interface is defined in AXI4Interface.sv, called axi_interface, where all the signals are defined, including those that are not used yet (e.g., axcache or axprot).

3. The modules (AXI4MasterToInPortSimple and AXI4SlaveToOutPortSimple) to bridge AXI4 interface and CONNECT network interfaces are defined in AXI4Bridge.sv, and the logics are handled by a simple FSM.

4. A wrapper (NetworkIdealSimpleAXI4Wrapper) is defined outside mkNetworkSimple module and can be used by a test bench (testbench_sample_peek_axi4.sv).

Results

# @  8: Injecting flit (data = 0000000000001234) into send port 0
# @ 11: Injecting flit (data = 0000000000002345) into send port 0
# @ 12: Ejecting flit (data = 0000000000001234) at receive port 1
# @ 16: Ejecting flit (data = 0000000000002345) at receive port 2

Notes

1. The width of flit data needs to be consistent with AXI data width. Meanwhile we simply use axaddr to find the receive port, but can be extended to a more complicated encoding.

2. At present we only implement the write logic, i.e., writing some data to a send port and to be received at a receive port.

3. The AXI request and response themselves can/should be transferred through the network?

4. Next step can be connect a CPU core to one endpoint of the network, and memory to another endpoint? If it works, connect multiple cores?

Reference

1. BSV by example, a good intro book to Bluespec.

2. Piccolo, a 3-stage in-order CPU core developed with Bluespec.