-
Notifications
You must be signed in to change notification settings - Fork 12
AvaliableComponents
TOPAZ has been actively in use by galerna project from the University of Cantabria. For this reason, as well as offering the infrastructure inherited from SICOSYS, we also offer many of the code generated in the last few years, in the shape of router architectures, traffic generation improvements, or additional capabilities such as event counting for energy analysis purposes.
| Router ID | Year | Level of detail |
|---|---|---|
| Adaptive Bubble | 2001 | complex |
| Deterministic Bubble | 1998 | complex & simple |
| Virtual Channel (Dally) | 2001 | complex & simple |
| VCTM | 2008 | complex & simple |
| Rotary Router | 2009 | complex |
| Bufferless Router | 2010 | simple |
| Bidirectional Router | 2011 | simple |
| Buffered Crossbar | 1987 | complex |
| Ligero Router | 2013 | simple |
The messages generated to evaluate a network proposal can have different sources. They can be generated according to communication patterns usually found in parallel numerical algorithms. Messages can also be generated through profiling tools able to extract all the characteristics of communications in a specific application. Finally, network messages can be directly generated at another simulation tool connected to TOPAZ, running both in parallel.
TOPAZ is able to work with any of the sources of network traffic described here. Fastest results are provided by simulations driven by synthetic traffic patterns, but conclusions from network results are not always valid at system leve. The tradeoff between simulation speed and accuracy is clear in this case, being user's decission the utilization of the most appropriate solution according to its requirements.
Among all the results provided, TOPAZ returns the total count of most significant network events for every simulation. With these values, power/energy evaulation tools such as DSENT or MCPAT can be easily employed to extract power and energy numbers from a certain network configuration. Configuring both tools to emulate similar router structures, power and energy values (even area) for each event are easily extracted from DSENT for a wide range of technologies and operation frequencies.
Event counting provides more flexibility for the final user than the full integration of TOPAZ and DSENT. This way, power and energy values extracted from VLSI synthesis tools can be employed instead of DSENT models.
Multicast support is a desireable feature in any network simulator, due to the important performance advantages in network performance provided by hardware multicast support. Most of the router architectures in TOPAZ are able to work with multi-destination traffic. Both Bubble Routers, any Virtual Channel Router configuration (VCTM) and Rotary are able to deal with this kind of messages. In contrast, structures such as Bufferless router cannot provide multicast support due to correctness issues, while Bidirectional router still needs to implement this feature.
The implementation of multicast support in TOPAZ is performed through routing masks. A bit-vector carried by the message indicates which are its destinations (positions to 1 value). The length of this vector must be equal to the number of network nodes. Additionally, each output port of each router holds another mask of the same length, indicating in this case which destinations are reachable through that output port with the routing algorithm selected. Two simple bit-to-bit operations between message and port masks provide the necessary information to route these messages. It is important to notice that the value provided to port masks is going to determine the routing policy selected.
Multicast support is limited at this moment, and active work is being carried to improve its features in future TOPAZ versions. As this is intended to be a collaborative tool, external collaborations will be wellcome. Next section describes the main current limitations faced by multicast support in TOPAZ
Limitations
The most important limitation might reside on the maximum number of network nodes supported in multicast traffic. Currently masks are coded through a 64-bit data type in c++ (unsigned long long), which limits the number of nodes to 64. An 8x8 mesh network or a three dimensions 4x4x4 network is the maximum size allowed for multicast. NoC typical sizes have made this size enough for the moment, but in the near future larger networks will be required.
Messages with multiple destinations have been limited to 1-flit size. This is the common case in on-chip environments and keeps coding much simple, two important reasons for this limitation. The parent version of TOPAZ, SICOSYS, provided support for multi-flit multi-destination messages. This feature was limited to routers with packet-level flow control (Cut-Through). If demanded, this feature could be made available in TOPAZ for Bubble routers with relatively small effort.
For the moment, functions that automatically calculate port masks (both DOR and ADAP) only work for Bi-dimensional networks. Re-building this code to provide 3-D support is a work in progress.
Most routers provided with TOPAZ have support to work with three dimensional networks. Sepparate sgml files are provided to move from 2D to 3D routers. The only step required is to change the content of TPZSimul.ini file. Enjoy.
TOPAZ has been programmed as a multi-threaded simulator to be executed on multi-core systems with shared memory. We have used pthreads (POSIX standard for threads), so the pthread library is needed by this option to be used. By default multithreading is disabled. To enable it you need to selecta at compile time it. Just enable the 'PTOPAZ' environment variable on TOPAZ/mak/Makefile. Multithread has a "small" overhead in single thread simulations, therefore it is advised to be disabled in such cases. If the simulator has been compiled using this option, when you run a simulation with:
$shell ./TPZSimul -s REALLY_BIG_NETWORK -N <num_threads>
***** Using TOPAZ with <num_threads> threads *****
Multithreaded simulation is advised for very big networks or the use of TOPAZ in conjunction with external full system simulators. TOPAZ has been successfully used with network with more than one million of routers running up to 12 cores. The optimal number of threads is strongly dependent on the size of the network and memory or cores available in the server.
Implementation
The current implementation uses a master thread that controls a set of configurable slave threads, that actually run the simulation. The nodes and links of the simulated network are assigned linearly to each slave thread. In this way there is no restriction in the number of threads used in the simulation and geometry of the simulated network.
The simulation is performed following sequential process:
-
The slave threads run during the first half of the clock cycle the simulation of the routers.
-
A subsequent barrier (that involves only the slave threads) control the access to the simulation of the connections in the second half of the clock cycle.
-
At the end of the clock cycle another barrier, now that involves all the threads (including the master) controls the next clock cycle simulation, simulation finalization or the synchronization with external full system simulator.
All shared variables (such as statistics) has been managed to avoid unnecessary synchronization beyond the clock barriers. With multithreaded simulation the use of the statically preallocated flit POOL is not advised. Remember to disable it in Makefile.