Targets ======= FireSim generates SoC models by transforming RTL emitted by a Chisel generator, such as the Rocket SoC generator. Subject to conditions outlined in :ref:`rtl-restrictions`, if it can be generated by Chisel, it can be simulated in FireSim. .. _rtl-restrictions: Restrictions on Target RTL -------------------------- Current limitations in Golden Gate place the following restrictions on the (FIR)RTL that can be transformed and thus used in FireSim: #. The top-level module must have no inputs or outputs. Input stimulus and output capture must be implemented using target RTL or target-to-host Bridges. #. All target clocks must be generated by a single ``RationalClockBridge``. #. Black boxes must be "clock-gateable" by replacing its input clock with a gated equivalent which will be used to stall simulation time in that module. a. As a consequence, target clock-gating cannot be implemented using black-box primitives, and must instead be modeled by adding clock-enables to all state elements of the gated clock domain (i.e., by adding an enable or feedback mux on registers to conditionally block updates, and by gating write-enables on memories). #. Asynchronous reset must only be implemented using Rocket Chip's black-box async reset. These are replaced with synchronously reset registers using a FIRRTL transformation. .. _verilog-ip: -------------------- Including Verilog IP -------------------- FireSim now supports target designs that incorporate Verilog IP using the standard ``BlackBox`` `interface from Chisel `_. For an example of how to add Verilog IP to a target system based on Rocket Chip, see the `Incorporating Verilog Blocks `_ section of the Chipyard documentation. #. For the transform to work, the Chisel Blackbox that wraps the Verilog IP must have input clocks that can safely be clock-gated. #. The compiler that produces the decoupled simulator ("FAME Transform") automatically recognizes such blackboxes inside the target design. #. The compiler automatically gates each clock of the Verilog IP to ensure that it deterministically advances in lockstep with the rest of the simulator. #. This allows any Verilog module, subject to the constraint above, to be instantiated anywhere in the target design using the standard Chisel Blackbox interface. ---------------------------- Multiple Clock Domains ---------------------------- FireSim can support simulating targets that have multiple clock domains. As stated above, all clocks must be generated using a single ``RationalClockBridge``. For most users the default FireSim test harness in Chipyard will suffice, if you need to define a custom test harness instantiate the ``RationalClockBridge`` like so: .. literalinclude:: ../../sim/src/main/scala/midasexamples/TrivialMulticlock.scala :language: scala :start-after: DOC include start: RationalClockBridge Usage :end-before: DOC include end: RationalClockBridge Usage Further documentation can be found in the source (``sim/midas/src/main/scala/midas/widgets/ClockBridge.scala``). ================= The Base Clock ================= By convention, target time is specified in cycles of the `base clock`, which is defined to be the clock of the ``RationalClockBridge`` whose clock ratio (multiplier / divisor) is one. While we suggest making the base clock the fastest clock in your system, as in any microprocessor-based system it will likely correspond to your core clock frequency, this is not a constraint. ============ Limitations: ============ * The number of target clocks FireSim can simulate is bounded by the number of BUFGCE resources available on the host FPGA, as these are used to independently clock-gate each target clock. * As its name suggests, the ``RationalClockBridge`` can only generate target clocks that are rationally related. Specifically, all requested frequencies must be expressable in the form: .. math:: f_{i} = \frac{f_{lcm}}{k_{i}} Where, * :math:`f_{i}` is the desired frequency of the :math:`i^{th}` clock * :math:`f_{lcm}`, is the least-common multiple of all requested frequencies * :math:`k_{i}` is a 16-bit unsigned integer An arbitrary frequency can be modeled using a sufficiently precise rational multiple. Golden Gate will raise a compile-time error if it cannot support a desired frequency. * Each bridge module must reside entirely within a single clock domain. The Bridge's target interface must contain a single input clock, and all inputs and outputs of the bridge module must be latched and launched, respectively, by registers in the same clock domain. .. _generating-different-targets: Target-Side FPGA Constraints ---------------------------- FireSim provides utilities to generate Xilinx Design Constraints (XDC) from string snippets in target's Chisel. Golden Gate collects these annotations and emits separate xdc files for synthesis and implementation. See :ref:`fpga-build-files` for a complete listing of output files used in FPGA compilation. ----------------------------- RAM Inference Hints ----------------------------- Vivado generally does a reasonable job inferring embedded memories from FireSim-generated RTL, though there are some cases in which it must be coaxed. For example: * Due to insufficient BRAM resources, you may wish to use URAM for a memory that'd infer as BRAM. * If Vivado can't find pipeline registers to absorb into a URAM or none exist in the target, you may get an warning like:: [Synth 8-6057] Memory: "" defined in module: "" implemented as Ultra-Ram has no pipeline registers. It is recommended to use pipeline registers to achieve high performance. Since Golden Gate modifies the module hierarchy extensively, it's highly desirable to annotate these memories in the Chisel source so that their hints may move with the memory instance. This is a more robust alternative to relying on wildcard / glob matches from a static XDC specification. Chisel memories can be annotated *in situ* like so: .. literalinclude:: ../../sim/midas/targetutils/src/test/scala/RAMStyleHintSpec.scala :language: scala :start-after: DOC include start: Basic RAM Hint :end-before: DOC include end: Basic RAM Hint Alternatively, you can "dot-in" (traverse public members of a Scala class hierarchy) to annotate a memory in a submodule. Here's an example: .. literalinclude:: ../../sim/midas/targetutils/src/test/scala/RAMStyleHintSpec.scala :language: scala :start-after: DOC include start: RAM Hint From Parent :end-before: DOC include end: RAM Hint From Parent These annotations can be deployed anywhere: in the target, in bridge modules, and in internal FireSim RTL. The resulting constraints should appear the synthesis xdc file emitted by Golden Gate. For more information see the ScalaDoc for ``RAMStyleHint`` or read the source located at: :gh-file-ref:`sim/midas/targetutils/src/main/scala/midas/xdc/RAMStyleHint.scala`. Provided Target Designs ----------------------- ----------------------------- Target Generator Organization ----------------------------- FireSim provides multiple `projects`, each for a different type of target. Each project has its own chisel generator that invokes Golden Gate, its own driver sources, and a makefrag that plugs into the Make-based build system that resides in ``sim/``. These projects are: 1. **firesim** (Default): rocket chip-based targets. These include targets with either BOOM or rocket pipelines, and should be your starting point if you're building an SoC with the Rocket Chip generator. 2. **midasexamples**: the Golden Gate example designs. Located at :gh-file-ref:`sim/src/main/scala/midasexamples`, these are a set of simple chisel circuits like GCD, that demonstrate how to use Golden Gate. These are useful test cases for bringing up new Golden Gate features. 3. **bridges**: tests for firesim-lib bridges. These have more dependencies and involve more logic than `midasexamples`. 4. **fasedtests**: designs to do integration testing of FASED memory-system timing models. Projects have the following directory structure: .. code-block:: text sim/ ├-Makefile # Top-level makefile for projects where FireSim is the top-level repo ├-Makefrag # Target-agnostic makefrag, with recipes to generate drivers and RTL simulators ├-src/main/scala/{target-project}/ │ └─Makefrag # Defines target-specific make variables and recipes. ├-src/main/cc/{target-project}/ │ ├─{driver-csrcs}.cc # The target's simulation driver, and sofware-model sources │ └─{driver-headers}.h └-src/main/makefrag/{target-project}/ ├─Generator.scala # Contains the main class that generates target RTL and calls Golden Gate └─{other-scala-sources}.scala ---------------------------- Specifying A Target Instance ---------------------------- To generate a specific instance of a target, the build system leverages five Make variables: 1. ``TARGET_PROJECT``: this points the Makefile (`sim/Makefile`) at the right target-specific Makefrag, which defines the generation and metasimulation software recipes. The makefrag for the default target project is defined at ``sim/src/main/makefrag/firesim``. 2. ``DESIGN``: the name of the top-level Chisel module to generate (a Scala class name). These are defined in FireChip Chipyard generator. 3. ``TARGET_CONFIG``: specifies a ``Config`` instance that is consumed by the target design's generator. For the default firesim target project, predefined configs are described in in the FireChip Chipyard generator. 4. ``PLATFORM_CONFIG``: specifies a ``Config`` instance that is consumed by Golden Gate and specifies compiler-level and host-land parameters, such as whether to enable assertion synthesis, or multi-ported RAM optimizations. Common platform configs are described in ``firesim-lib/sim/src/main/scala/configs/CompilerConfigs.scala``). 5. ``PLATFORM``: this points the Makefile (`sim/Makefile`) at the right FPGA platform to build for. This must correspond to a platform defined at :gh-file-ref:`platforms`. ``TARGET_CONFIG`` and ``PLATFORM_CONFIG`` are strings that are used to construct a ``Config`` instance (derives from RocketChip's parameterization system, ``Config``, see the `CDE repo `_). These strings are of the form "{..._}{\_}". Only the final, base class name is compulsory: class names that are prepended with "_" are used to create a compound Config instance. .. code-block:: scala // Specify by setting TARGET_CONFIG=Base class Base extends Config((site, here, up) => {...}) class Override1 extends Config((site, here, up) => {...}) class Override2 extends Config((site, here, up) => {...}) // Specify by setting TARGET_CONFIG=Compound class Compound extends Config(new Override2 ++ new Override1 ++ new Base) // OR by setting TARGET_CONFIG=Override2_Override1_Base // Can specify undefined classes this way. ex: TARGET_CONFIG=Override2_Base With this scheme, you don't need to define a Config class for every instance you wish to generate, making it very useful for sweeping over a parameterization space. **Note that the precedence of Configs decreases from left to right in a string**. Appending a config to an existing one will only have an effect if it sets a field not already set in higher precendence Configs. For example, "BaseF1Config_SetFieldAtoX" is equivalent to "BaseF1Config_SetFieldAtoX_SetFieldAtoY". How a particular Config resolves it's ``Field`` s can be unintuitive for complex compound ``Config`` s. One precise way to check a config is doing what you expect is to open the scala REPL, instantiate an instance of the desired ``Config``, and inspect its fields. .. code-block:: shell $ make sbt # Launch into SBT's shell with extra FireSim arguments sbt:firechip> console # Launch the REPL scala> val inst = (new firesim.firesim.FireSimRocketChipConfig).toInstance # Make an instance inst: freechips.rocketchip.config.Config = FireSimRocketChipConfig scala> import freechips.rocketchip.subsystem._ # Get some important Fields import freechips.rocketchip.subsystem.RocketTilesKey scala> inst(RocketTilesKey).size # Query number of cores res2: Int = 1 scala> inst(RocketTilesKey).head.dcache.get.nWays # Query L1 D$ associativity res3: Int = 4 Rocket Chip Generator-based SoCs (firesim project) -------------------------------------------------- Using the Make variables listed above, we give examples of generating different targets using the default Rocket Chip-based target project. ----------------- Rocket-based SoCs ----------------- Three design classes use Rocket scalar in-order pipelines. Single core, Rocket pipeline (default) .. code-block:: bash make TARGET_CONFIG=FireSimRocketConfig Single-core, Rocket pipeline, with network interface .. code-block:: bash make TARGET_CONFIG=WithNIC_FireSimRocketChipConfig Quad-core, Rocket pipeline .. code-block:: bash make TARGET_CONFIG=FireSimQuadRocketConfig --------------- BOOM-based SoCs --------------- The BOOM (`Berkeley Out-of-Order Machine `_) superscalar out-of-order pipelines can also be used with the same design classes that the Rocket pipelines use. Only the TARGET_CONFIG needs to be changed, as shown below: Single-core BOOM .. code-block:: bash make TARGET_CONFIG=FireSimLargeBoomConfig Single-core BOOM, with network interface .. code-block:: bash make TARGET_CONFIG=WithNIC_FireSimBoomConfig ---------------------------------------------------------- Generating A Different FASED Memory-Timing Model Instance ---------------------------------------------------------- Golden Gate's memory-timing model generator, FASED, can elaborate a space of different DRAM model instances: we give some typical ones here. These targets use the Makefile-defined defaults of ``DESIGN=FireSim PLATFORM_CONFIG=BaseF1Config``. Quad-rank DDR3 first-ready, first-come first-served memory access scheduler .. code-block:: bash make TARGET_CONFIG=DDR3FRFCFS_FireSimRocketConfig As above, but with a 4 MiB (maximum simulatable capacity) last-level-cache model .. code-block:: bash make TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimRocketConfig FASED *timing-model* configurations are passed to the FASED Bridges in your Target's FIRRTL, and so must be prepended to ``TARGET_CONFIG``. Midas Examples (midasexamples project) -------------------------------------------------- This project can generate a handful of toy target-designs (set with the make variable ``DESIGN``). Each of these designs has their own chisel source file and serves to demostrate the features of Golden Gate. Some notable examples are: #. ``GCD``: the "Hello World!" of hardware. #. ``WireInterconnect``: demonstrates how combinational paths can be modeled with Golden Gate. #. ``PrintfModule``: demonstrates synthesizable printfs #. ``AssertModule``: demonstrates synthesizable assertions To generate a target, set the make variable ``TARGET_PROJECT=midasexamples``. so that the right project makefrag is sourced. -------- Examples -------- To generate the GCD midasexample: .. code-block:: bash make DESIGN=GCD TARGET_PROJECT=midasexamples FASED Tests (fasedtests project) -------------------------------------------------- This project generates target designs capable of driving considerably more bandwidth to an AXI4-memory slave than current FireSim targets. These are used to do integration and stress testing of FASED instances. -------- Examples -------- Generate a synthesizable AXI4Fuzzer (based off of Rocket Chip's TL fuzzer), driving a DDR3 FR-FCFS-based FASED instance. .. code-block:: bash make TARGET_PROJECT=fasedtests DESIGN=AXI4Fuzzer TARGET_CONFIG=FRFCFSConfig As above, now configured to drive 10 million transactions through the instance. .. code-block:: bash make TARGET_PROJECT=fasedtests DESIGN=AXI4Fuzzer TARGET_CONFIG=NT10e7_FRFCFSConfig