Last week, I asked my 8-year-old nephew, "What do you want to do when you grow up?"
"I want to be a doctor!" he answered without hesitation. Nice! I was thinking, the kid wants to heal the world. But there was more to it than that. "Yeah, doctors have the coolest tools!" he added. "They have stethoscopes to hear the heart beats, and TV monitors that show them how well the patient is doing."
Ha! We have lots of cool tools in the engineering world as well; maybe I can convince my nephew to switch his dream from becoming an ER doctor to being an SoC verification engineer like his uncle. But then I thought about the gap that still exists between what verification engineers need to do their job efficiently and what tools are currently available to help them.
This is especially true in the verification of standard SoC interfaces like the ARM fabrics, PCI Express (PCIe), USB, Ethernet, etc. The specifications that define these interfaces are long, complex, and daunting to all but a few super experts. What's needed is an easy-to-use tool - a verification engineer's "stethoscope," you might say - that provides a test suite, a coverage model, and a verification plan.
As every experienced engineer knows, the most efficient way to deal with a complex problem is to give it to someone else to solve! And that is exactly what verification engineers do when faced with the challenge of verifying standard interfaces. Just as design engineers purchase commercial design IP to offload the design problem, verification engineers purchase verification IP to offload the verification problem. Commercial verification IP (VIP) components will mimic a host or device in order to stimulate and respond to the device under test (DUT). The best VIP components will support the advanced techniques that verification engineers employ including constrained-random test sequence generation, coverage collection, and error injection. The VIP will also include built-in assertions to automatically monitor protocol specification conformance.
High quality commercial VIP is a huge benefit to verification engineers, but as is so often the case with SoC development, increasing complexity drives the need for still more automation, and that is indeed the case with the verification of standard interfaces.
To illustrate this, let's take a look at the process of verifying a PCIe 3.0 device. The 3.0 specification doubles the bandwidth of PCIe to support the world's insatiable demand for multimedia content (including all the video games my nephew plays). Unfortunately for verification engineers, the spec has also grown tremendously and now approaches 900 pages (see figure 1.)
Figure 1. Growing complexity of PCI Express specification
In order to thoroughly verify a PCIe 3.0 device, one needs VIP that includes at least two basic components:
- 1) A Bus Functional Model (BFM) that emulates a PCIe 3.0 device (Root Complex, End Point, Switch) and that follows all the specification's rules and is capable of generating the appropriate stimulus to the Device Under Test (DUT).
- 2) A Monitor that shadows the DUT and contains assertions for each and every protocol violation.
So far so good; several vendors offer PCIe 3.0 BFM and Monitor support. But - is that enough for effective verification of PCIe 3.0 devices? The PCIe 3.0 spec has 860 pages (!) with hundreds of state machines, complex symbol encoding, power management procedures, and error handling routines.
Figure 2. Example PCIe 3.0 State machine
The poor verification engineer that needs to write tests for the example PCIe 3.0 state machine shown in figure 2 will spend long months in creating tests that trigger the DUT to move from one state to another through all the various path combinations. Once those tests are done, ensuring that they work properly and the DUT is *actually* tested in all those scenarios, will be a tedious work of sifting through waveforms, logs and functional coverage reports. And this is just one sub-section of the PCIe 3.0 physical layer spec.
There must be another way! What would a complete PCIe 3.0 verification solution look like in an ideal world? (a world in which kids want to be verification engineers!).
The first element that comes to mind is a GUI that would help the engineer visualize the verification task. The GUI would simplify the management of the hundreds of protocol parameters and let the user to choose the relevant parameters for his specific DUT (with PCIe, for instance, the number of lanes, number of virtual channels, power management options and many many more). All parameters should be fed to the testbench to match it to the design. Such a GUI would dramatically simplify the job of configuring the verification environment.
The second component would be a comprehensive test suite. A test suite that contains all tests for PCIe 3.0 features. But not only that, out of this large set of tests (1000s would be needed), the test suite should be 'smart' enough to run only the relevant subset that make sense for the specific DUT's functionality.
Now that we have an up and running verification environment, with a test suite that runs on top of it, the next desired component would be a way to track the verification progress. In the verification world, tracking equals functional coverage and that's exactly what it takes: A large, comprehensive coverage model that contains metrics for each PCIE 3.0 spec feature. It must be open in order to enable user extensions and should be available in the major verification languages, SystemVerilog and e, per user preference.
Such ideal solution will not be complete without another level of abstraction on top of the functional coverage model. Like candy, functional coverage is great in small pieces but will make you sick if you have too much (and verifying that huge PCIe 3.0 spec will generate a boat load of coverage).
A verification plan, structured just like the specification itself, and linked to the relevant functional coverage buckets, would do the trick. With such verification plan, the verification progress tracking is much easier and enables 'divide and conquer' approach that is essential in big projects like PCIe 3.0 device verification.
Now, here's the good news for verification engineers. The Cadence Verification IP team has worked hard to translate the above ideal solution to a real product. The TripleCheck IP Validator is the latest addition to the Cadence Verification IP (VIP) Catalog, and it greatly simplifies and accelerates compliance testing of interface IP. Built on previous generations of Cadence compliance solutions (PureSuite and the Compliance Management System), TripleCheck combines the three most critical components of verification in a single, easy-to-use environment: a test suite, coverage model, and verification plan.
An important requirement for TripleCheck was to enable a single flow and connectivity for all the solution's components. This is not a trivial requirement given that users employ various test bench languages and simulators.
Figure 3. TripleCheck usage flow
The flow starts with PureView, the GUI Configurator tool that walks the user through the different protocol and VIP options. Once all parameters are set, a user configuration file is generated. This file contains all the DUT specific parameters and used by all other solution components.
The VIP uses the configuration file to match the DUT setting and with an auto-generated HDL wrapper, the DUT and VIP are ready to be integrated.
The TripleCheck Testsuite stimulates the VIP and is activated by a 'push button'. The tests that are irrelevant for the specific configuration are filtered out automatically (e.g. if the user sets a 1 lane configuration, all multi-lane tests will be filtered out).
The Test suite includes all the required scenarios to achieve >95% coverage and verification plan grade.
TripleCheck has a built-in functional coverage model (both in e and System Verilog). The coverage is collected and accumulated according to the actual traffic on the bus. The user can check at any time the coverage status (the grade) and can also extend and filter it (as the coverage model is visible to the user).
The vPlan is an executable XML doc that links the coverage model to a meaningful, spec oriented, verification plan. The vPlan includes a complete representation of the spec, but the irrelevant sections/scenarios, per the user configuration file, are masked out automatically. An example is shown in figure 4.
Figure 4. A section of a PCI Express 3.0 vPlan, showing the percentage of coverage goals completed for each spec paragraph
Will my nephew trade in the Doctor's stethoscope for the verification engineer's TripleCheck? This is yet to be determined, but a lack of cool engineering tools should no longer be a factor!