Lab 4 Testing Hardware
Following completion of this lab you should be able to:
- Design robust tests for hardware components.
- Write verilog test benches.
- Read and interpret Xilinx waveforms.
3 Setting up a project
- The module
The diagram below describes the component you will be testing in this lab.
This component counts up or down based on the
direction input. Every clock cycle the register data should update on the rising edge of the clock and the output should change at that time.
Your test bench should be concerned with these signals:
- direction - a 1 bit input that makes the component count up on 0 and down on 1
- CLK - the single bit clock input that updates the register on the rising edge
- reset - a single bit input that resets the counter's register to 0 on a rising edge
- dout - the 6 bit signed output from the component, this is the number stored in the register
Before you move on your should consider the behavior of this component and what sorts of errors might exist in the implementation of this component. Write down a list of possible errors on the answer sheet.
Setting up the Xilinx Project
Start Xilinx ISE 14.7
Select File > New Project... to open the "New Project Wizard."
In the "Create New Project" dialog:
- enter "counter" for the "Name",
- choose the "Location" (Importantly, Xilinx hates spaces in paths, do not put projects in directories with spaces in the names; your OneDrive folders include spaces, do not put them there),
- CRITICAL: change the "Working directory" to "the location selected above/work",
- select "HDL" as the "Top-Level Source Type", and
- click "Next".
In the "Project Settings" dialog, enter the following properties:
- Family: Spartan3E
- Device: XC3S500E
- Package: FG320
- Speed: -4
- Synthesis Tool: XST (VHDL/Verilog)
- Simulator: ISim (VHDL/Verilog)
- Preferred Language: Verilog
and click "Next".
Click "Finish" in the "Project Summary" dialog.
Note: You can change the project properties by right clicking on the device (i.e. xc3s500e-4fg320) in the "Hierarchy" section of the "Design" tab and selecting "Design Properties..."
Add a the components to the project. There are a number of ways that you can do this:
- Right click on the device in the "Hierarchy" section of the "Design" tab and select "Add Copy of Source..."
- Select Project > Add Copy of Source...
Navigate to and select the all the
.vfiles from your git repository then click okay (you can also do this one at a time, or shift+click to select all the files at once). Take a look at Counter_A.v, this is a working version of the counter component, you should use this component to test your test bench.
Add a new test bench to the project.
- Select Project > New Source...
- Select "Verilog Test Fixture", name the test "counter_test.v"
- Press Next
- Select the "Counter_A.v" component in the "Associate Source" dialogue.
- Press Next
- Finish the test bench creation.
This will create a new verilog test bench of the counter component. The inputs and outputs will be set up in the header. You will write tests in the lower
initial begin ... endblock under the "Add stimulus here" comment.
4 Writing Tests
Editing the test bench
Before we write tests we will set up the CLK input.
- We will specify the clock settings by adding a parameter at the top of the file. Below the Input and Output initialization add the following code:
parameter HALF_PERIOD = 50; initial begin CLK = 0; forever begin #(HALF_PERIOD); CLK = ~CLK; end end
Note that this code should be in a new
initialblock, not in the autogenerated one.
- This creates a parameter
HALF_PERIODthat sets the length of the clock cycle. This number is arbitrary and is only used for simulation, it does not reflect the actual speed of the component in meat-space. Your tests should work even if this number is changed, so all waiting your tests do should use this parameter, do not hardcode any wait times in your tests!
This block initially sets the clock signal to 0, then starts an infinite loop that pauses for the length of HALF_PERIOD then inverts the clock.
Based on that above warning we need to fix some of the autogenerated code. The testing block has a
#100that we need to change. Modify this to read
#(100*HALF_PERIOD);This will change the test to wait 100 half clock cycles in the initialization step (aka 50 cycles).
To confirm that your clock is set up correctly go to the simulation tab, select the test bench in the browser on the left, and click the
Simulate Behavioral ModelOption in the
ISim Simulatorgroup below it. After Xilinx elaborates for a while you will see the waveform appear, you should see the clock ticking up and down at regular 50ns intervals.
You might notice the output signal is
XXXXXthis means the signal was never initialized and needs to be set up properly. This is because out component needs to receive the
resetsignal to initialize it.
Go back to your test bench and in the autogenerated code change
reset = 0;to
reset = 1;, then after the wait you modified previously add another line to set reset back to 0. Open up the waveform window again and press the "Re-launch" button (or close it and re-simulate as before).
You should now see reset starts off at 1 and after 50 cycles goes to 0.
doutshould have a value of 0 for the first 50 cycles and then should start ticking up.
Writing the first test.
This section will guide you through writing a single test for the component as an example of the kind of considerations you should make. A few things you should keep in mind:
- Tests should be reproducible.
- Failures should be loud.
- Which test caused a failure should be obvious.
- Tests should isolate errors.
To these ends our tests will be well documented and they should not depend on a human noticing a failure in the waveform. Therefore, "we looked at the waveform for confirmation" is not a valid test. Additionally, we want our tests to be as specific as possible, we should not rely on one test that fails for a variety of reasons.
- We'll probably want a few helper variables for our tests, lets set those up. Below where you set up the
HALF_PERIODparameter create three variables:
integer cycle_counter = 0; integer counter = 0; integer failures = 0;
We'll use these in each of our tests to help track down errors. Note:
parametersin Xilinx should only be used in test fixtures, do not include them in any components you write in the future.
- Add a comment that marks an area for the first test and describes the test, as well as a print to the console so we can see the test ran in the simulation:
//-----TEST 1----- //Testing counting up $display("Testing counting up.");
- The first thing we want to do in every test is reset the inputs to what this test requires. We should do this even if a variable has not changed from a previous test (we might reorder tests in the future!). So lets set the inputs up for the test right after the display statement:
reset = 1; counter = 0; cycle_counter = 0; #(2*HALF_PERIOD); reset = 0; direction = 0; //we are testing counting up
reset1 and wait a single clock cycle and then set
resetto 0. This is so we know the state of the machine at the beginning of the test: It should have 0 in the register and should start counting up.
Add a line that will wait for 50 clock cycles after the input initialization so we can investigate the waveform.
Next lets set up an output that will tell us our tests have finished and how many failures occurred. At the end of the testing block add the following line:
$display("TESTS COMPLETE. \n Failures = %d", failures)
Save your test and run the simulation. Investigate the output and make sure the behavior makes sense to you. Ensure that you have run the test long enough for the "TESTS COMPLETE" message to display in the console, if not you can run the simulation a little longer.
Now lets actually write a test that ensures the counter goes up by one every single clock cycle. We need to consider the device we are testing, what is the largest and smallest number our counter can contain? What happens if it tries to keep counting once it goes past the maximum? Its important to remember that the adder in our component uses signed numbers (it can take in a -1, after all), so the output from the adder will be signed as well. We need to indicate in our testbench that the number in our register is signed, therefore edit the output line to read: ``wire signed [5:0] dout''.
Lets make a loop that will count up to that maximum and a few extra cycles so we can test the overflow behavior, put this right after the
direction = 0line we added:
repeat (40) begin #(2*HALF_PERIOD); counter = counter + 1; cycle_counter = cycle_counter + 1; if (dout != counter) begin failures = failures + 1; $display("%t (COUNT UP) Error at cycle %d, output = %d, expecting = %d", $time, cycle_counter, dout, counter); end end
Look at this code and explain briefly what it does in your own words.
It is a good idea to make sure messages are labeled with information about which test they are associated with. Here our code outputs (COUNT UP) in all messages about this test. It also uses the
$timevariable to output the time during the simulation that the message was printed, this will help you track down where in a waveform an error occurred. Use this output to figure out what time in the simulation cycle 23 of this test occurs, you may need to modify the code or use the waveform.
Your output should include some failed tests, explain why this is happening.
These failures are not a problem with our component, but an issue with our test not accurately testing the true behavior of the module. Lets modify the test a bit by keeping track of when we have overflowed the register:
repeat (40) begin #(2*HALF_PERIOD); counter = counter + 1; if (cycle_counter == 31) counter = -32; cycle_counter = cycle_counter + 1; if (dout != counter) begin failures = failures + 1; $display("%t (COUNT UP) Error at cycle %d, output = %d, expecting = %d", $time, cycle_counter, dout, counter); end end
Run this new test, explain what it is doing.
5 Working with waveforms
The default waveform shows you the inputs and outputs from the unit under test (aka UUT, or the module you are testing) as well as any parameters you set up in the test. We're going to make the waveform a little easier to read and useful for us.
You can rearrange any signals in the waveform, find the CLK label in the "Name" column of the waveform window, click and drag it to the top so it is the very top signal.
When you do this the line might disappear, that is fine. Press the "Re-launch" button on the top right. A pop-up will ask you to save your settings, press "Yes", lets name this configuration something useful: "counter_tests_config", then press "Save". You may want to consider making a new folder in your project directory to hold wave configs, you'll end up with many later on.
Lets also re-color the
CLKsignal so that it is easy to tell that it is different from the others at a glance. Right click "CLK" and go to "Signal Color" select a new color for the signal.
Now lots move on to the other signals,
doutis in binary, but really our tests are treating it like a signed integer, so lets make it display as one. Right click "dout[5:0]", go to Radix > Signed Decimal.
You can do the same with the
cycle_countparameters. You can select multiple signals by selecting the first one then shift+clicking on others, then you can change the settings for all of them at once. Set these to "Unsigned Decimal" signals.
HALF_PERIODsignals are not super helpful to us, so lets remove them from the waveform. Simply select them and press the delete key.
Lets save this configuration File > Save, then close the simulation and re-run it. You'll notice that all the work you put in is suddenly gone! Don't worry, we can load the config: Go to File > Open and select the config you just saved (it will be a
.wcfgfile). This should load the config, sometimes after loading a config you may need to "Re-launch" to get all the signals to show up in the waveform.
Lets say we are tracking down a bug and want some signals added to the waveform that aren't there. We can just navigate to them using the "Instances and Processes" and "Object" panes on the left. Lets add the
failuressignal back into the waveform. With "counter_test" selected in "Instances and Processes" click and drag "failures[31:0]" from the Objects pane onto the waveform. It should be listed in the Name column and the waveform may immediately appear, or need a re-launch.
Next we'll see how to investigate one of the signals inside the component we are testing. Expand the "counter_test" in the Instance pane, expand "uut", you should see the three sub components of the module: the adder, mux, and register. Select the mux, then drag the selector bit (
s) onto the waveform. You'll need to re-launch, do not save the waveform configuration for now. You should now see the selector bit in the waveform. Using these tools can help you track down which components are not behaving as expected during tests.
Close the simulation without saving the config.
Re-loading the config every time we simulate will get annoying fast, so lets make it open by default. In the Xilinx window with your test selected in the hierarchy right click "Simulate Behavioral Model" in the lower pane then select "Process properties".
Select "Use Custom Waveform Configuration File" then in the "Custom Waveform Configuration File" section use the "..." button to navigate to and select your
Press OK. Now if you run the simulation again it will automatically load your configuration settings each time.
6 Testing other components
You have been provided with several different implementations of the counter component described above. All but one of them are implemented in a way that does not meet the specifications. You should not edit any of these module components. The only work you will do in this lab is in the test bench you created above.
We now have a test and configuration file that will confirm that a component correctly counts up by one each clock cycle. Lets test some of the other versions of the components in the project.
Open up "counter_test.v" locate the line that initializes the UUT. This is like a constructor in any other language, it gives the type that is being constructed, "counter_A" currently, and then a name to this instance of that component, "uut" by default. You an rename the component and your test will still work perfectly, try changing "uut" to "unit_under_test", for example. You'll see that it's name will change in the Instance pane when you simulate.
Lets test a different component, all of our components have the same structure to their constructor, so we just need to call a different module (the equivalent to just changing the Class of the object you are constructing in Java). Change "counter_A" to "counter_B", save the test and run the simulation.
Investigate the waveform and the output of your test. Does it pass your test?
You can run each of the other counter implementations through your test if you would like.
7 Write more tests
Your job for this lab is to expand your test bench such that it can find the errors in all of the implementations of the counters you have been provided.
You should end up with at least one test per error you find in the counters. All the counters have a unique error, so you should have at least 4 unique tests.
Any one counter might fail multiple tests, but the combination of failed tests should help us pinpoint the exact error in the component.
After you have written all of your tests write a brief explanation of how each component is broken and which of your tests detects the error.
Remember to reset all of your inputs and parameters between each test.
Make sure that you run the simulation long enough for all the tests to run for each module.
Some of these modules have very subtle errors that you will need to carefully design tests to detect.
You should only create a single test fixture that can run any one of the modules, do not make different files for each one.
8 Turning It In
Submit the answers to the questions contained in the lab guide using the question sheet via gradescope, only 1 per team (make sure all team member's names are included). In gradescope you are able add your team members names to the submission, make sure you do so. When you are prompted to indicate where the "code question" is on the answer sheet, just select the first page of the assignment, you DO NOT upload code to gradescope.
Submit your test bench to ALL team member's git repositories in the lab04 directory. Do NOT commit the Xilinx project, only add the new test fixture to the repo.