Every iteration my team explores ways of reducing time required for QA/Testing. We are a small team and do not have dedicated resources for Quality Assurance. That being said, what we can do is reduce the amount of defects found during that time. The best way to do that is to follow a Test Driven Development (TDD) methodology where you write your test from requirements before writing application code.
Now of course you cannot test everything but you can test close to one hundred percent of requirement and code coverage. Now Test Driven Development usually is done with Unit Test but it can be done with Functional Verification Test (FVT) and Performance Verification Test (PVT) as well. However FVT and PVT require a level of requirement elaboration/documentation that is not always available. That being said Automated Unit Test done right will reduce a large amount of possible defects.
How to perform Test Driven Development?
Now I alread described Test Driven Development (TDD) above but I thought I would go a bit deeper first before jumping into implementation across languages. TDD is more than writing a test before code. Rather it is a repetitive iterative process of adding changes in bite sized pieces.
I recommend the following path:
- Start by writing a test for the smallest requirement, defect, or change request that has little to no dependencies. The smaller the change the less understanding you will need of the larger picture.
- Run the test you just wrote. If it succeeds then the existing code supports it and you should move on to the next smallest requirement.
- Write just enough code to make all of the test pass. If you have a dependency on another component that has yet to be defined or written then create an interface and mock it (I talk more about this later).
- Refactor your code to make it cleaner and smaller while rerunning all the test to ensure you don’t break any functionality. This also removes any dead code from retired functional requirements.
The basic test
Although different Unit Test frameworks can behave differently the “effective layout of a test case ensures all required actions are completed, improves the readability of the test case, and smooths the flow of execution. Consistent structure helps in building a self-documenting test case. A commonly applied structure for test cases has (1) setup, (2) execution, (3) validation, and (4) cleanup.
- Setup: Put the Unit Under Test (UUT) or the overall test system in the state needed to run the test.
- Execution: Trigger/drive the UUT to perform the target behavior and capture all output, such as return values and output parameters. This step is usually very simple.
- Validation: Ensure the results of the test are correct. These results may include explicit outputs captured during Execution or state changes in the UUT.
- Cleanup: Restore the UUT or the overall test system to the pre-test state. This restoration permits another test to execute immediately after this one.”
Yes I basically stole from the Wikipedia article but I couldn’t write it any better.
Testing what doesn’t exist
I mentioned before that If you have a dependency on another component that has yet to be defined or written then you should create an interface and mock it. “Mock Objects are simulated objects that mimic the behavior of real objects in controlled ways.” It is important to note that these objects do not perform that functionality of what they are mocking. Rather they simply provide expected output for expected input.
Across three languages
In order to demonstrate how to write unit test and mock objects across three languages I decided to write the same component three times in each language.
The component is a simple application to read an email from a string and look up the title in an external directory (i.e. LDAP). Now the external directory doesn’t exist so I use a Mock object to test the components.
Unit Testing Perl
Unit testing in Perl has been around for a long time. However I often describe Perl code as being pre or post 2008. That is when Adam Kennedy gave a talk about managing a massive amount of perl code and why testable code is important.
My favorite quote from the talk is “Our ability to create software is limited, primarily, by our ability to test software.” I also love the references to the city of London’s 19th century sewers. I recommend every Perl programmer watch this presentation. You can look at any Perl code and tell if the developer used principals from this talk.
Typically I use the Test Anything Protocol for testing Perl. It has an extension for JUnit reports which allow it to fit nicely into a Rational Team Concert Development suite. I should also reference Test::Mock::Class which I use for mocking objects in Perl.
Now perl is a very dynamic language where you may only be able to identify dead code during runtime. I typically use Devel::Cover to identify dead code.
On one hand you get the benefit of an interface but the failure to implement a function is only known during runtime which is bad. As always I am open to suggestions and feedback.
Unit Testing in Java
JUnit is possibly the most popular Unit Testing frameworks. It is also included within Eclipse by default and there are tools provided to make Test Driven Development a first class citizen. There are numerous tutorials available on the topic.
For mock objects in Java I recommend Mockito. If you are using Eclipse for Java development then I also suggest using the EclEmma. It has outstanding integration but it doesn’t report against exceptions very well.
Conclusion and follow-up topics
Test Driven Development is a very powerful tool. Looking at code writen using this method there is an obvious difference. Code is less buggy and easier to manage. You have confidence your code is sound without having to implement the entire application. I also want to mention Jurgen Appelo excellent article on implementing a requirement across multiple iterations TDD style.
Now I purposely left some topics out for future blog post. For instance as I mentioned earlier it is possible to use Functional Verification Test (FVT), Business Verification Test (BVT). Performance Verification Test (PVT) as part of Test Driven Development. You can do more than simple Unit Test but there are more than a few gotchas.