Skip to content

Milestone 12: Testing

Due Date

Initial due date: Sunday, February 3
Final due date: Same as 1.0 Release
Grading: Team


For this milestone you will implement and run both unit and behavioral tests. You will likely use third-party libraries and tools for creating and running your unit tests. Typically, the official documentation for your platform will have a section on “Testing” which tells you how they recommend you do automated tests. You will also devise a method for automating some of this testing and making it part of your workflow.

The first part of the milestone requires you to research these testing tools, chose some, learn to use them, and create and add to your repo one simple unit and one behavior test to demonstrate your progress.

The second part, due at the end of the semester, requires you to have added multiple tests to your repo. You should add the unit tests as you write the code.


Automated tests are are code that you write to test your program. Depending on the situation you might have different sets of tests: some you run before every commit, some before every merge, some before every release, etc.

The goal of the automated tests is to catch bugs early, before they get to the user. When you have a large set of test cases you feel more confident that the change you made to your code will not break something else in the program. The use of tests becomes more and more important as the size of the codebase increases beyond the ability of any one person to understand it all. It is rumored that Oracle database’s 25 Million lines of C code only work because they have millions of automated tests that must pass before any new code is merged.

Unit Testing

In unit testing you write tests which verify that your methods/functions do what you think they should do. If a function is meant to sort an array of integers, then you write a unit test that calls that functions with a few example arrays and then check the result to make sure they are sorted. You are a programmer, so you know that it is most likely that errors will appear in boundary conditions, such as when the array is empty, when it has 1 element, when all the elements are the same, when they are in inverse order, etc. So, your examples are informed, not just random.

Unit testing has been around for decades and there are now many Unit testing frameworks for every programming language imaginable. The most popular of these tie in with your IDE to give you pretty output. In most IDEs you can run junit tests and get a special window with red marks for the tests that failed and green marks for those that passed. You will use a unit testing framework in your code.

The unit tests are also part of your repo. Make sure to push them to github.

You will also set it up so that running these tests is a simple matter of just a click, or one command. Everyone should run all unit tests to make sure they pass before they commit their code and push their changes to github.

Behavior Testing

In behavior testing you make sure your app does what the user expects it to do, that is, what your Specifications say. This is functionality at a much higher level than the unit-testing functionality, so you are testing many parts of your code.

Some of this testing is done by just having people use the app. Unfortunately, that does not scale well. You want to be able to perform some of these tests quickly and automatically for every major code change. Luckily there are tools that help us automate this type of testing.

For example, if you are building a webapp then you might want to use Selenium which lets you automate web browsing, that is, write a program that emulates a user using a web browser to access your site, do some stuff, get results, and check that those results are what he should be getting. There are many other website testing libraries out there, such as casper.js, Nightwatch, Phantom, and others. Android has built-in libraries for both unit and behaviors (which they call “instrumented”) testing.

All major web framework have a section in their manual which gives their recommendations for unit and behavior testing strategies, for example, see Django testing.

All major mobile platforms have full documentation on how to do testing on their platform, see android testing, iOS XCTest, ionic guide to testing, etc.


For the first deadline:

  1. Have at least one unit and one behavioral test in your repo. Typically in a tests folder.
  2. In your file add a Testing section which lists
    1. the commands/instructions one must type to run your tests
    2. the directory (or filename regexep) where all your tests are located.

For the final deadline:

  1. Multiple unit tests (even if they are silly)
  2. Multiple behavioral tests. Test the core functionality of the app.
  3. In your file add a Testing section which lists
    1. the commands/instructions one must type to run your tests
    2. the directory (or filename regexep) where all your tests are located
  4. Optionally, setup a continuous integration service, like travis to automatically run your tests.