TDD (Fri Oct 13, lecture 17)

Homework due for today

Legend: : Participation | : Early | : PDF | : Portfolio | : Zipped

  1. Read The F35s terible bug list to see the real world impact that bugs have. Also tht Fighter jets are really computers with wings on them!
  2. Read There’s more to Ruby Debugging than puts. Lots of very practical tips and ideas about debugging, as well as some really useful Ruby tricks.
  3. Read Test Doubles in Theory, in Minitest and Rspec. Test doubles, mocks, and similar concepts are a powerful abstraction for independently testing complex subsystems. Of course we’re using minitest, but rspec is a popular alternative test framework. Bottom line they are both quite similar to each other.
  4. Rails PA2: Complete and submit PA Rails 2. Begin working on PA Rails 3. Deliverable: PA 2 directory zipped up and submitted to Latte

Discussion on yesterday’s homework

  • Did you really do TDD? What is the key to it?
  • How did it feel to program in this way? Was it easier, harder, different?
  • Conventions: file names, separating the tests from the code (why?)

Testing Intro

There are different kinds of testing and why they are important
  • Unit testing: test a small unit of code that has few side effects
  • Integration testing: test a subsystem that has many components
  • End-to-end testing: test the complete system from top to bottom
  • Code Coverage: Seeing how much of your actual code is run during a test or production
  • Performance testing: Checking the speed of your code
  • and there are some other categories
  • NB: Ruby community is especially focused on quality and testing



Test Data
  • Sometimes known as test fixtures
  • What is the role of data in developing tests for your application
  • Some ways of thinking about test data
  • Avoiding Brittle tests
  • Mocking objects
    • Let’s look at some of the code that runs this web site.
    • Note slightly different syntax because of RSpec vs. minitest/spec
    • citem_spec.rb
    • toc_spec.rb

Testing in Rails (if we have time)

  • Principle is the same
  • Remember there’s no good reason to test whether Rails is doing what it is supposed to do
  • For example, whether validations are validating, or whether has_many is actually working
    • Maybe a few tests to poke some holes, but certainly not each and every test.

Let’s take a tour of all the test bits that are created for free for you in Rails

Testing Concepts

Credit: Some of this is directly quoting POODR
  • Testing and Agile
    • Notion of a ‘release’ is gone
    • Notion of a ‘spec’ is gone
    • Cycles are much faster
    • Automation becomes a requirement
  • Testing and Design
    • Insight (not shared by all) that TDD supports design
    • Insist on looking at the class/module/unit first as a black box that delivers services
    • Good design delays all decisions that can be delayed until further requirements are nailed down.
    • Refactoring is how new requirements are incorporated
    • Good test suites is what gives you the confidence to refactor
  • Testing and Refactoring:
    • Recall: Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which ‘too small to be worth doing’
    • So you always should have solid tests before you start refactoring
    • Each small change is verified to not have changed behavior

Good Testing/Quality/Testability

  • Fundamental motivation for testing: reduce costs
    • Writing tests should over the lifetime of the code cost less
    • Otherwise it’s not worth doing
    • How does it reduce costs?
  • Breaking it down: Why Test?
    • Finding bugs early
    • Reducing errors
    • Making assumptions explicit - documenting code with code [how do test do that?]
    • Make refactoring possible and safe [how safe?]
    • Driving design decisions (TDD?)
    • Discover bad design decisions [how?]
  • Philosophy of Commenting
    • Believe or not, it’s controversial
    • Comments should not duplicate what the code clearly says
    • Of course, “clearly says” is in the eyes of the beholder
    • One philosophy: If you feel the need to put in a block comment, extract that section into a method with an intention revealing name

Testing Pyramid

Costs of testing
  • Code to write
  • Tests to run
  • As the software (inevitably) evolves
    • interfaces change
    • The code changes in two places: implementation and test
    • Sometimes more: test fixtures, database schemas
  • Conclusion:
    • More is not necessarily better
Testing Pyramid
  • From (fewest -> most) tests and brittleness
    1. User Interface tests
    2. Integration tests
    3. Unit Tests

Test Everything?

What not to test
  • Try not to test the same thing in more than one place
  • Try not to test a library which is itself well tests
  • Try not to test something that is self-evidently trivial (eye of the beholder)
What to test
  • Try to test code which is prone to off-by-one errors
  • Try to test code with intricate algorithms
  • Try to test to verify assumption
Natural ‘borders’ in the architecture
  • If I check your passport (i.e. validate inputs) at the border
  • Then it makes no sense to check it every time you enter a building
  • As long as there are no unguarded borders :)
About Brittle Tests
  • When a change in design suddenly makes a large number of tests fail
  • Sign that either your design or your test approach can be improved

Testing Techniques


  • Overview
    • SUT (“System under test”)
    • Collaborators (other objects needed to test SUT)
    • Decoupling is a major goal
  • What is a Mock?
    • An object that stands in for another object during testing
    • Make sure that only the right methods are called
    • Related terms: Test Doubles, stubs, and others. Don’t get confused.
    • More distinctions
      • Broadest term: Test Double. They are all part of this category
      • Test Dummy: An object that stands in for another, that has no methods. Not very useful.
      • Stub: Responds to the key methods that are required for the test but the return values are fixed.
      • Spy: Like a Stub, except it “remembers” what calls were made so it can verify that the expected calls were indeed made.
  • Test Doubles: in Theory, in Minitest and in Rspec
  • Martin Fowler-Test Double

  • Advantages
    • Decoupling is always a good thing.
      • Performance
      • Maintainability
      • Testability
    • If you introduce a bug in a class
      • Only the tests that are testing that class will fail
      • So, fewer false positives
    • Disadvantages
      • Need to maintain to consistency between the mock and the class-being-mocked
      • Otherwise you may be testing somethingn that doesn’t even exist
  • Real world example:
    • Let’s look at some of the code that runs this web site.
    • Note slightly different syntax because of RSpec vs. minitest/spec
    • citem_spec.rb
    • toc_spec.rb

Fixtures (and seeds)

  • Intro
    • Data or objects that exist exclusively for testing
    • Sometimes known as test or seed data
    • What is the role of data in developing tests for your application
  • Where are they stored?
    • in memory (i.e. objects and classes)
    • on disk (i.e. files, images, …)
    • in a database (i.e. records in a dbms)
  • When are they created?
    • At “build” time, i.e. putting files in the right directories or adding records to an empty database
    • At “launch” time, i.e. when the app launches it creates the fixtures
    • At “test run time”, i.e just before the test needs them
  • Tooling
    • In rails, we have “seeds” and “fixtures” built in
    • In sinatra, just seeds

Look at next class