Python Test and Debug Guideline

When should we test and debug

Design your code for ease of testing and debugging:

• Break program into compnents that can be tested and debugged independently
• Document constraints on modules(expectation on inputs/outputs, assumption of the code)

First you need to ensure that code will actually run, we need to remove the syntax error before the test. That is very easy. most of the error can be detected by the IDE or syntax checker tools.

 

Test suite:

We want to find a collection of inputs that has high likelihood of revealing bugs, yet is efficient. Your test case is very important.

Consider the boundary cases:

Like list: empty list, singleton list, many elements list

For numbers: very small, very large, “typical”

 

Black-box testing and Glass-box testing:

Black-box testing: use heuristics based on exploring paths through the specifications.

Advantage: 

• can be done by someone other than implementer
• Will avoid inherent biases of implementer, exposing potential bugs more easily
• Testing designed without knowledge of implementation, thus can be reused even if implementation changed.

Glass-box testing: use heuristics based on exploring paths through the code.

Some rules of thumb:

• Exercise both branches of all if statement
• Ensure each except clauses is executed
• For each for loop, have tests where loop is not entered, once and more than once
• For while loop, same cases as for loop, cases that catch all ways to exit loop
• For recursive functions, test with no recursive calls, one recursive call and more than one calls.

Path-completed testing: if every potential path through the code is tested at least once.

  

How to conduct the testing

• Start with unit testing: check that each modue is working. Usually it will catch algorithm bugs
• Move to integration testing: check that system as whole works correctly. Usually it will catch integration bugs.
• Regression testing: Cycle between these path (especially when bug fixing done)

Drivers care code that set up environment needed to run code

Invoke code on predefined sequence of inputs

Save results and then reports

Drivers simulate parts of program that use unit being tested

Different bugs:

• Overt  has  an  obvious  manifestation  –  code  crashes  or  runs  forever  
• Covert  has  no  obvious  manifestation   - code  returns  a  value,  which  may  be  incorrect  but  hard  to  determine  
• Persistent  occurs  every  time code  is  run  
• intermittent only  occurs  some  times,  even  if  run  on  same  input

Overt and persistent: Obvious to detect. Good programmers use defensive programming to try to ensure that if error is made, bug will fall into this category

Overt and intermiSent: More frastracting,can be harder to debug, but if conditions that prompt bug can be reproduced, can be handled

Covert Highly dangerous, as users may not realize answers are incorrect until code has been run for long