Python – How to run all Python unit tests in a directory

pythonpython-unittesttestingunit testing

I have a directory that contains my Python unit tests. Each unit test module is of the form test_*.py. I am attempting to make a file called all_test.py that will, you guessed it, run all files in the aforementioned test form and return the result. I have tried two methods so far; both have failed. I will show the two methods, and I hope someone out there knows how to actually do this correctly.

For my first valiant attempt, I thought "If I just import all my testing modules in the file, and then call this unittest.main() doodad, it will work, right?" Well, turns out I was wrong.

import glob
import unittest

testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]

if __name__ == "__main__":
     unittest.main()

This did not work, the result I got was:

$ python all_test.py 

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

For my second try, I though, ok, maybe I will try to do this whole testing thing in a more "manual" fashion. So I attempted to do that below:

import glob
import unittest

testSuite = unittest.TestSuite()
test_file_strings = glob.glob('test_*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
[__import__(str) for str in module_strings]
suites = [unittest.TestLoader().loadTestsFromName(str) for str in module_strings]
[testSuite.addTest(suite) for suite in suites]
print testSuite 

result = unittest.TestResult()
testSuite.run(result)
print result

#Ok, at this point I have a result
#How do I display it as the normal unit test command line output?
if __name__ == "__main__":
    unittest.main()

This also did not work, but it seems so close!

$ python all_test.py 
<unittest.TestSuite tests=[<unittest.TestSuite tests=[<unittest.TestSuite tests=[<test_main.TestMain testMethod=test_respondes_to_get>]>]>]>
<unittest.TestResult run=1 errors=0 failures=0>

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK

I seem to have a suite of some sort, and I can execute the result. I am a little concerned about the fact that it says I have only run=1, seems like that should be run=2, but it is progress. But how do I pass and display the result to main? Or how do I basically get it working so I can just run this file, and in doing so, run all the unit tests in this directory?

Best Answer

With Python 2.7 and higher you don't have to write new code or use third-party tools to do this; recursive test execution via the command line is built-in. Put an __init__.py in your test directory and:

python -m unittest discover <test_directory>
# or
python -m unittest discover -s <directory> -p '*_test.py'

You can read more in the python 2.7 or python 3.x unittest documentation.


Update for 2021:

Lots of modern python projects use more advanced tools like pytest. For example, pull down matplotlib or scikit-learn and you will see they both use it.

It is important to know about these newer tools because when you have more than 7000 tests you need:

  • more advanced ways to summarize what passes, skipped, warnings, errors
  • easy ways to see how they failed
  • percent complete as it is running
  • total run time
  • ways to generate a test report
  • etc etc