Clash Royale CLAN TAG#URR8PPP

Running unittest with typical test directory structure

The very common directory structure for even a simple Python module seems to be to separate the unit tests into their own test directory:

test

new_project/
antigravity/
antigravity.py
test/
test_antigravity.py
setup.py
etc.


for example see this Python project howto.

My question is simply What's the usual way of actually running the tests? I suspect this is obvious to everyone except me, but you can't just run python test_antigravity.py from the test directory as its import antigravity will fail as the module is not on the path.

python test_antigravity.py

import antigravity

I know I could modify PYTHONPATH and other search path related tricks, but I can't believe that's the simplest way - it's fine if you're the developer but not realistic to expect your users to use if they just want to check the tests are passing.

The other alternative is just to copy the test file into the other directory, but it seems a bit dumb and misses the point of having them in a separate directory to start with.

So, if you had just downloaded the source to my new project how would you run the unit tests? I'd prefer an answer that would let me say to my users: "To run the unit tests do X."

unittest

15 Answers
15

The best solution in my opinion is to use the unittest command line interface which will add the directory to the sys.path so you don't have to (done in the TestLoader class).

unittest

sys.path

TestLoader

For example for a directory structure like this:

new_project
├── antigravity.py
└── test_antigravity.py


You can just run:

$ cd new_project
$ python -m unittest test_antigravity


For a directory structure like yours:

new_project
├── antigravity
│   ├── __init__.py # make it a package
│   └── antigravity.py
└── test
├── __init__.py # also make test a package
└── test_antigravity.py


And in the test modules inside the test package, you can import the antigravity package and its modules as usual:

test

antigravity

# import the package
import antigravity

# import the antigravity module
from antigravity import antigravity

# or an object inside the antigravity module
from antigravity.antigravity import my_object


Running a single test module:

To run a single test module, in this case test_antigravity.py:

test_antigravity.py

$ cd new_project
$ python -m unittest test.test_antigravity


Just reference the test module the same way you import it.

Running a single test case or test method:

Also you can run a single TestCase or a single test method:

TestCase

$ python -m unittest test.test_antigravity.GravityTestCase
$ python -m unittest test.test_antigravity.GravityTestCase.test_method


Running all tests:

You can also use test discovery which will discover and run all the tests for you, they must be modules or packages named test*.py (can be changed with the -p, --pattern flag):

test*.py

-p, --pattern

$ cd new_project
$ python -m unittest discover


This will run all the test*.py modules inside the test package.

test*.py

test

python -m unittest discover

test

test*.py

tests

python -m unittest discover -s tests

antigravity_test.py

python -m unittest discover -s tests -p '*test.py'

antigravity.py

test_antigravity.py

import .antigravity

import antigravity

ImportError: No module named 'test.test_antigravity'

test_antigravity.py

import antigravity

from antigravity import antigravity

__init_.py

python3 -m unittest discover

new project

The simplest solution for your users is to provide an executable script (runtests.py or some such) which bootstraps the necessary test environment, including, if needed, adding your root project directory to sys.path temporarily. This doesn't require users to set environment variables, something like this works fine in a bootstrap script:

runtests.py

import sys, os

sys.path.insert(0, os.path.dirname(__file__))


Then your instructions to your users can be as simple as "python runtests.py".

Of course, if the path you need really is os.path.dirname(__file__), then you don't need to add it to sys.path at all; Python always puts the directory of the currently running script at the beginning of sys.path, so depending on your directory structure, just locating your runtests.py at the right place might be all that's needed.

os.path.dirname(__file__)

sys.path

sys.path

runtests.py

Also, the unittest module in Python 2.7+ (which is backported as unittest2 for Python 2.6 and earlier) now has test discovery built-in, so nose is no longer necessary if you want automated test discovery: your user instructions can be as simple as "python -m unittest discover".

I generally create a "run tests" script in the project directory (the one that is common to both the source directory and test) that loads my "All Tests" suite. This is usually boilerplate code, so I can reuse it from project to project.

test

run_tests.py:

import unittest
import test.all_tests
testSuite = test.all_tests.create_test_suite()
text_runner = unittest.TextTestRunner().run(testSuite)


test/all_tests.py (from How do I run all Python unit tests in a directory?)

import glob
import unittest

def create_test_suite():
test_file_strings = glob.glob('test/test_*.py')
module_strings = ['test.'+str[5:len(str)-3] for str in test_file_strings]
suites = [unittest.defaultTestLoader.loadTestsFromName(name)
for name in module_strings]
testSuite = unittest.TestSuite(suites)
return testSuite


With this setup, you can indeed just include antigravity in your test modules. The downside is you would need more support code to execute a particular test... I just run them all every time.

include antigravity

run tests

From the article you linked to:

Create a test_modulename.py file and
put your unittest tests in it. Since
the test modules are in a separate
directory from your code, you may need
to add your module’s parent directory
to your PYTHONPATH in order to run
them:

$ cd /path/to/googlemaps

$ export PYTHONPATH=$PYTHONPATH:/path/to/googlemaps/googlemaps

$ python test/test_googlemaps.py


Finally, there is one more popular
unit testing framework for Python
(it’s that important!), nose. nose
helps simplify and extend the builtin
unittest framework (it can, for
example, automagically find your test
code and setup your PYTHONPATH for
you), but it is not included with the
standard Python distribution.

Perhaps you should look at nose as it suggests?

if you run "python setup.py develop" then the package will be in the path. But you may not want to do that because you could infect your system python installation, which is why tools like virtualenv and buildout exist.

I had the same problem, with a separate unit tests folder. From the mentioned suggestions I add the absolute source path to sys.path.

sys.path

The benefit of the following solution is, that one can run the file test/test_yourmodule.py without changing at first into the test-directory:

test/test_yourmodule.py

import sys, os
testdir = os.path.dirname(__file__)
srcdir = '../antigravity'
sys.path.insert(0, os.path.abspath(os.path.join(testdir, srcdir)))

import antigravity
import unittest


Use setup.py develop to make your working directory be part of the installed Python environment, then run the tests.

setup.py develop

invalid command 'develop'

setup.py --help-commands

setup.py

import setuptools

setup.py

pip install -e .

pip install -e .

python setup.py develop

setup.py

If you use VS Code and your tests are located on the same level as your project then running and debug your code doesn't work out of the box. What you can do is change your launch.json file:


"version": "0.2.0",
"configurations": [

"name": "Python",
"type": "python",
"request": "launch",
"stopOnEntry": false,
"pythonPath": "$config:python.pythonPath",
"program": "$file",
"cwd": "$workspaceRoot",
"env": ,
"envFile": "$workspaceRoot/.env",
"debugOptions": [
"WaitOnAbnormalExit",
"WaitOnNormalExit",
"RedirectOutput"
]

]



The key line here is envFile

"envFile": "$workspaceRoot/.env",


In the root of your project add .env file

Inside of your .env file add path to the root of your project. This will temporarily add

PYTHONPATH=C:YOURPYTHONPROJECTROOT_DIRECTORY

path to your project and you will be able to use debug unit tests from VS Code

Solution/Example for Python unittest module

Given the following project structure:

ProjectName
├── project_name
| ├── models
| | └── thing_1.py
| └── __main__.py
└── test
├── models
| └── test_thing_1.py
└── __main__.py


You can run your project from the root directory with python project_name, which calls ProjectName/project_name/__main__.py.

python project_name

ProjectName/project_name/__main__.py

To run your tests with python test, effectively running ProjectName/test/__main__.py, you need to do the following:

python test

ProjectName/test/__main__.py

1) Turn your test/models directory into a package by adding a __init__.py file. This makes the test cases within the sub directory accessible from the parent test directory.

test/models

__init__.py

test

# ProjectName/test/models/__init__.py

from .test_thing_1 import Thing1TestCase


2) Modify your system path in test/__main__.py to include the project_name directory.

test/__main__.py

project_name

# ProjectName/test/__main__.py

import sys
import unittest

sys.path.append('../project_name')

loader = unittest.TestLoader()
testSuite = loader.discover('test')
testRunner = unittest.TextTestRunner(verbosity=2)
testRunner.run(testSuite)


Now you can successfully import things from project_name in your tests.

project_name

# ProjectName/test/models/test_thing_1.py

import unittest
from project_name.models import Thing1 # this doesn't work without 'sys.path.append' per step 2 above

class Thing1TestCase(unittest.TestCase):

def test_thing_1_init(self):
thing_id = 'ABC'
thing1 = Thing1(thing_id)
self.assertEqual(thing_id, thing.id)


Following is my project structure:

ProjectFolder:
- project:
- __init__.py
- item.py
- tests:
- test_item.py


I found it better to import in the setUp() method:

import unittest
import sys

class ItemTest(unittest.TestCase):

def setUp(self):
sys.path.insert(0, "../project")
from project import item
# further setup using this import

def test_item_props(self):
# do my assertions

if __name__ == "__main__":
unittest.main()


What's the usual way of actually running the tests

I use Python 3.6.2

cd new_project

pytest test/test_antigravity.py


To install pytest: sudo pip install pytest

sudo pip install pytest

I didn't set any path variable and my imports are not failing with the same "test" project structure.

I commented out this stuff: if __name__ == '__main__' like this:

if __name__ == '__main__'

test_antigravity.py

import antigravity

class TestAntigravity(unittest.TestCase):

def test_something(self):

# ... test stuff here


# if __name__ == '__main__':
#
# if __package__ is None:
#
# import something
# sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
# from .. import antigravity
#
# else:
#
# from .. import antigravity
#
# unittest.main()


I noticed that if you run the unittest command line interface from your "src" directory, then imports work correctly without modification.

python -m unittest discover -s ../test


If you want to put that in a batch file in your project directory, you can do this:

setlocal & cd src & python -m unittest discover -s ../test


It's possible to use wrapper which runs selected or all tests.

For instance:

./run_tests antigravity/*.py


or to run all tests recursively use globbing (tests/**/*.py) (enable by shopt -s globstar).

tests/**/*.py

shopt -s globstar

The wrapper can basically use argparse to parse the arguments like:

argparse

parser = argparse.ArgumentParser()
parser.add_argument('files', nargs='*')


Then load all the tests:

for filename in args.files:
exec(open(filename).read())


then add them into your test suite (using inspect):

inspect

alltests = unittest.TestSuite()
for name, obj in inspect.getmembers(sys.modules[__name__]):
if inspect.isclass(obj) and name.startswith("FooTest"):
alltests.addTest(unittest.makeSuite(obj))


and run them:

result = unittest.TextTestRunner(verbosity=2).run(alltests)


Check this example for more details.

See also: How to run all Python unit tests in a directory?

This BASH script will execute the python unittest test directory from anywhere in the file system, no matter what working directory you are in.

This is useful when staying in the ./src or ./example working directory and you need a quick unit test:

./src

./example

#!/bin/bash

this_program="$0"
dirname="`dirname $this_program`"
readlink="`readlink -e $dirname`"

python -m unittest discover -s "$readlink"/test -v


No need for a test/__init__.py file to burden your package/memory-overhead during production.

test/__init__.py

You should really use the pip tool.

Use pip install -e to install your package in development mode. This is a very good practise.

pip install -e

In the Ref url given below, 2 classic project (with test) layout are given, you can follow any of them.

Ref:

pytest

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.