Python testing notes with pytest
I just started experimenting with pytest to performing testing of my application. Because it initially took some trial and error, I figured I can try to document this.
I definitely am not an expert on
Set up testing directories
The current project that I’m leveraging has the following structure
/img
/static
/templates
/tests/conftest.py
/tests/test_db.py
/tests/test_utilties.py
/db.py
/utilities.pyI created a directory called tests. In this directory, I have all my testing files located. I mainly want to test the db.py and utilities.py modules.
Installing pytest and the remaining modules
To install pytest, go to the prompt and enter:
pip install pytest
pip install pytest-covPytest is the normal pytest testing application and pytest-cov is an addition application that allows us to also show the coverage of the tests that we run. NOTE: this will also install coverage so you have the benefit to run a coverage report on you main application outside of pytest as well.
Writing the modules
All the testing modules should begin with the prefix test. This will allow pytest to automatically find the testing modules and run them. In addition, there is a conftest.py module. This module is used to provide any fixtures or functions that can run before during or after your test. Examples of these applications are shown below:
conftest.py
import pytest
from flask import Flask
@pytest.fixture(scope="session")
def temp_database(tmpdir_factory):
""" Initalize the Database """
tmpdb = str(tmpdir_factory.mktemp('temp'))+"/testdb.sqlite"
return tmpdbtest_db.py
import db
import pytest
def test_uuid():
uuid1 = db.get_a_uuid()
uuid2 = db.get_a_uuid()
assert uuid1 != uuid2
def test_invalid_db():
ret = db.initialize_database()
assert ret != Falsetest_utilities.py
import utilities
import pytest
def test_validate_byte():
ret = utilities.validate_byte_as_printable(chr(30))
assert ret=='.'
ret = utilities.validate_byte_as_printable(chr(32))
assert ret==' '
ret = utilities.validate_byte_as_printable(chr(128))
assert ret=='.'
ret = utilities.validate_byte_as_printable(chr(76))
assert ret==chr(76)Run the testing modules
So from the root directory of my project, you can run the tests by doing the following:
python -m pytest testsYou should see output, similar to the following. In this example, you can see that we ran all the tests in the tests directory.
$ python -m pytest tests
========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.1, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
rootdir: /Users/cbogdon/coding/platinum-onboard/broker, inifile:
plugins: cov-2.6.1
collected 12 items
tests/test_db.py ........... [ 91%]
tests/test_utilities.py . [100%]
======================================================= 12 passed in 0.10 seconds ========================================================
$ You can also specify what file to run instead of all the testing modules:
$ python -m pytest tests/test_utilities.py
========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.1, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
rootdir: /Users/cbogdon/coding/platinum-onboard/broker, inifile:
plugins: cov-2.6.1
collected 1 item
tests/test_utilities.py . [100%]
======================================================== 1 passed in 0.01 seconds ========================================================
$ Or you can run only a testing function within a file. For example we only want to run the test_uuid function within test_db.py
$ python -m pytest tests/test_db.py::test_uuid
========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.1, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
rootdir: /Users/cbogdon/coding/platinum-onboard/broker, inifile:
plugins: cov-2.6.1
collected 1 item
tests/test_db.py . [100%]
======================================================== 1 passed in 0.02 seconds ========================================================
$ Options that you may want to run
-v
In Verbose mode, on each test we are running we are showing the actual function name that is being executed.
$ python -m pytest -v tests/test_db.py
========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.1, pytest-4.1.1, py-1.7.0, pluggy-0.8.1 -- /Users/cbogdon/virtualenv/platinum-onboard/bin/python
cachedir: .pytest_cache
rootdir: /Users/cbogdon/coding/platinum-onboard/broker, inifile:
plugins: cov-2.6.1
collected 11 items
tests/test_db.py::test_uuid PASSED [ 9%]
tests/test_db.py::test_invalid_db PASSED [ 18%]
tests/test_db.py::test_create_new_database PASSED [ 27%]
tests/test_db.py::test_create_duplicate_database PASSED [ 36%]
tests/test_db.py::test_insert_good_records PASSED [ 45%]
tests/test_db.py::test_insert_duplicate_records[domain] PASSED [ 54%]
tests/test_db.py::test_insert_duplicate_records[device] PASSED [ 63%]
tests/test_db.py::test_insert_bad_record PASSED [ 72%]
tests/test_db.py::test_insert_bad_field PASSED [ 81%]
tests/test_db.py::test_search_db[domain] PASSED [ 90%]
tests/test_db.py::test_search_db[device] PASSED [100%]
======================================================= 11 passed in 0.10 seconds ========================================================
$ –setup-show
This parameter is showing the the setup of fixtures while executing tests. It is very useful in showing when and if they are executing.
$ python -m pytest -v --setup-show tests/test_db.py
========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.1, pytest-4.1.1, py-1.7.0, pluggy-0.8.1 -- /Users/cbogdon/virtualenv/platinum-onboard/bin/python
cachedir: .pytest_cache
rootdir: /Users/cbogdon/coding/platinum-onboard/broker, inifile:
plugins: cov-2.6.1
collected 11 items
tests/test_db.py::test_uuid
tests/test_db.py::test_uuidPASSED
tests/test_db.py::test_invalid_db
tests/test_db.py::test_invalid_dbPASSED
tests/test_db.py::test_create_new_database
SETUP S tmpdir_factory
SETUP S temp_database (fixtures used: tmpdir_factory)
tests/test_db.py::test_create_new_database (fixtures used: temp_database, tmpdir_factory)PASSED
tests/test_db.py::test_create_duplicate_database
tests/test_db.py::test_create_duplicate_database (fixtures used: temp_database, tmpdir_factory)PASSED
tests/test_db.py::test_insert_good_records
tests/test_db.py::test_insert_good_records (fixtures used: temp_database, tmpdir_factory)PASSED
tests/test_db.py::test_insert_duplicate_records[domain]
SETUP F database[domain]
tests/test_db.py::test_insert_duplicate_records[domain] (fixtures used: database, temp_database, tmpdir_factory)PASSED
TEARDOWN F database[domain]
tests/test_db.py::test_insert_duplicate_records[device]
SETUP F database[device]
tests/test_db.py::test_insert_duplicate_records[device] (fixtures used: database, temp_database, tmpdir_factory)PASSED
TEARDOWN F database[device]
tests/test_db.py::test_insert_bad_record
tests/test_db.py::test_insert_bad_record (fixtures used: temp_database, tmpdir_factory)PASSED
tests/test_db.py::test_insert_bad_field
tests/test_db.py::test_insert_bad_field (fixtures used: temp_database, tmpdir_factory)PASSED
tests/test_db.py::test_search_db[domain]
SETUP F database[domain]
tests/test_db.py::test_search_db[domain] (fixtures used: database, temp_database, tmpdir_factory)PASSED
TEARDOWN F database[domain]
tests/test_db.py::test_search_db[device]
SETUP F database[device]
tests/test_db.py::test_search_db[device] (fixtures used: database, temp_database, tmpdir_factory)PASSED
TEARDOWN F database[device]
TEARDOWN S temp_database
TEARDOWN S tmpdir_factory
======================================================= 11 passed in 0.14 seconds ========================================================
$ Coverage running with tests
It is always useful to see how much of the module was covered during
python -m pytest --cov=tests$ python -m pytest --cov=tests
========================================================== test session starts ===========================================================
platform darwin -- Python 3.6.1, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
rootdir: /Users/cbogdon/coding/platinum-onboard/broker, inifile:
plugins: cov-2.6.1
collected 12 items
tests/test_db.py ........... [ 91%]
tests/test_utilities.py . [100%]
---------- coverage: platform darwin, python 3.6.1-final-0 -----------
Name Stmts Miss Cover
---------------------------------------------
tests/conftest.py 5 0 100%
tests/test_db.py 52 3 94%
tests/test_utilities.py 11 0 100%
---------------------------------------------
TOTAL 68 3 96%
======================================================= 12 passed in 0.22 seconds ========================================================
(platinum-onboard) CBOGDON-M-D1FU:broker cbogdon$