Unit Testing
1.Introduction
Frappe provides some basic tooling to quickly write automated tests. There are some basic rules:
- Test can be anywhere in your repository but must begin with
test_
and should be a.py
file. - Tests must run on a site that starts with
test_
. This is to prevent accidental loss of data. - Test stubs are automatically generated for new DocTypes.
- Frappe test runner will automatically build test records for dependant DocTypes identified by the
Link
type field (Foreign Key) - Tests can be executed using
bench run-tests
- For non-DocType tests, you can write simple unittests and prefix your file names with
test_
.
Writing Tests for DocTypes
2.1. Writing DocType Tests:
- Test cases are in a file named
test_[doctype].py
- You must create all dependencies in the test file
- Create a Python module structure to create fixtures / dependencies
Example (for test_event.py
):
# Copyright (c) 2015, Frappe Technologies Pvt. Ltd. and Contributors # MIT License. See license.txt
import frappe import frappe.defaults
from frappe.tests.utils import FrappeTestCase
def create_events(): if frappe.flags.test_events_created: return
frappe.set_user("Administrator") doc = frappe.get_doc({ "doctype": "Event", "subject":"_Test Event 1", "starts_on": "2014-01-01", "event_type": "Public" }).insert()
doc = frappe.get_doc({ "doctype": "Event", "subject":"_Test Event 2", "starts_on": "2014-01-01", "event_type": "Private" }).insert()
doc = frappe.get_doc({ "doctype": "Event", "subject":"_Test Event 3", "starts_on": "2014-01-01", "event_type": "Public" "event_individuals": [{ "person": "test1@example.com" }] }).insert()
frappe.flags.test_events_created = True
class TestEvent(FrappeTestCase): def setUp(self): create_events()
def tearDown(self): frappe.set_user("Administrator")
def test_allowed_public(self): frappe.set_user("test1@example.com") doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject":"_Test Event 1"})) self.assertTrue(frappe.has_permission("Event", doc=doc))
def test_not_allowed_private(self): frappe.set_user("test1@example.com") doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject":"_Test Event 2"})) self.assertFalse(frappe.has_permission("Event", doc=doc))
def test_allowed_private_if_in_event_user(self): doc = frappe.get_doc("Event", frappe.db.get_value("Event", {"subject":"_Test Event 3"}))
frappe.set_user("test1@example.com") self.assertTrue(frappe.has_permission("Event", doc=doc))
def test_event_list(self): frappe.set_user("test1@example.com") res = frappe.get_list("Event", filters=[["Event", "subject", "like", "_Test Event%"]], fields=["name", "subject"]) self.assertEqual(len(res), 2) subjects = [r.subject for r in res] self.assertTrue("_Test Event 1" in subjects) self.assertTrue("_Test Event 3" in subjects) self.assertFalse("_Test Event 2" in subjects)
2. Running Tests
This function will build all the test dependencies and run your tests. You should run tests from "frappe_bench" folder. Without options all tests will be run.
bench run-tests
If you need more information about test execution - you can use verbose log level for bench.
bench --verbose run-tests
Options:
--app
2.1. Example for app:
All applications are located in folder: "~/frappe-bench/apps". We can run tests for each application.
- frappe-bench/apps/erpnext/
- frappe-bench/apps/erpnext_demo/
frappe-bench/apps/frappe/
bench run-tests --app erpnext bench run-tests --app erpnext_demo bench run-tests --app frappe
2.2. Example for doctype:
frappe@erpnext:~/frappe-bench$ bench run-tests --doctype "Activity Cost" .
Ran 1 test in 0.008s
OK
2.3. Example for test:
Run a specific case in User:
frappe@erpnext:~/frappe-bench$ bench run-tests --doctype User --test test_get_value .
Ran 1 test in 0.005s
OK
2.4. Example for module:
If we want to run tests in the module:
/home/frappe/frappe-bench/apps/erpnext/erpnext/support/doctype/issue/test_issue.py
We should use module name like this (related to application folder)
erpnext.support.doctype.issue.test_issue
Example:
frappe@erpnext:~/frappe-bench$ bench run-tests --module "erpnext.stock.doctype.stock_entry.test_stock_entry" ...........................
Ran 27 tests in 30.549s
2.5. Example for profile:
frappe@erpnext:~/frappe-bench$ bench run-tests --doctype "Activity Cost" --profile .
Ran 1 test in 0.010s
OK 9133 function calls (8912 primitive calls) in 0.011 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function) 2 0.000 0.000 0.008 0.004 /home/frappe/frappe-bench/apps/frappe/frappe/model/document.py:187(insert) 1 0.000 0.000 0.003 0.003 /home/frappe/frappe-bench/apps/frappe/frappe/model/document.py:386(_validate) 13 0.000 0.000 0.002 0.000 /home/frappe/frappe-bench/apps/frappe/frappe/database.py:77(sql) 255 0.000 0.000 0.002 0.000 /home/frappe/frappe-bench/apps/frappe/frappe/model/base_document.py:91(get) 12 0.000 0.000 0.002 0.000
2.6 Running Tests without creating fixtures or before_tests hook
- When you are building a feature it is useful to write tests without building test dependencies (i.e build fixtures for linked objects), with
--skip-test-records
- You can also skip the test initialisation script with
--skip-before-tests
Example
bench --site school.erpnext.local run-tests --doctype "Student Group" --skip-test-records --skip-before-tests
3.0 FrappeTestCase
FrappeTestCase
is Frappe Framework specific TestCase class extended from unittest.TestCase
. Inherting this class in your tests ensures:
frappe.local.flags
and other most used local proxies are reset after test case runs.- database - a new database transaction is started before testcase begins and rolled back after tests are finished.
Usage
# app/doctype/dt/test_dt.py
from frappe.tests.utils import FrappeTestCase
class TestDt(FrappeTestCase):
@classmethod
def setUpClass(cls):
super().setUpClass() # important to call super() methods when extending TestCase.
...
Writing XUnit XML Tests
How to run:
bench run-tests --junit-xml-output=/reports/junit_test.xml
Example of test report:
It’s designed for the CI Jenkins, but will work for anything else that understands an XUnit-formatted XML representation of test results.
Jenkins configuration support:
- You should install xUnit plugin - https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
- After installation open Jenkins job configuration, click the box named “Publish JUnit test result report” under the "Post-build Actions" and enter path to XML report: (Example: _reports/*.xml_)