Skip to content

Instantly share code, notes, and snippets.

@codeinthehole
Last active April 9, 2024 00:37

Revisions

  1. codeinthehole revised this gist May 12, 2023. 1 changed file with 4 additions and 2 deletions.
    6 changes: 4 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -141,7 +141,7 @@ ways that mock objects are used:
    - _Stubs_: where the behaviour of the mock object is specified _before_ the act
    phase of a test

    - _Spys_: where the ways a mock object has been called are inspected _after_ the
    - _Spys_: where the mock calls are inspected _after_ the
    act phase of a test.

    Equivalently you can think of mocks as either being actors (stubs) or critics
    @@ -181,6 +181,7 @@ def test_with_spec():
    `mock.create_autospec` or via `mock.Mock()`. Instead, either call
    `mock.configure_mock(name=...)` or assign the `name` attribute in a separate
    statement:

    ```py
    m = mock.create_autospec(spec=SomeClass, instance=True)
    m.name = "..."
    @@ -536,7 +537,7 @@ def test_passing_sentinel(collaborator):

    Reading:

    - https://www.seanh.cc/2017/03/17/sentinel/
    - <https://www.seanh.cc/2017/03/17/sentinel/>

    ## Controlling external dependencies

    @@ -766,6 +767,7 @@ Fixtures defined in a `conftest.py` module can be used in several ways:
    ```

    - Apply to every test in a test suite using the `pytest.ini` file:

    ```dosini
    [pytest]
    usefixtures = ...
  2. codeinthehole revised this gist May 12, 2023. 1 changed file with 5 additions and 0 deletions.
    5 changes: 5 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -804,6 +804,11 @@ Some anti-patterns for unit tests:
    related objects, which can expose test flakiness around ordering (as the test
    assumes there's only one of something).

    - _Conditional logic in tests_ - this is sometimes done to share some set-up
    steps but often makes the test much harder to understand. It's almost always
    better to create separate tests (with no conditional logic) and find another
    way to share common code.

    Rules of thumb:

    - Design code to use dependency injection and pass in adapters that handle IO.
  3. codeinthehole revised this gist Jan 26, 2023. 1 changed file with 30 additions and 29 deletions.
    59 changes: 30 additions & 29 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -11,22 +11,22 @@ Contents:
    - [Creating Django model fixtures](#creating-django-model-fixtures)
    - [Creating other forms of fixture](#creating-other-forms-of-fixture)
    - [Mocks](#mocks)
    - [Stubbing](#stubbing)
    - [Passing stubs as arguments](#passing-stubs-as-arguments)
    - [Stubbing Django model instances](#stubbing-django-model-instances)
    - [Stubbing multiple return values](#stubbing-multiple-return-values)
    - [Stubbing function calls that raise an exception](#stubbing-function-calls-that-raise-an-exception)
    - [Stubbing HTTP responses](#stubbing-http-responses)
    - [Overriding Django settings](#overriding-django-settings)
    - [Controlling the system clock](#controlling-the-system-clock)
    - [Spying](#spying)
    - [How to use spys](#how-to-use-spys)
    - [Verifying a spy was called correctly](#verifying-a-spy-was-called-correctly)
    - [Stubbing](#stubbing)
    - [Passing stubs as arguments](#passing-stubs-as-arguments)
    - [Stubbing Django model instances](#stubbing-django-model-instances)
    - [Stubbing multiple return values](#stubbing-multiple-return-values)
    - [Stubbing function calls that raise an exception](#stubbing-function-calls-that-raise-an-exception)
    - [Stubbing HTTP responses](#stubbing-http-responses)
    - [Overriding Django settings](#overriding-django-settings)
    - [Controlling the system clock](#controlling-the-system-clock)
    - [Spying](#spying)
    - [How to use spys](#how-to-use-spys)
    - [Verifying a spy was called correctly](#verifying-a-spy-was-called-correctly)
    - [Verifying _all_ calls to a spy](#verifying-_all_-calls-to-a-spy)
    - [Verifying unordered calls](#verifying-unordered-calls)
    - [Verifying partial calls](#verifying-partial-calls)
    - [Extracting information about how spy was called](#extracting-information-about-how-spy-was-called)
    - [Spying without stubbing](#spying-without-stubbing)
    - [Extracting information about how spy was called](#extracting-information-about-how-spy-was-called)
    - [Spying without stubbing](#spying-without-stubbing)
    - [Checking values with sentinels](#checking-values-with-sentinels)
    - [Controlling external dependencies](#controlling-external-dependencies)
    - [Using temporary files](#using-temporary-files)
    @@ -43,6 +43,7 @@ Contents:
    - [Shared fixtures](#shared-fixtures)
    - [Prefer to inject factories](#prefer-to-inject-factories)
    - [Writing high quality code and tests](#writing-high-quality-code-and-tests)
    - [Anti-patterns](#anti-patterns)
    - [Resources](#resources)

    ## Set-up and tear-down
    @@ -137,21 +138,21 @@ This is useful for writing concise tests that pass a complex object as an input.
    Python's mock library is very flexible. It's helpful to distinguish between two
    ways that mock objects are used:

    - Stubs: where the behaviour of the mock object is specified _before_ the act
    - _Stubs_: where the behaviour of the mock object is specified _before_ the act
    phase of a test

    - Spys: where the ways a mock object has been called are inspected _after_ the
    - _Spys_: where the ways a mock object has been called are inspected _after_ the
    act phase of a test.

    Equivalently you can think of mocks as either being actors (stubs) or critics
    (spys).

    ## Stubbing
    ### Stubbing

    Stubbing involves replacing an argument or collaborator with your own version so
    you can specify its behaviour.
    you can specify its behaviour in advance.

    ### Passing stubs as arguments
    #### Passing stubs as arguments

    When passing stubs as arguments to the target, prefer
    [`mock.create_autospec`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.create_autospec)
    @@ -185,7 +186,7 @@ def test_with_spec():
    m.name = "..."
    ```

    ### Stubbing Django model instances
    #### Stubbing Django model instances

    Use the following formula to create stubbed Django model instances that can be
    assigned as foreign keys:
    @@ -208,7 +209,7 @@ def test_django_model_instance():
    This is useful for writing isolated unit tests that involve Django model
    instances.

    ### Stubbing multiple return values
    #### Stubbing multiple return values

    Assign an iterable as a mock `side_effect`:

    @@ -221,7 +222,7 @@ assert stub.method() == 2
    assert stub.method() == 3
    ```

    ### Stubbing function calls that raise an exception
    #### Stubbing function calls that raise an exception

    Assign an exception as a mock `side_effect`:

    @@ -233,7 +234,7 @@ with pytest.raises(ValueError):
    assert stub.method()
    ```

    ### Stubbing HTTP responses
    #### Stubbing HTTP responses

    Use the [`responses`](https://pypi.org/project/responses/) library. It provides
    a decorator and a clean API for stubbing the responses to HTTP requests:
    @@ -258,7 +259,7 @@ def test_something():
    - Can pass `body` instead of `json`.
    - `url` can be a compiled regex.

    ### Overriding Django settings
    #### Overriding Django settings

    Use Django's
    [`@override_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.override_settings)
    @@ -300,7 +301,7 @@ def test_something_with_middleware():
    Both `override_settings` and `modify_settings` can be used as class decorators
    but only on `TestCase` subclasses.

    ### Controlling the system clock
    #### Controlling the system clock

    Calling the system clock in tests is generally a bad idea as it can lead to
    flakiness. Better to pass in relevant dates or datetimes, or if that isn't
    @@ -343,7 +344,7 @@ def test_that_failed_last_night():
    ...
    ```

    ## Spying
    ### Spying

    Spying involves replacing an argument to the system-under-test or one of its
    collaborators with a fake version so you can verify how it was called.
    @@ -353,7 +354,7 @@ Spys can be created as `unitest.mock.Mock` instances using

    If stubs are _actors_, then spys are _critics_.

    ### How to use spys
    #### How to use spys

    Here's an example of passing a spy as an **argument to the system-under-test**:

    @@ -398,7 +399,7 @@ def test_client_called_correctly(get_client):
    As you can see, the use of dependency injection in the first example leads to
    simpler tests.

    ### Verifying a spy was called correctly
    #### Verifying a spy was called correctly

    Objects from Python's `unittest.mock` library provide several
    [`assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called)
    @@ -446,7 +447,7 @@ which pass equality checks with _everything_:
    m.assert_called_with(x=100, y=ANY)
    ```

    ### Extracting information about how spy was called
    #### Extracting information about how spy was called

    Spys have several attributes that store how they were called.

    @@ -480,7 +481,7 @@ _, call_kwargs = some_mocked_function.call_args
    assert "succeeded" in call_kwargs["message"]
    ```

    ### Spying without stubbing
    #### Spying without stubbing

    You can wrap an object with a mock so that method calls are forwarded on but
    also recorded for later examination:
  4. codeinthehole revised this gist Sep 14, 2022. 1 changed file with 14 additions and 14 deletions.
    28 changes: 14 additions & 14 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -319,26 +319,26 @@ def test_something():
    or [`freezegun`](https://github.com/spulec/freezegun):

    ```py
    import freezegun
    import time_machine

    def test_something():
    # Can pass a string, date/datetime instance, lambda function or iterable.
    with freezegun.freeze_time(dt, tick=True):
    with time_machine.travel(dt, tick=True):
    pass
    ```

    Note:

    - Can be used as a decorator
    - Within the context block, use `freezegun.move_to(other_dt)` to move time to a
    specified value.
    - Within the context block, use `time_machine.travel(other_dt)` or
    `freezegun.move_to(other_dt)` to move time to a specified value.

    The freezegun decorator is useful for debugging flakey tests that fail when run
    at certain times (like during the DST changeover day). To recreate the flakey
    fail, freeze time to when the test failed on your CI service:
    The `time_machine.travel` decorator is useful for debugging flakey tests that
    fail when run at certain times (like during the DST changeover day). To recreate
    the flakey fail, pin time to when the test failed on your CI service:

    ```py
    @freezegun.freeze_time("2021-03-28T23:15Z")
    @time_machine.travel("2021-03-28T23:15Z")
    def test_that_failed_last_night():
    ...
    ```
    @@ -652,7 +652,7 @@ Use something like this:
    import io
    import datetime

    import freezegun
    import time_machine
    from django.core.management import call_command
    from dateutil import tz

    @@ -664,7 +664,7 @@ def test_some_command():

    # Control time when MC runs.
    run_at = datetime(2021, 2, 14, 12, tzinfo=tz.gettz('Europe/London'))
    with freezegun.freeze_time(run_at):
    with time_machine.travel(run_at):
    call_command("some_command_name", stdout=stdout, stderr=stderr)

    # Check command output (if any).
    @@ -677,15 +677,15 @@ def test_some_command():
    or using Octo's private pytest fixtures:

    ```py
    import freezegun
    import time_machine
    from django.core.management import call_command


    def test_some_command(command, factory):
    run_at = factory.local.dt("2021-03-25 15:12:00")

    # Run command with a smaller number of prizes to create.
    with freezegun.freeze_time(run_at, tick=True):
    with time_machine.travel(run_at):
    result = command.run("some_command_name")

    # Check command output (if any).
    @@ -717,11 +717,11 @@ def runner():

    # tests/functional/test_command.py
    import main
    import freezegun
    import time_machine

    def test_some_command(runner):
    # Run command at a fixed point in time, specifying any relevant env vars.
    with freezegun.freeze_time(dt, tick=True):
    with time_machine.travel(dt, tick=True):
    result = runner.invoke(
    main.cli,
    args=["name-of-command"],
  5. codeinthehole revised this gist Sep 14, 2022. 1 changed file with 7 additions and 3 deletions.
    10 changes: 7 additions & 3 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -471,8 +471,6 @@ assert mock_function.call_args_list == [mock.call(x=1), mock.call(x=2)]
    assert mock_object.method_calls == [mock.call.add(1), mock.call.delete(x=1)]
    ```

    ```
    To make fine-grained assertions about function or method calls, you can use the
    `call_args` property:

    @@ -566,7 +564,7 @@ def test_csv_import():

    # Call the management command passing the CSV filepath as an argument.
    call_command("import_csv_file", f.name)
    ````
    ```

    The same thing can be done using
    [Pytest's `tmp_path` fixture](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-fixture)
    @@ -787,6 +785,8 @@ function/class that can be called with configuration arguments.
    High quality code is
    [easy to change](https://codeinthehole.com/tips/easy-to-change/).

    ### Anti-patterns

    Some anti-patterns for unit tests:

    - _Lots of mocks_ - this indicates your unit under test has to many
    @@ -799,6 +799,10 @@ Some anti-patterns for unit tests:
    of a unit being tested, not those further down the call chain. Use of
    `mock.patch` (instead of `mock.patch.object)` is a smell of this problem.

    - _Careless factory usage_ - beware of factories creating lots of unnecessary
    related objects, which can expose test flakiness around ordering (as the test
    assumes there's only one of something).

    Rules of thumb:

    - Design code to use dependency injection and pass in adapters that handle IO.
  6. codeinthehole revised this gist Jul 5, 2022. 1 changed file with 152 additions and 84 deletions.
    236 changes: 152 additions & 84 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -7,38 +7,43 @@ Contents:

    <!-- Run :InsertToc to update -->

    * [Set-up and tear-down](#set-up-and-tear-down)
    * [Creating Django model fixtures](#creating-django-model-fixtures)
    * [Creating other forms of fixture](#creating-other-forms-of-fixture)
    * [Mocks](#mocks)
    * [Stubbing](#stubbing)
    * [Passing stubs as arguments](#passing-stubs-as-arguments)
    * [Stubbing Django model instances](#stubbing-django-model-instances)
    * [Stubbing multiple return values](#stubbing-multiple-return-values)
    * [Stubbing function calls that raise an exception](#stubbing-function-calls-that-raise-an-exception)
    * [Stubbing HTTP responses](#stubbing-http-responses)
    * [Overriding Django settings](#overriding-django-settings)
    * [Controlling the system clock](#controlling-the-system-clock)
    * [Spying](#spying)
    * [Asserting a spy was called correctly](#asserting-a-spy-was-called-correctly)
    * [Extracting information about how spy was called](#extracting-information-about-how-spy-was-called)
    * [Spying without stubbing](#spying-without-stubbing)
    * [Checking values with sentinels](#checking-values-with-sentinels)
    * [Controlling external dependencies](#controlling-external-dependencies)
    * [Using temporary files](#using-temporary-files)
    * [Functional testing](#functional-testing)
    * [High quality functional tests](#high-quality-functional-tests)
    * [Django views](#django-views)
    * [Testing error responses](#testing-error-responses)
    * [Filling in forms](#filling-in-forms)
    * [Django management commands](#django-management-commands)
    * [Click commands](#click-commands)
    * [Running tests](#running-tests)
    * [Capturing output](#capturing-output)
    * [Using Pytest fixtures](#using-pytest-fixtures)
    * [Prefer to inject factories](#prefer-to-inject-factories)
    * [Writing high quality code and tests](#writing-high-quality-code-and-tests)
    * [Resources](#resources)
    - [Set-up and tear-down](#set-up-and-tear-down)
    - [Creating Django model fixtures](#creating-django-model-fixtures)
    - [Creating other forms of fixture](#creating-other-forms-of-fixture)
    - [Mocks](#mocks)
    - [Stubbing](#stubbing)
    - [Passing stubs as arguments](#passing-stubs-as-arguments)
    - [Stubbing Django model instances](#stubbing-django-model-instances)
    - [Stubbing multiple return values](#stubbing-multiple-return-values)
    - [Stubbing function calls that raise an exception](#stubbing-function-calls-that-raise-an-exception)
    - [Stubbing HTTP responses](#stubbing-http-responses)
    - [Overriding Django settings](#overriding-django-settings)
    - [Controlling the system clock](#controlling-the-system-clock)
    - [Spying](#spying)
    - [How to use spys](#how-to-use-spys)
    - [Verifying a spy was called correctly](#verifying-a-spy-was-called-correctly)
    - [Verifying _all_ calls to a spy](#verifying-_all_-calls-to-a-spy)
    - [Verifying unordered calls](#verifying-unordered-calls)
    - [Verifying partial calls](#verifying-partial-calls)
    - [Extracting information about how spy was called](#extracting-information-about-how-spy-was-called)
    - [Spying without stubbing](#spying-without-stubbing)
    - [Checking values with sentinels](#checking-values-with-sentinels)
    - [Controlling external dependencies](#controlling-external-dependencies)
    - [Using temporary files](#using-temporary-files)
    - [Functional testing](#functional-testing)
    - [High quality functional tests](#high-quality-functional-tests)
    - [Django views](#django-views)
    - [Testing error responses](#testing-error-responses)
    - [Filling in forms](#filling-in-forms)
    - [Django management commands](#django-management-commands)
    - [Click commands](#click-commands)
    - [Running tests](#running-tests)
    - [Capturing output](#capturing-output)
    - [Using Pytest fixtures](#using-pytest-fixtures)
    - [Shared fixtures](#shared-fixtures)
    - [Prefer to inject factories](#prefer-to-inject-factories)
    - [Writing high quality code and tests](#writing-high-quality-code-and-tests)
    - [Resources](#resources)

    ## Set-up and tear-down

    @@ -47,7 +52,8 @@ afterwards).

    ### Creating Django model fixtures

    For Django models, the basic pattern with [factory boy](https://factoryboy.readthedocs.io/) is:
    For Django models, the basic pattern with
    [factory boy](https://factoryboy.readthedocs.io/) is:

    ```py
    import factory
    @@ -71,7 +77,8 @@ class Frob(factory.django.DjangoModelFactory):
    model = models.Frob
    ```

    Using [post-generation hooks](https://factoryboy.readthedocs.io/en/stable/reference.html#factory.post_generation):
    Using
    [post-generation hooks](https://factoryboy.readthedocs.io/en/stable/reference.html#factory.post_generation):

    ```py
    class MyFactory(factory.Factory):
    @@ -87,8 +94,8 @@ MyFactory(

    ### Creating other forms of fixture

    Factory boy can also be used to create other object types, such as dicts. Do this by
    specifying the class to be instantiated in the `Meta.model` field:
    Factory boy can also be used to create other object types, such as dicts. Do
    this by specifying the class to be instantiated in the `Meta.model` field:

    ```py
    import factory
    @@ -123,10 +130,8 @@ class AwkwardDict(factory.DictFactory):
    assert AwkwardDict() == {"from": "Person", "is-nice": False}
    ```


    This is useful for writing concise tests that pass a complex object as an input.


    ## Mocks

    Python's mock library is very flexible. It's helpful to distinguish between two
    @@ -141,7 +146,6 @@ ways that mock objects are used:
    Equivalently you can think of mocks as either being actors (stubs) or critics
    (spys).


    ## Stubbing

    Stubbing involves replacing an argument or collaborator with your own version so
    @@ -150,8 +154,8 @@ you can specify its behaviour.
    ### Passing stubs as arguments

    When passing stubs as arguments to the target, prefer
    [`mock.create_autospec`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.create_autospec) so
    function/attribute calls are checked.
    [`mock.create_autospec`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.create_autospec)
    so function/attribute calls are checked.

    ```py
    from unittest import mock
    @@ -256,8 +260,9 @@ def test_something():

    ### Overriding Django settings

    Use Django's [`@override_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.override_settings) decorator
    to override scalar settings:
    Use Django's
    [`@override_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.override_settings)
    decorator to override scalar settings:

    ```py
    from django.test import override_settings
    @@ -267,7 +272,8 @@ def test_something():
    ...
    ```

    Pytest-Django includes an equivalent [`settings` Pytest fixture](https://pytest-django.readthedocs.io/en/latest/helpers.html#settings):
    Pytest-Django includes an equivalent
    [`settings` Pytest fixture](https://pytest-django.readthedocs.io/en/latest/helpers.html#settings):

    ```py
    def test_run(settings):
    @@ -276,8 +282,9 @@ def test_run(settings):
    run()
    ```

    Use Django's [`@modify_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.modify_settings) decorator
    to prepend/append to dict settings:
    Use Django's
    [`@modify_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.modify_settings)
    decorator to prepend/append to dict settings:

    ```py
    from django.test import override_settings
    @@ -293,7 +300,6 @@ def test_something_with_middleware():
    Both `override_settings` and `modify_settings` can be used as class decorators
    but only on `TestCase` subclasses.


    ### Controlling the system clock

    Calling the system clock in tests is generally a bad idea as it can lead to
    @@ -307,6 +313,7 @@ def test_something():
    # Can pass a string, date/datetime instance, lambda function or iterable.
    with time_machine.travel(dt, tick=True):
    pass
    pass
    ```

    or [`freezegun`](https://github.com/spulec/freezegun):
    @@ -326,9 +333,9 @@ Note:
    - Within the context block, use `freezegun.move_to(other_dt)` to move time to a
    specified value.

    The freezegun decorator is useful for debugging flakey tests that
    fail when run at certain times (like during the DST changeover day). To recreate
    the flakey fail, freeze time to when the test failed on your CI service:
    The freezegun decorator is useful for debugging flakey tests that fail when run
    at certain times (like during the DST changeover day). To recreate the flakey
    fail, freeze time to when the test failed on your CI service:

    ```py
    @freezegun.freeze_time("2021-03-28T23:15Z")
    @@ -339,12 +346,16 @@ def test_that_failed_last_night():
    ## Spying

    Spying involves replacing an argument to the system-under-test or one of its
    collaborators with a fake version so you can verify how it was called. Spys are
    normally created as mock objects using `mock.create_autospec`.
    collaborators with a fake version so you can verify how it was called.

    ### Asserting a spy was called correctly
    Spys can be created as `unitest.mock.Mock` instances using
    `mock.create_autospec`.

    Here's an example of passing a spy as an argument to the system-under-test:
    If stubs are _actors_, then spys are _critics_.

    ### How to use spys

    Here's an example of passing a spy as an **argument to the system-under-test**:

    ```py
    from unittest import mock
    @@ -362,7 +373,8 @@ def test_client_called_correctly():
    client.do_the_thing.assert_called_with(x=100)
    ```

    Here's an example of using a spy for a collaborator of the system-under-test:
    Here's an example of using a spy for a **collaborator of the
    system-under-test**:

    ```py
    from unittest import mock
    @@ -382,11 +394,15 @@ def test_client_called_correctly(get_client):
    # Check spy was called correctly.
    client.do_the_thing.assert_called_with(x=100)
    ```

    As you can see, the use of dependency injection in the first example leads to
    simpler tests.

    Objects from Python's `unittest.mock` library provide several [`assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called) that can be used
    to verify how a spy was called:
    ### Verifying a spy was called correctly

    Objects from Python's `unittest.mock` library provide several
    [`assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called)
    that can be used to verify how a spy was called:

    - `assert_called`
    - `assert_called_once`
    @@ -396,41 +412,57 @@ to verify how a spy was called:
    - `assert_not_called`
    - `assert_has_calls`

    #### Verifying _all_ calls to a spy

    Note `assert_has_calls` shouldn't be used to check _all_ calls to the spy as it
    won't fail if additional calls are made. For that it's better to use the
    `call_args_list` property. E.g.

    ```py
    assert spy.call_args_list == [
    mock.call(x=1),
    mock.call(x=2),
    ]
    ```

    #### Verifying unordered calls

    If the order in which a spy is called is not important, then use this pattern:

    ```py
    assert len(spy.call_args_list) == 2
    assert mock.call(x=1) in spy.call_args_list
    assert mock.call(x=2) in spy.call_args_list
    ```

    #### Verifying partial calls

    If you only want to make an assertion about _some_ of the arguments passed to a
    spy, use the [`unittest.mock.ANY` helper](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.ANY), which
    pass equality checks with _everything_:
    spy, use the
    [`unittest.mock.ANY` helper](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.ANY),
    which pass equality checks with _everything_:

    ```py
    m.assert_called_with(x=100, y=ANY)
    ```

    ### Extracting information about how spy was called

    Mock objects have several attributes that store how it was called.
    Spys have several attributes that store how they were called.

    ```py
    Mock.called # bool for whether the spy was called
    Mock.call_count # how many times the spy was called
    Mock.call_args # a tuple of (args, kwargs) of how m was LAST called
    Mock.call_args # a tuple of (args, kwargs) of how the spy was LAST called
    Mock.call_args_list # a list of calls
    Mock.method_calls # a list of methods and attributes called
    Mock.mock_calls # a list of ALL calls to the mock (and its methods and attributes)
    Mock.mock_calls # a list of ALL calls to the spy (and its methods and attributes)
    ```

    The `call` objects returned by `Mock.call_args` and `Mock.call_args_list` are
    two-tuples of (positional args, keyword args) but the `call` objects returned by
    `Mock.method_calls` and `Mock.mock_calls` are three-tuples of
    (name, positional args, keyword args).
    `Mock.method_calls` and `Mock.mock_calls` are three-tuples of (name, positional
    args, keyword args).

    Use `unittest.mock.call` objects to make assertions about calls:

    @@ -439,8 +471,10 @@ assert mock_function.call_args_list == [mock.call(x=1), mock.call(x=2)]
    assert mock_object.method_calls == [mock.call.add(1), mock.call.delete(x=1)]
    ```

    To make fine-grained assertions about function or method calls, you
    can use the `call_args` property:
    ```
    To make fine-grained assertions about function or method calls, you can use the
    `call_args` property:
    ```py
    _, call_kwargs = some_mocked_function.call_args
    @@ -490,7 +524,7 @@ system-under-test when the actual value of the argument isn't important.

    ```py
    @mock.patch.object(somemodule, "collaborator")
    def test_passing_sentinel():
    def test_passing_sentinel(collaborator):
    arg = mock.sentinel.BAZ

    somemodule.target(arg)
    @@ -505,7 +539,6 @@ Reading:

    - https://www.seanh.cc/2017/03/17/sentinel/


    ## Controlling external dependencies

    ### Using temporary files
    @@ -533,11 +566,12 @@ def test_csv_import():

    # Call the management command passing the CSV filepath as an argument.
    call_command("import_csv_file", f.name)
    ```
    ````

    The same thing can be done using [Pytest's `tmp_path`
    fixture](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-fixture)
    with provides a [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html) object:
    The same thing can be done using
    [Pytest's `tmp_path` fixture](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-fixture)
    with provides a [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html)
    object:

    ```python
    import csv
    @@ -558,9 +592,15 @@ def test_csv_import(tmp_path):

    Pytest provides a few other fixtures for creating temporary files and folders:

    - [`tmp_path_factory`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-factory-fixture) — a _session_-scoped fixture for creating `pathlib.Path` temporary directories.
    - [`tmpdir`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmpdir-fixture) — a _function_-scoped fixture for creating `py.path.local` temporary directories.
    - [`tmpdir_factory`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmpdir-factory-fixture) — a _session_-scoped fixture for creating `py.path.local` temporary directories.
    - [`tmp_path_factory`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-factory-fixture)
    — a _session_-scoped fixture for creating `pathlib.Path` temporary
    directories.
    - [`tmpdir`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmpdir-fixture) —
    a _function_-scoped fixture for creating `py.path.local` temporary
    directories.
    - [`tmpdir_factory`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmpdir-factory-fixture)
    — a _session_-scoped fixture for creating `py.path.local` temporary
    directories.

    ## Functional testing

    @@ -585,7 +625,6 @@ Follow these patterns when writing functional tests:
    - Ensure all relevant settings are explicitly defined in the test set-up. Don't
    rely on implicit setting values.


    ### Django views

    Use [`django-webtest`](https://github.com/django-webtest/django-webtest) for
    @@ -700,7 +739,6 @@ def test_some_command(runner):
    # Check side-effects.
    ```


    ## Running tests

    ### Capturing output
    @@ -712,25 +750,52 @@ to work but not for `pdb` or `pdbpp`.

    ## Using Pytest fixtures

    ### Shared fixtures

    Fixtures defined in a `conftest.py` module can be used in several ways:

    - Apply to a single test by adding the fixture name as an argument.

    - Apply to every test in a class by decorating with
    `@pytest.mark.usefixtures("...")`.

    - Apply to every test in a module class by defining a module-level `pytestmark`
    variable:

    ```py
    pytestmark = pytest.mark.usefixtures("...")
    ```

    - Apply to every test in a test suite using the `pytest.ini` file:
    ```dosini
    [pytest]
    usefixtures = ...
    ```

    See [docs on the `usefixtures` fixture][usefixtures].

    [usefixtures]:
    https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#use-fixtures-in-classes-and-modules-with-usefixtures

    ### Prefer to inject factories

    It's tricky to configure Pytest fixtures and so it's best to inject a _factory_
    function/class that can be called with configuration arguments.


    ## Writing high quality code and tests

    High quality code is [easy to change](https://codeinthehole.com/tips/easy-to-change/).
    High quality code is
    [easy to change](https://codeinthehole.com/tips/easy-to-change/).

    Some anti-patterns for unit tests:

    - *Lots of mocks* - this indicates your unit under test has to many
    - _Lots of mocks_ - this indicates your unit under test has to many
    collaborators.

    - *Nested mocks* - this indicates your unit under test know intimate details
    - _Nested mocks_ - this indicates your unit under test know intimate details
    about its collaborators (that it shouldn't know).

    - *Mocking indirect collaborators* - it's best to mock the direct collaborators
    - _Mocking indirect collaborators_ - it's best to mock the direct collaborators
    of a unit being tested, not those further down the call chain. Use of
    `mock.patch` (instead of `mock.patch.object)` is a smell of this problem.

    @@ -747,5 +812,8 @@ Rules of thumb:

    Useful talks:

    - [Fast test, slow test](https://www.youtube.com/watch?v=RAxiiRPHS9k&ab_channel=NextDayVideo) by Gary Bernhardt, Pycon 2012
    - [Stop using mocks](https://www.youtube.com/watch?v=rk-f3B-eMkI&ab_channel=PyConUS) by Harry Percival, Pycon 2020
    - [Fast test, slow test](https://www.youtube.com/watch?v=RAxiiRPHS9k&ab_channel=NextDayVideo)
    by Gary Bernhardt, Pycon 2012
    - [Stop using mocks](https://www.youtube.com/watch?v=rk-f3B-eMkI&ab_channel=PyConUS)
    by Harry Percival, Pycon 2020 This includes clients for third party APIs and
    services for talking to the network, file system or database.
  7. codeinthehole revised this gist Jan 26, 2022. 1 changed file with 31 additions and 0 deletions.
    31 changes: 31 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -55,8 +55,16 @@ import factory
    from foobar import models

    class Frob(factory.django.DjangoModelFactory):
    # For fields that need to be unique.
    sequence_field = factory.Sequence(lambda n: "Bar" + n)

    # For fields where we want to compute the value at runtime.
    datetime_field = factory.LazyFunction(datetime.now)

    # For fields computed from the value of other fields.
    computed_field = factory.LazyAttribute(lambda obj: f"foo-{obj.sequence_field}")

    # Referring to other factories.
    bar = factory.SubFactory("tests.factories.foobar.Bar")

    class Meta:
    @@ -91,8 +99,31 @@ class Payload(factory.Factory):

    class Meta:
    model = dict

    assert Payload() == {"name": "Alan", "age": 40}
    ```

    There's also a convenient `factory.DictFactory` class that can used for `dict`
    factories

    If the dict has fields that aren't valid Python keyword args (e.g. they include
    hyphens or shadow built-in keywords like `from`), use the `rename` meta arg:

    ```py
    class AwkwardDict(factory.DictFactory):
    # Named with trailing underscore as we can't use 'from'
    from_ = "Person"

    # Named with underscore as we can't use a hyphen
    is_nice = False

    class Meta:
    rename = {"from_": "from", "is_nice": "is-nice"}

    assert AwkwardDict() == {"from": "Person", "is-nice": False}
    ```


    This is useful for writing concise tests that pass a complex object as an input.


  8. codeinthehole revised this gist Jan 26, 2022. 1 changed file with 23 additions and 0 deletions.
    23 changes: 23 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -9,6 +9,7 @@ Contents:

    * [Set-up and tear-down](#set-up-and-tear-down)
    * [Creating Django model fixtures](#creating-django-model-fixtures)
    * [Creating other forms of fixture](#creating-other-forms-of-fixture)
    * [Mocks](#mocks)
    * [Stubbing](#stubbing)
    * [Passing stubs as arguments](#passing-stubs-as-arguments)
    @@ -26,13 +27,16 @@ Contents:
    * [Controlling external dependencies](#controlling-external-dependencies)
    * [Using temporary files](#using-temporary-files)
    * [Functional testing](#functional-testing)
    * [High quality functional tests](#high-quality-functional-tests)
    * [Django views](#django-views)
    * [Testing error responses](#testing-error-responses)
    * [Filling in forms](#filling-in-forms)
    * [Django management commands](#django-management-commands)
    * [Click commands](#click-commands)
    * [Running tests](#running-tests)
    * [Capturing output](#capturing-output)
    * [Using Pytest fixtures](#using-pytest-fixtures)
    * [Prefer to inject factories](#prefer-to-inject-factories)
    * [Writing high quality code and tests](#writing-high-quality-code-and-tests)
    * [Resources](#resources)

    @@ -73,6 +77,25 @@ MyFactory(
    )
    ```

    ### Creating other forms of fixture

    Factory boy can also be used to create other object types, such as dicts. Do this by
    specifying the class to be instantiated in the `Meta.model` field:

    ```py
    import factory

    class Payload(factory.Factory):
    name = "Alan"
    age = 40

    class Meta:
    model = dict
    ```

    This is useful for writing concise tests that pass a complex object as an input.


    ## Mocks

    Python's mock library is very flexible. It's helpful to distinguish between two
  9. codeinthehole revised this gist Jan 6, 2022. 1 changed file with 22 additions and 3 deletions.
    25 changes: 22 additions & 3 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -244,19 +244,31 @@ but only on `TestCase` subclasses.

    Calling the system clock in tests is generally a bad idea as it can lead to
    flakiness. Better to pass in relevant dates or datetimes, or if that isn't
    possible, use [`freezegun`](https://github.com/spulec/freezegun):
    possible, use [`time_machine`](https://github.com/adamchainz/time-machine):

    ```py
    import time_machine

    def test_something():
    # Can pass a string, date/datetime instance, lambda function or iterable.
    with time_machine.travel(dt, tick=True):
    pass
    ```

    or [`freezegun`](https://github.com/spulec/freezegun):

    ```py
    import freezegun

    def test_something():
    # Can pass a string, date/datetime instance, lambda function or iterable.
    with freezegun.freeze_time(dt):
    with freezegun.freeze_time(dt, tick=True):
    pass
    ```

    Note:

    - Can be used as a decorator
    - Pass `tick=true` to allow time to continue from the specified value.
    - Within the context block, use `freezegun.move_to(other_dt)` to move time to a
    specified value.

    @@ -644,6 +656,13 @@ Default is for pytest to capture but show output if the test fails.
    Use `-s` to prevent output capturing — this is required for `ipdb` breakpoints
    to work but not for `pdb` or `pdbpp`.

    ## Using Pytest fixtures

    ### Prefer to inject factories

    It's tricky to configure Pytest fixtures and so it's best to inject a _factory_
    function/class that can be called with configuration arguments.


    ## Writing high quality code and tests

  10. codeinthehole revised this gist Aug 25, 2021. 1 changed file with 21 additions and 1 deletion.
    22 changes: 21 additions & 1 deletion python-testing.md
    Original file line number Diff line number Diff line change
    @@ -9,8 +9,9 @@ Contents:

    * [Set-up and tear-down](#set-up-and-tear-down)
    * [Creating Django model fixtures](#creating-django-model-fixtures)
    * [Mocks](#mocks)
    * [Stubbing](#stubbing)
    * [Passing mocks as arguments](#passing-mocks-as-arguments)
    * [Passing stubs as arguments](#passing-stubs-as-arguments)
    * [Stubbing Django model instances](#stubbing-django-model-instances)
    * [Stubbing multiple return values](#stubbing-multiple-return-values)
    * [Stubbing function calls that raise an exception](#stubbing-function-calls-that-raise-an-exception)
    @@ -500,6 +501,25 @@ Pytest provides a few other fixtures for creating temporary files and folders:
    End-to-end tests that trigger the system by an external interface such as a HTTP
    request or CLI invocation.

    ### High quality functional tests

    Functional tests will necessarily be slow and fail with less-than-helpful error
    messages. That's ok - the value they provide is regression protection. You can
    sleep well at night knowing that all your units are plumbed together correctly.

    Follow these patterns when writing functional tests:

    - Explicitly comment each phase of a test to explain what is going on. Don't
    rely on the test name or a docstring.

    - Strive to make the test as end-to-end as possible. Exercise the system using
    an external call (like a HTTP request) and only mock calls to external
    services.

    - Ensure all relevant settings are explicitly defined in the test set-up. Don't
    rely on implicit setting values.


    ### Django views

    Use [`django-webtest`](https://github.com/django-webtest/django-webtest) for
  11. codeinthehole revised this gist Aug 5, 2021. 1 changed file with 14 additions and 2 deletions.
    16 changes: 14 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -72,15 +72,27 @@ MyFactory(
    )
    ```

    ## Mocks

    Python's mock library is very flexible. It's helpful to distinguish between two
    ways that mock objects are used:

    - Stubs: where the behaviour of the mock object is specified _before_ the act
    phase of a test

    - Spys: where the ways a mock object has been called are inspected _after_ the
    act phase of a test.

    Equivalently you can think of mocks as either being actors (stubs) or critics
    (spys).

    <!-- -->

    ## Stubbing

    Stubbing involves replacing an argument or collaborator with your own version so
    you can specify its behaviour.

    ### Passing mocks as arguments
    ### Passing stubs as arguments

    When passing stubs as arguments to the target, prefer
    [`mock.create_autospec`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.create_autospec) so
  12. codeinthehole revised this gist Aug 3, 2021. 1 changed file with 24 additions and 2 deletions.
    26 changes: 24 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -25,15 +25,16 @@ Contents:
    * [Controlling external dependencies](#controlling-external-dependencies)
    * [Using temporary files](#using-temporary-files)
    * [Functional testing](#functional-testing)
    * [Django views](#django-views)
    * [Testing error responses](#testing-error-responses)
    * [Filling in forms](#filling-in-forms)
    * [Django management commands](#django-management-commands)
    * [Click commands](#click-commands)
    * [Running tests](#running-tests)
    * [Capturing output](#capturing-output)
    * [Writing high quality code and tests](#writing-high-quality-code-and-tests)
    * [Resources](#resources)

    <!-- -->

    ## Set-up and tear-down

    Tools and patterns for setting up the world how you want it (and cleaning up
    @@ -487,6 +488,27 @@ Pytest provides a few other fixtures for creating temporary files and folders:
    End-to-end tests that trigger the system by an external interface such as a HTTP
    request or CLI invocation.

    ### Django views

    Use [`django-webtest`](https://github.com/django-webtest/django-webtest) for
    testing Django views. It provides a readable API for clicking on buttons and
    [submitting forms](https://docs.pylonsproject.org/projects/webtest/en/latest/forms.html).

    #### Testing error responses

    Pass `status="*"` so 4XX or 5XX responses don't raise an exception.

    #### Filling in forms

    To fill in a multi-checkbox widget, assign a list of the values to select. For
    Django model widgets, this is the PKs of the selected models:

    ```py
    form = page.forms["my_form"]
    form["roles"] = [some_role.pk, other_role.pk]
    response = form.submit()
    ```

    ### Django management commands

    Use something like this:
  13. codeinthehole revised this gist Jul 23, 2021. 1 changed file with 14 additions and 7 deletions.
    21 changes: 14 additions & 7 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -339,20 +339,27 @@ m.assert_called_with(x=100, y=ANY)
    Mock objects have several attributes that store how it was called.

    ```py
    m.called # bool for whether m was called
    m.call_count # how many times m was called
    m.call_args # a tuple of (args, kwargs) of how m was LAST called
    m.call_args_list # a list of calls made to m
    m.method_calls # a list of methods and attributes called on m
    Mock.called # bool for whether the spy was called
    Mock.call_count # how many times the spy was called
    Mock.call_args # a tuple of (args, kwargs) of how m was LAST called
    Mock.call_args_list # a list of calls
    Mock.method_calls # a list of methods and attributes called
    Mock.mock_calls # a list of ALL calls to the mock (and its methods and attributes)
    ```

    The `call` objects returned by `Mock.call_args` and `Mock.call_args_list` are
    two-tuples of (positional args, keyword args) but the `call` objects returned by
    `Mock.method_calls` and `Mock.mock_calls` are three-tuples of
    (name, positional args, keyword args).

    Use `unittest.mock.call` objects to make assertions about calls:

    ```py
    assert m.call_args_list == [mock.call(x=1), mock.call(x=2)]
    assert mock_function.call_args_list == [mock.call(x=1), mock.call(x=2)]
    assert mock_object.method_calls == [mock.call.add(1), mock.call.delete(x=1)]
    ```

    To make assertions about individual arguments that the spy was called with you
    To make fine-grained assertions about function or method calls, you
    can use the `call_args` property:

    ```py
  14. codeinthehole revised this gist Jul 23, 2021. 1 changed file with 22 additions and 5 deletions.
    27 changes: 22 additions & 5 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -366,16 +366,33 @@ assert "succeeded" in call_kwargs["message"]
    You can wrap an object with a mock so that method calls are forwarded on but
    also recorded for later examination:

    For _direct_ collaborators, use something like:

    ```py
    from unittest import mock
    from foobar import f, A
    from foobar.vendors import client
    from foobar import usecases

    def test_client_called_correctly():
    wrapper = mock.Mock(wraps=A())
    def test_injected_client_called_correctly():
    client_spy = mock.Mock(wraps=client)

    usecases.do_the_thing(client_spy, x=100)

    client_spy.some_method.assert_called_with(x=100)
    ```

    For _indirect_ collaborators, use `mock.patch.object`:

    ```py
    from unittest import mock
    from foobar.vendors import client
    from foobar import usecases

    f(wrapper, x=100)
    @mock.patch.object(usecases, "client", wraps=client):
    def test_collaborator_client_called_correctly(client_spy):
    usecases.do_the_thing(x=100)

    wrapper.some_method.assert_called_with(x=100)
    client_spy.some_method.assert_called_with(x=100)
    ```

    ### Checking values with sentinels
  15. codeinthehole revised this gist Jul 20, 2021. 1 changed file with 12 additions and 2 deletions.
    14 changes: 12 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -310,11 +310,21 @@ to verify how a spy was called:

    - `assert_called`
    - `assert_called_once`
    - `assert_called_with`
    - `assert_called_with` (only checks the _last_ call to the spy)
    - `assert_called_once_with`
    - `assert_any_call`
    - `assert_has_calls`
    - `assert_not_called`
    - `assert_has_calls`

    Note `assert_has_calls` shouldn't be used to check _all_ calls to the spy as it
    won't fail if additional calls are made. For that it's better to use the
    `call_args_list` property. E.g.
    ```py
    assert spy.call_args_list == [
    mock.call(x=1),
    mock.call(x=2),
    ]
    ```

    If you only want to make an assertion about _some_ of the arguments passed to a
    spy, use the [`unittest.mock.ANY` helper](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.ANY), which
  16. codeinthehole revised this gist Jul 9, 2021. 1 changed file with 35 additions and 0 deletions.
    35 changes: 35 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -29,6 +29,8 @@ Contents:
    * [Click commands](#click-commands)
    * [Running tests](#running-tests)
    * [Capturing output](#capturing-output)
    * [Writing high quality code and tests](#writing-high-quality-code-and-tests)
    * [Resources](#resources)

    <!-- -->

    @@ -553,3 +555,36 @@ Default is for pytest to capture but show output if the test fails.

    Use `-s` to prevent output capturing — this is required for `ipdb` breakpoints
    to work but not for `pdb` or `pdbpp`.


    ## Writing high quality code and tests

    High quality code is [easy to change](https://codeinthehole.com/tips/easy-to-change/).

    Some anti-patterns for unit tests:

    - *Lots of mocks* - this indicates your unit under test has to many
    collaborators.

    - *Nested mocks* - this indicates your unit under test know intimate details
    about its collaborators (that it shouldn't know).

    - *Mocking indirect collaborators* - it's best to mock the direct collaborators
    of a unit being tested, not those further down the call chain. Use of
    `mock.patch` (instead of `mock.patch.object)` is a smell of this problem.

    Rules of thumb:

    - Design code to use dependency injection and pass in adapters that handle IO.
    This includes clients for third party APIs and services for talking to the
    network, file system or database.

    - Keep IO separate from business logic. You want your business logic to live in
    side-effect free, pure functions.

    ### Resources

    Useful talks:

    - [Fast test, slow test](https://www.youtube.com/watch?v=RAxiiRPHS9k&ab_channel=NextDayVideo) by Gary Bernhardt, Pycon 2012
    - [Stop using mocks](https://www.youtube.com/watch?v=rk-f3B-eMkI&ab_channel=PyConUS) by Harry Percival, Pycon 2020
  17. codeinthehole revised this gist Jul 9, 2021. 1 changed file with 0 additions and 2 deletions.
    2 changes: 0 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -1,7 +1,5 @@
    # Python testing reference

    **This document is a work-in-progress.**

    This document is a reference for common testing patterns in a Django/Python
    project using Pytest.

  18. codeinthehole revised this gist Jun 23, 2021. 1 changed file with 11 additions and 2 deletions.
    13 changes: 11 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -92,10 +92,10 @@ from foobar.vendor import acme
    def test_with_spec():
    # Function stubs will have their arguments checked against real function
    # signature.
    fn = mock.create_autospec(spec=acme.do_thing)
    fn = mock.create_autospec(spec=acme.do_thing, **attributes)

    # Use instance=True when stubbing a class instance.
    client = mock.create_autospec(spec=acme.Client, instance=True)
    client = mock.create_autospec(spec=acme.Client, instance=True, **attributes)
    ```

    - Don't pass instantiated class instances as the `spec` argument, use
    @@ -104,6 +104,15 @@ def test_with_spec():
    - Be aware that `create_autospec` can have poor performance if it needs to
    traverse a large graph of objects.

    - Be aware that you can't stub a `name` attribute when calling
    `mock.create_autospec` or via `mock.Mock()`. Instead, either call
    `mock.configure_mock(name=...)` or assign the `name` attribute in a separate
    statement:
    ```py
    m = mock.create_autospec(spec=SomeClass, instance=True)
    m.name = "..."
    ```

    ### Stubbing Django model instances

    Use the following formula to create stubbed Django model instances that can be
  19. codeinthehole revised this gist Jun 18, 2021. 1 changed file with 44 additions and 0 deletions.
    44 changes: 44 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -28,6 +28,7 @@ Contents:
    * [Using temporary files](#using-temporary-files)
    * [Functional testing](#functional-testing)
    * [Django management commands](#django-management-commands)
    * [Click commands](#click-commands)
    * [Running tests](#running-tests)
    * [Capturing output](#capturing-output)

    @@ -494,6 +495,49 @@ def test_some_command(command, factory):
    # Check side-effects.
    ```

    ### Click commands

    Use something like this:

    ```py
    # tests/functional/conftest.py
    import pytest
    from click.testing import CliRunner

    @pytest.fixture
    def runner():

    yield CliRunner(
    # Provide a dictionary of environment variables so that configuration
    # parsing works. Don't provide any values though - ensure tests specific
    # values relevant to them.
    env=dict(...)
    )


    # tests/functional/test_command.py
    import main
    import freezegun

    def test_some_command(runner):
    # Run command at a fixed point in time, specifying any relevant env vars.
    with freezegun.freeze_time(dt, tick=True):
    result = runner.invoke(
    main.cli,
    args=["name-of-command"],
    catch_exceptions=False,
    env={
    "VENDOR_API_KEY": "xxx",
    },
    )

    # Check exit code.
    assert result.exit_code == 0, result.exception

    # Check side-effects.
    ```


    ## Running tests

    ### Capturing output
  20. codeinthehole revised this gist Jun 10, 2021. 1 changed file with 9 additions and 0 deletions.
    9 changes: 9 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -332,6 +332,15 @@ Use `unittest.mock.call` objects to make assertions about calls:
    assert m.call_args_list == [mock.call(x=1), mock.call(x=2)]
    ```

    To make assertions about individual arguments that the spy was called with you
    can use the `call_args` property:

    ```py
    _, call_kwargs = some_mocked_function.call_args

    assert "succeeded" in call_kwargs["message"]
    ```

    ### Spying without stubbing

    You can wrap an object with a mock so that method calls are forwarded on but
  21. codeinthehole revised this gist Jun 9, 2021. 1 changed file with 16 additions and 4 deletions.
    20 changes: 16 additions & 4 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -268,6 +268,7 @@ def test_client_called_correctly():
    # Pass spy object as an argument.
    usecase.run(client=client, x=100)

    # Check spy was called correctly.
    client.do_the_thing.assert_called_with(x=100)
    ```

    @@ -288,11 +289,14 @@ def test_client_called_correctly(get_client):
    # a `get_client` factory function.
    usecase.run(x=100)

    # Check spy was called correctly.
    client.do_the_thing.assert_called_with(x=100)
    ```
    As you can see, the use of dependency injection in the first example leads to
    simpler tests.

    Objects from Python's `mock` library provide [several `assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called) that can be used
    to verifying how a spy was called:
    Objects from Python's `unittest.mock` library provide several [`assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called) that can be used
    to verify how a spy was called:

    - `assert_called`
    - `assert_called_once`
    @@ -302,6 +306,13 @@ to verifying how a spy was called:
    - `assert_has_calls`
    - `assert_not_called`

    If you only want to make an assertion about _some_ of the arguments passed to a
    spy, use the [`unittest.mock.ANY` helper](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.ANY), which
    pass equality checks with _everything_:

    ```py
    m.assert_called_with(x=100, y=ANY)
    ```

    ### Extracting information about how spy was called

    @@ -315,7 +326,7 @@ m.call_args_list # a list of calls made to m
    m.method_calls # a list of methods and attributes called on m
    ```

    Use `mock.call` objects to make assertions about calls:
    Use `unittest.mock.call` objects to make assertions about calls:

    ```py
    assert m.call_args_list == [mock.call(x=1), mock.call(x=2)]
    @@ -341,7 +352,8 @@ def test_client_called_correctly():
    ### Checking values with sentinels

    [Sentinels](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.sentinel)
    provide on-demand unique objects and are useful to passing into targets:
    provide on-demand unique objects and are useful for passing into the
    system-under-test when the actual value of the argument isn't important.

    ```py
    @mock.patch.object(somemodule, "collaborator")
  22. codeinthehole revised this gist Jun 9, 2021. 1 changed file with 6 additions and 2 deletions.
    8 changes: 6 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -2,21 +2,25 @@

    **This document is a work-in-progress.**

    This document is a reference for common patterns in Python testing with Pytest.
    This document is a reference for common testing patterns in a Django/Python
    project using Pytest.

    Contents:

    <!-- Run :InsertToc to update -->

    * [Set-up and tear-down](#set-up-and-tear-down)
    * [Creating Django model fixtures](#creating-django-model-fixtures)
    * [Stubbing](#stubbing)
    * [Passing mocks as arguments](#passing-mocks-as-arguments)
    * [Stubbing Django model instances](#stubbing-django-model-instances)
    * [Stubbing multiple return values](#stubbing-multiple-return-values)
    * [Stubbing function calls that raise an exception](#stubbing-function-calls-that-raise-an-exception)
    * [Stubbing HTTP responses](#stubbing-http-responses)
    * [Overriding Django settings](#overriding-django-settings)
    * [Controlling the system clock](#controlling-the-system-clock)
    * [Spying](#spying)
    * [Asserting spy was called correctly](#asserting-spy-was-called-correctly)
    * [Asserting a spy was called correctly](#asserting-a-spy-was-called-correctly)
    * [Extracting information about how spy was called](#extracting-information-about-how-spy-was-called)
    * [Spying without stubbing](#spying-without-stubbing)
    * [Checking values with sentinels](#checking-values-with-sentinels)
  23. codeinthehole revised this gist Jun 4, 2021. 1 changed file with 33 additions and 12 deletions.
    45 changes: 33 additions & 12 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -31,7 +31,8 @@ Contents:

    ## Set-up and tear-down

    Tools and patterns for setting up the world how you want it.
    Tools and patterns for setting up the world how you want it (and cleaning up
    afterwards).

    ### Creating Django model fixtures

    @@ -70,8 +71,8 @@ MyFactory(

    ## Stubbing

    Stubbing involves replacing an object or collaborator with your own version so
    you can tightly control its behaviour.
    Stubbing involves replacing an argument or collaborator with your own version so
    you can specify its behaviour.

    ### Passing mocks as arguments

    @@ -88,13 +89,16 @@ def test_with_spec():
    # signature.
    fn = mock.create_autospec(spec=acme.do_thing)

    # Use instance=True when stubbing an instance object.
    # Use instance=True when stubbing a class instance.
    client = mock.create_autospec(spec=acme.Client, instance=True)
    ```

    - Don't pass instantiated class instances as the `spec` argument, use
    `instance=True` instead.

    - Be aware that `create_autospec` can have poor performance if it needs to
    traverse a large graph of objects.

    ### Stubbing Django model instances

    Use the following formula to create stubbed Django model instances that can be
    @@ -115,9 +119,12 @@ def test_django_model_instance():
    )
    ```

    This is useful for writing isolated unit tests that involve Django model
    instances.

    ### Stubbing multiple return values

    Assign an iterable to `side_effect`:
    Assign an iterable as a mock `side_effect`:

    ```py
    stub = mock.create_autospec(spec=SomeClass)
    @@ -128,9 +135,22 @@ assert stub.method() == 2
    assert stub.method() == 3
    ```

    ### Stubbing function calls that raise an exception

    Assign an exception as a mock `side_effect`:

    ```py
    stub = mock.create_autospec(spec=SomeClass)
    stub.method.side_effect = ValueError("Bad!")

    with pytest.raises(ValueError):
    assert stub.method()
    ```

    ### Stubbing HTTP responses

    Use [`responses`](https://pypi.org/project/responses/):
    Use the [`responses`](https://pypi.org/project/responses/) library. It provides
    a decorator and a clean API for stubbing the responses to HTTP requests:

    ```py
    import responses
    @@ -154,8 +174,8 @@ def test_something():

    ### Overriding Django settings

    Use the [`@override_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.override_settings) decorator
    to override scaler settings:
    Use Django's [`@override_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.override_settings) decorator
    to override scalar settings:

    ```py
    from django.test import override_settings
    @@ -168,12 +188,13 @@ def test_something():
    Pytest-Django includes an equivalent [`settings` Pytest fixture](https://pytest-django.readthedocs.io/en/latest/helpers.html#settings):

    ```py
    def test_something(settings):
    def test_run(settings):
    # Assignments to the `settings` object will be reverted when this test completes.
    settings.FOO = 1
    run()
    ```

    Use the [`@modify_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.modify_settings) decorator
    Use Django's [`@modify_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.modify_settings) decorator
    to prepend/append to dict settings:

    ```py
    @@ -183,11 +204,11 @@ from django.test import override_settings
    "prepend": "some.other.thing",
    "append": "some.alternate.thing",
    })
    def test_something():
    def test_something_with_middleware():
    ...
    ```

    Both `override_settings` and `modify_settings` can be used as class decorators
    Both `override_settings` and `modify_settings` can be used as class decorators
    but only on `TestCase` subclasses.


  24. codeinthehole revised this gist May 19, 2021. 1 changed file with 10 additions and 2 deletions.
    12 changes: 10 additions & 2 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -165,7 +165,13 @@ def test_something():
    ...
    ```

    - Can be used a class decorator but only on `TestCase` subclasses.
    Pytest-Django includes an equivalent [`settings` Pytest fixture](https://pytest-django.readthedocs.io/en/latest/helpers.html#settings):

    ```py
    def test_something(settings):
    settings.FOO = 1
    run()
    ```

    Use the [`@modify_settings`](https://docs.djangoproject.com/en/dev/topics/testing/tools/#django.test.modify_settings) decorator
    to prepend/append to dict settings:
    @@ -181,7 +187,9 @@ def test_something():
    ...
    ```

    - Can be used a class decorator but only on `TestCase` subclasses.
    Both `override_settings` and `modify_settings` can be used as class decorators
    but only on `TestCase` subclasses.


    ### Controlling the system clock

  25. codeinthehole revised this gist May 19, 2021. 1 changed file with 6 additions and 1 deletion.
    7 changes: 6 additions & 1 deletion python-testing.md
    Original file line number Diff line number Diff line change
    @@ -229,9 +229,10 @@ from foobar.vendor import acme
    from foobar import usecase

    def test_client_called_correctly():
    # Create spy.
    client = mock.create_autospec(spec=acme.Client, instance=True)

    # Here the client object is passed in as an argument.
    # Pass spy object as an argument.
    usecase.run(client=client, x=100)

    client.do_the_thing.assert_called_with(x=100)
    @@ -246,6 +247,7 @@ from foobar import usecase

    @mock.patch.object(usecase, "get_client")
    def test_client_called_correctly(get_client):
    # Create spy and ensure factory function returns it.
    client = mock.create_autospec(spec=acme.Client, instance=True)
    get_client.return_value = client

    @@ -267,8 +269,11 @@ to verifying how a spy was called:
    - `assert_has_calls`
    - `assert_not_called`


    ### Extracting information about how spy was called

    Mock objects have several attributes that store how it was called.

    ```py
    m.called # bool for whether m was called
    m.call_count # how many times m was called
  26. codeinthehole revised this gist Apr 30, 2021. 1 changed file with 29 additions and 7 deletions.
    36 changes: 29 additions & 7 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -215,27 +215,49 @@ def test_that_failed_last_night():

    ## Spying

    Spying involves replacing an object or collaborator with your own version so
    you can verify how it was called.
    Spying involves replacing an argument to the system-under-test or one of its
    collaborators with a fake version so you can verify how it was called. Spys are
    normally created as mock objects using `mock.create_autospec`.

    ### Asserting spy was called correctly
    ### Asserting a spy was called correctly

    Use
    Here's an example of passing a spy as an argument to the system-under-test:

    ```py
    from unittest import mock
    from foobar.vendor import acme
    from foobar import f
    from foobar import usecase

    def test_client_called_correctly():
    client = mock.create_autospec(spec=acme.Client, instance=True)

    f(client=client, x=100)
    # Here the client object is passed in as an argument.
    usecase.run(client=client, x=100)

    client.do_the_thing.assert_called_with(x=100)
    ```

    [Other `assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called) on `mock.Mock` instances:
    Here's an example of using a spy for a collaborator of the system-under-test:

    ```py
    from unittest import mock
    from foobar.vendor import acme
    from foobar import usecase

    @mock.patch.object(usecase, "get_client")
    def test_client_called_correctly(get_client):
    client = mock.create_autospec(spec=acme.Client, instance=True)
    get_client.return_value = client

    # Here the client object is constructed from within the use case by calling
    # a `get_client` factory function.
    usecase.run(x=100)

    client.do_the_thing.assert_called_with(x=100)
    ```

    Objects from Python's `mock` library provide [several `assert_*` methods](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called) that can be used
    to verifying how a spy was called:

    - `assert_called`
    - `assert_called_once`
  27. codeinthehole revised this gist Apr 30, 2021. 1 changed file with 39 additions and 20 deletions.
    59 changes: 39 additions & 20 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -308,36 +308,55 @@ Reading:
    For tests that need to write something to a file location but we don't leave
    detritus around after the test run is finished.

    This should only be needed where a filepath is an argument, such as functional
    tests.
    This should only be needed where a _filepath_ is an argument to the
    system-under-test, such as functional tests. For other types of tests, it is
    preferable to pass file-like objects as arguments so tests can pass
    `io.StringIO` instances.

    Here's how to create a temporary CSV file using Python's `tempfile` module:

    ```python
    import io
    import csv
    import tempfile

    import pytest
    from django.core.management import call_command

    def test_csv_import():
    with tempfile.NamedTemporaryFile(mode="w") as f:
    # Create CSV file with 4 rows.
    lines = (
    '"EA:E2001BND",01/10/2020,0.584955,-0.229834',
    '"EA:E2001BNI",01/10/2020,0.723305,-0.186026',
    '"EA:E2001BPD",01/10/2020,0.598306,-0.208303',
    '"EA:E2001BPI",01/10/2020,0.723305,-0.186026',
    )
    for line in lines:
    f.write(line + "\n")
    f.flush()

    # Capture output from command.
    stdout = io.StringIO()
    call_command("import_future_alps", f.name, stdout=stdout)

    assert stdout.getvalue() == "Imported 4 records\n"
    writer = csv.writer(f)
    writer.writerow(["EA:E2001BND", "01/10/2020", 0.584955, -0.229834])

    # Call the management command passing the CSV filepath as an argument.
    call_command("import_csv_file", f.name)
    ```

    The same thing can be done using [Pytest's `tmp_path`
    fixture](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-fixture)
    with provides a [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html) object:

    ```python
    import csv

    import pytest
    from django.core.management import call_command

    def test_csv_import(tmp_path):
    # Create temporary CSV file
    csv_file = tmp_path / "temp.csv"
    with csv_file.open("w") as f:
    writer = csv.writer(f)
    writer.writerow(["EA:E2001BND", "01/10/2020", 0.584955, -0.229834])

    # Call the management command passing the CSV filepath as an argument.
    call_command("import_csv_file", csv_file)
    ```

    Pytest provides a few other fixtures for creating temporary files and folders:

    - [`tmp_path_factory`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmp-path-factory-fixture) — a _session_-scoped fixture for creating `pathlib.Path` temporary directories.
    - [`tmpdir`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmpdir-fixture) — a _function_-scoped fixture for creating `py.path.local` temporary directories.
    - [`tmpdir_factory`](https://docs.pytest.org/en/stable/tmpdir.html#the-tmpdir-factory-fixture) — a _session_-scoped fixture for creating `py.path.local` temporary directories.

    ## Functional testing

    End-to-end tests that trigger the system by an external interface such as a HTTP
  28. codeinthehole revised this gist Apr 20, 2021. 1 changed file with 21 additions and 11 deletions.
    32 changes: 21 additions & 11 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -58,10 +58,10 @@ class MyFactory(factory.Factory):
    blah = factory.PostGeneration(lambda obj, create, extracted, **kwargs: 42)

    MyFactory(
    blah=42, # Passed in the 'extracted' argument of the lambda
    blah__foo=1, # Passed in kwargs as 'foo': 1
    blah__baz=2, # Passed in kwargs as 'baz': 2
    blah_bar=3, # Not passed
    blah=42, # Passed in the 'extracted' argument of the lambda
    blah__foo=1, # Passed in kwargs as 'foo': 1
    blah__baz=2, # Passed in kwargs as 'baz': 2
    blah_bar=3, # Not passed
    )
    ```

    @@ -75,18 +75,26 @@ you can tightly control its behaviour.

    ### Passing mocks as arguments

    When passing stubs as arguments to the target, prefer `mock.create_autospec` so
    When passing stubs as arguments to the target, prefer
    [`mock.create_autospec`](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.create_autospec) so
    function/attribute calls are checked.

    ```py
    from unittest import mock
    from foobar.vendor import acme

    def test_with_spec():
    # Use instance=True when using a class as the spec
    # Function stubs will have their arguments checked against real function
    # signature.
    fn = mock.create_autospec(spec=acme.do_thing)

    # Use instance=True when stubbing an instance object.
    client = mock.create_autospec(spec=acme.Client, instance=True)
    ```

    - Don't pass instantiated class instances as the `spec` argument, use
    `instance=True` instead.

    ### Stubbing Django model instances

    Use the following formula to create stubbed Django model instances that can be
    @@ -212,6 +220,8 @@ you can verify how it was called.

    ### Asserting spy was called correctly

    Use

    ```py
    from unittest import mock
    from foobar.vendor import acme
    @@ -238,11 +248,11 @@ def test_client_called_correctly():
    ### Extracting information about how spy was called

    ```py
    m.called # bool for whether m was called
    m.call_count # how many times m was called
    m.call_args # a tuple of (args, kwargs) of how m was LAST called
    m.call_args_list # a list of calls made to m
    m.method_calls # a list of methods and attributes called on m
    m.called # bool for whether m was called
    m.call_count # how many times m was called
    m.call_args # a tuple of (args, kwargs) of how m was LAST called
    m.call_args_list # a list of calls made to m
    m.method_calls # a list of methods and attributes called on m
    ```

    Use `mock.call` objects to make assertions about calls:
  29. codeinthehole revised this gist Apr 16, 2021. 1 changed file with 28 additions and 7 deletions.
    35 changes: 28 additions & 7 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -7,9 +7,10 @@ This document is a reference for common patterns in Python testing with Pytest.
    Contents:

    * [Set-up and tear-down](#set-up-and-tear-down)
    * [Creating Django model instances](#creating-django-model-instances)
    * [Creating Django model fixtures](#creating-django-model-fixtures)
    * [Stubbing](#stubbing)
    * [Passing mocks as arguments](#passing-mocks-as-arguments)
    * [Stubbing Django model instances](#stubbing-django-model-instances)
    * [Stubbing multiple return values](#stubbing-multiple-return-values)
    * [Stubbing HTTP responses](#stubbing-http-responses)
    * [Overriding Django settings](#overriding-django-settings)
    @@ -32,10 +33,9 @@ Contents:

    Tools and patterns for setting up the world how you want it.

    ### Creating Django model instances
    ### Creating Django model fixtures

    For Django models, the basic pattern with [factory
    boy](https://factoryboy.readthedocs.io/) is:
    For Django models, the basic pattern with [factory boy](https://factoryboy.readthedocs.io/) is:

    ```py
    import factory
    @@ -70,8 +70,8 @@ MyFactory(

    ## Stubbing

    That is, replacing an object or collaborator with your own version so you can
    tightly control its behaviour.
    Stubbing involves replacing an object or collaborator with your own version so
    you can tightly control its behaviour.

    ### Passing mocks as arguments

    @@ -87,6 +87,26 @@ def test_with_spec():
    client = mock.create_autospec(spec=acme.Client, instance=True)
    ```

    ### Stubbing Django model instances

    Use the following formula to create stubbed Django model instances that can be
    assigned as foreign keys:

    ```py
    from unittest import mock
    from django.db import models

    def test_django_model_instance():
    instance = mock.create_autospec(
    spec=models.SomeModel,
    instance=True,
    **fields,
    _state=mock.create_autospec(
    spec=models.base.ModelState, spec_set=True, db=None, adding=True
    ),
    )
    ```

    ### Stubbing multiple return values

    Assign an iterable to `side_effect`:
    @@ -187,7 +207,8 @@ def test_that_failed_last_night():

    ## Spying

    For when you want to verify how an object or collaborator was called.
    Spying involves replacing an object or collaborator with your own version so
    you can verify how it was called.

    ### Asserting spy was called correctly

  30. codeinthehole revised this gist Mar 29, 2021. 1 changed file with 30 additions and 0 deletions.
    30 changes: 30 additions & 0 deletions python-testing.md
    Original file line number Diff line number Diff line change
    @@ -13,6 +13,7 @@ Contents:
    * [Stubbing multiple return values](#stubbing-multiple-return-values)
    * [Stubbing HTTP responses](#stubbing-http-responses)
    * [Overriding Django settings](#overriding-django-settings)
    * [Controlling the system clock](#controlling-the-system-clock)
    * [Spying](#spying)
    * [Asserting spy was called correctly](#asserting-spy-was-called-correctly)
    * [Extracting information about how spy was called](#extracting-information-about-how-spy-was-called)
    @@ -154,6 +155,35 @@ def test_something():

    - Can be used a class decorator but only on `TestCase` subclasses.

    ### Controlling the system clock

    Calling the system clock in tests is generally a bad idea as it can lead to
    flakiness. Better to pass in relevant dates or datetimes, or if that isn't
    possible, use [`freezegun`](https://github.com/spulec/freezegun):

    ```py
    import freezegun

    def test_something():
    # Can pass a string, date/datetime instance, lambda function or iterable.
    with freezegun.freeze_time(dt):
    pass
    ```

    - Can be used as a decorator
    - Pass `tick=true` to allow time to continue from the specified value.
    - Within the context block, use `freezegun.move_to(other_dt)` to move time to a
    specified value.

    The freezegun decorator is useful for debugging flakey tests that
    fail when run at certain times (like during the DST changeover day). To recreate
    the flakey fail, freeze time to when the test failed on your CI service:

    ```py
    @freezegun.freeze_time("2021-03-28T23:15Z")
    def test_that_failed_last_night():
    ...
    ```

    ## Spying