BDD library for the pytest runner

pytest-bdd implements a subset of the Gherkin language to enable automating project requirements testing and to facilitate behavioral driven development.

Unlike many other BDD tools, it does not require a separate runner and benefits from the power and flexibility of pytest. It enables unifying unit and functional tests, reduces the burden of continuous integration server configuration and allows the reuse of test setups.

Pytest fixtures written for unit tests can be reused for setup and actions mentioned in feature steps with dependency injection. This allows a true BDD just-enough specification of the requirements without maintaining any context object containing the side effects of Gherkin imperative declarations.

Install pytest-bdd

pip install pytest-bdd

Example

An example test for a blog hosting software could look like this. Note that pytest-splinter is used to get the browser fixture.

# content of publish_article.feature

Feature: Blog
    A site where you can publish your articles.

    Scenario: Publishing the article
        Given I'm an author user
        And I have an article

        When I go to the article page
        And I press the publish button

        Then I should not see the error message
        And the article should be published  # Note: will query the database

Note that only one feature is allowed per feature file.

# content of test_publish_article.py

from pytest_bdd import scenario, given, when, then

@scenario('publish_article.feature', 'Publishing the article')
def test_publish():
    pass


@given("I'm an author user")
def author_user(auth, author):
    auth['user'] = author.user


@given("I have an article", target_fixture="article")
def article(author):
    return create_test_article(author=author)


@when("I go to the article page")
def go_to_article(article, browser):
    browser.visit(urljoin(browser.url, '/manage/articles/{0}/'.format(article.id)))


@when("I press the publish button")
def publish_article(browser):
    browser.find_by_css('button[name=publish]').first.click()


@then("I should not see the error message")
def no_error_message(browser):
    with pytest.raises(ElementDoesNotExist):
        browser.find_by_css('.message.error').first


@then("the article should be published")
def article_is_published(article):
    article.refresh()  # Refresh the object in the SQLAlchemy session
    assert article.is_published

Scenario decorator

Functions decorated with the scenario decorator behave like a normal test function, and they will be executed after all scenario steps.

from pytest_bdd import scenario, given, when, then

@scenario('publish_article.feature', 'Publishing the article')
def test_publish(browser):
    assert article.title in browser.html

Step aliases

Sometimes, one has to declare the same fixtures or steps with different names for better readability. In order to use the same step function with multiple step names simply decorate it multiple times:

@given("I have an article")
@given("there's an article")
def article(author, target_fixture="article"):
    return create_test_article(author=author)

Note that the given step aliases are independent and will be executed when mentioned.

For example if you associate your resource to some owner or not. Admin user can’t be an author of the article, but articles should have a default author.

Feature: Resource owner
    Scenario: I'm the author
        Given I'm an author
        And I have an article


    Scenario: I'm the admin
        Given I'm the admin
        And there's an article

Step arguments

Often it's possible to reuse steps giving them a parameter(s). This allows to have single implementation and multiple use, so less code. Also opens the possibility to use same step twice in single scenario and with different arguments! And even more, there are several types of step parameter parsers at your disposal (idea taken from behave implementation):

string (the default)
This is the default and can be considered as a null or exact parser. It parses no parameters and matches the step name by equality of strings.

parse (based on: pypi_parse)
Provides a simple parser that replaces regular expressions for step parameters with a readable syntax like {param:Type}. The syntax is inspired by the Python builtin string.format() function. Step parameters must use the named fields syntax of pypi_parse in step definitions. The named fields are extracted, optionally type converted and then used as step function arguments. Supports type conversions by using type converters passed via extra_types

cfparse (extends: pypi_parse, based on: pypi_parse_type)
Provides an extended parser with "Cardinality Field" (CF) support. Automatically creates missing type converters for related cardinality as long as a type converter for cardinality=1 is provided. Supports parse expressions like: * {values:Type+} (cardinality=1..N, many) * {values:Type*} (cardinality=0..N, many0) * {value:Type?} (cardinality=0..1, optional) Supports type conversions (as above).

re
This uses full regular expressions to parse the clause text. You will need to use named groups "(?P<name>...)" to define the variables pulled from the text and passed to your step() function. Type conversion can only be done via converters step decorator argument (see example below).

The default parser is string, so just plain one-to-one match to the keyword definition. Parsers except string, as well as their optional arguments are specified like:

for cfparse parser

from pytest_bdd import parsers

@given(
    parsers.cfparse("there are {start:Number} cucumbers", extra_types={"Number": int}),
    target_fixture="cucumbers",
)
def given_cucumbers(start):
    return {"start": start, "eat": 0}

for re parser

from pytest_bdd import parsers

@given(
    parsers.re(r"there are (?P\d+) cucumbers"),
    converters={"start": int},
    target_fixture="cucumbers",
)
def given_cucumbers(start):
    return {"start": start, "eat": 0}

Example:

Feature: Step arguments
    Scenario: Arguments for given, when, then
        Given there are 5 cucumbers

        When I eat 3 cucumbers
        And I eat 2 cucumbers

        Then I should have 0 cucumbers

The code will look like:

from pytest_bdd import scenarios, given, when, then, parsers


scenarios("arguments.feature")


@given(parsers.parse("there are {start:d} cucumbers"), target_fixture="cucumbers")
def given_cucumbers(start):
    return {"start": start, "eat": 0}


@when(parsers.parse("I eat {eat:d} cucumbers"))
def eat_cucumbers(cucumbers, eat):
    cucumbers["eat"] += eat


@then(parsers.parse("I should have {left:d} cucumbers"))
def should_have_left_cucumbers(cucumbers, left):
    assert cucumbers["start"] - cucumbers["eat"] == left

Example code also shows possibility to pass argument converters which may be useful if you need to postprocess step arguments after the parser.

You can implement your own step parser. It's interface is quite simple. The code can look like:

import re
from pytest_bdd import given, parsers


class MyParser(parsers.StepParser):
    """Custom parser."""

    def __init__(self, name, **kwargs):
        """Compile regex."""
        super().__init__(name)
        self.regex = re.compile(re.sub("%(.+)%", "(?P<\1>.+)", self.name), **kwargs)

    def parse_arguments(self, name):
        """Get step arguments.

        :return: `dict` of step arguments
        """
        return self.regex.match(name).groupdict()

    def is_matching(self, name):
        """Match given name with the step name."""
        return bool(self.regex.match(name))


@given(parsers.parse("there are %start% cucumbers"), target_fixture="cucumbers")
def given_cucumbers(start):
    return {"start": start, "eat": 0}

Override fixtures via given steps

Dependency injection is not a panacea if you have complex structure of your test setup data. Sometimes there's a need such a given step which would imperatively change the fixture only for certain test (scenario), while for other tests it will stay untouched. To allow this, special parameter target_fixture exists in the given decorator:

from pytest_bdd import given

@pytest.fixture
def foo():
    return "foo"


@given("I have injecting given", target_fixture="foo")
def injecting_given():
    return "injected foo"


@then('foo should be "injected foo"')
def foo_is_foo(foo):
    assert foo == 'injected foo'

Feature: Target fixture
    Scenario: Test given fixture injection
        Given I have injecting given
        Then foo should be "injected foo"

In this example, the existing fixture foo will be overridden by given step I have injecting given only for the scenario it's used in.

Sometimes it is also useful to let when and then steps provide a fixture as well. A common use case is when we have to assert the outcome of an HTTP request:

# content of test_blog.py

from pytest_bdd import scenarios, given, when, then

from my_app.models import Article

scenarios("blog.feature")


@given("there is an article", target_fixture="article")
def there_is_an_article():
    return Article()


@when("I request the deletion of the article", target_fixture="request_result")
def there_should_be_a_new_article(article, http_client):
    return http_client.delete(f"/articles/{article.uid}")


@then("the request should be successful")
def article_is_published(request_result):
    assert request_result.status_code == 200

# content of blog.feature

Feature: Blog
    Scenario: Deleting the article
        Given there is an article

        When I request the deletion of the article

        Then the request should be successful

Multiline steps

As Gherkin, pytest-bdd supports multiline steps (a.k.a. Doc Strings). But in much cleaner and powerful way:

Feature: Multiline steps
    Scenario: Multiline step using sub indentation
        Given I have a step with:
            Some
            Extra
            Lines
        Then the text should be parsed with correct indentation

A step is considered as a multiline one, if the next line(s) after it's first line is indented relatively to the first line. The step name is then simply extended by adding further lines with newlines. In the example above, the Given step name will be:

'I have a step with:\nSome\nExtra\nLines'

You can of course register a step using the full name (including the newlines), but it seems more practical to use step arguments and capture lines after first line (or some subset of them) into the argument:

from pytest_bdd import given, then, scenario, parsers


scenarios("multiline.feature")


@given(parsers.parse("I have a step with:\n{content}"), target_fixture="text")
def given_text(content):
    return content


@then("the text should be parsed with correct indentation")
def text_should_be_correct(text):
    assert text == "Some\nExtra\nLines"

Scenarios shortcut

If you have a relatively large set of feature files, it's boring to manually bind scenarios to the tests using the scenario decorator. Of course with the manual approach you get all the power to be able to additionally parametrize the test, give the test function a nice name, document it, etc, but in the majority of the cases you don't need that. Instead, you want to bind all the scenarios found in the features folder(s) recursively automatically, by using the scenarios helper.

from pytest_bdd import scenarios

# assume 'features' subfolder is in this file's directory
scenarios('features')

That's all you need to do to bind all scenarios found in the features folder! Note that you can pass multiple paths, and those paths can be either feature files or feature folders.

from pytest_bdd import scenarios

# pass multiple paths/files
scenarios('features', 'other_features/some.feature', 'some_other_features')

But what if you need to manually bind a certain scenario, leaving others to be automatically bound? Just write your scenario in a "normal" way, but ensure you do it before the call of scenarios helper.

from pytest_bdd import scenario, scenarios

@scenario('features/some.feature', 'Test something')
def test_something():
    pass

# assume 'features' subfolder is in this file's directory
scenarios('features')

In the example above, the test_something scenario binding will be kept manual, other scenarios found in the features folder will be bound automatically.

Scenario outlines

Scenarios can be parametrized to cover multiple cases. These are called Scenario Outlines in Gherkin, and the variable templates are written using angular brackets (e.g. ``).

Example:

# content of scenario_outlines.feature

Feature: Scenario outlines
    Scenario Outline: Outlined given, when, then
        Given there are  cucumbers
        When I eat  cucumbers
        Then I should have  cucumbers

        Examples:
        | start | eat | left |
        |  12   |  5  |  7   |

from pytest_bdd import scenarios, given, when, then, parsers


scenarios("scenario_outlines.feature")


@given(parsers.parse("there are {start:d} cucumbers"), target_fixture="cucumbers")
def given_cucumbers(start):
    return {"start": start, "eat": 0}


@when(parsers.parse("I eat {eat:d} cucumbers"))
def eat_cucumbers(cucumbers, eat):
    cucumbers["eat"] += eat


@then(parsers.parse("I should have {left:d} cucumbers"))
def should_have_left_cucumbers(cucumbers, left):
    assert cucumbers["start"] - cucumbers["eat"] == left

Organizing your scenarios

The more features and scenarios you have, the more important the question of their organization becomes. The things you can do (and that is also a recommended way):

• organize your feature files in the folders by semantic groups:

  features │ ├──frontend │ │ │ └──auth │ │ │ └──login.feature └──backend │ └──auth │ └──login.feature

This looks fine, but how do you run tests only for a certain feature? As pytest-bdd uses pytest, and bdd scenarios are actually normal tests. But test files are separate from the feature files, the mapping is up to developers, so the test files structure can look completely different:

tests
│
└──functional
   │
   └──test_auth.py
      │
      └ """Authentication tests."""
        from pytest_bdd import scenario

        @scenario('frontend/auth/login.feature')
        def test_logging_in_frontend():
            pass

        @scenario('backend/auth/login.feature')
        def test_logging_in_backend():
            pass

For picking up tests to run we can use the tests selection technique. The problem is that you have to know how your tests are organized, knowing only the feature files organization is not enough. Cucumber uses tags as a way of categorizing your features and scenarios, which pytest-bdd supports. For example, we could have:

@login @backend
Feature: Login

  @successful
  Scenario: Successful login

pytest-bdd uses pytest markers as a storage of the tags for the given scenario test, so we can use standard test selection:

pytest -m "backend and login and successful"

The feature and scenario markers are not different from standard pytest markers, and the @ symbol is stripped out automatically to allow test selector expressions. If you want to have bdd-related tags to be distinguishable from the other test markers, use a prefix like bdd. Note that if you use pytest with the --strict option, all bdd tags mentioned in the feature files should be also in the markers setting of the pytest.ini config. Also for tags please use names which are python-compatible variable names, i.e. start with a non-number, only underscores or alphanumeric characters, etc. That way you can safely use tags for tests filtering.

You can customize how tags are converted to pytest marks by implementing the pytest_bdd_apply_tag hook and returning True from it:

def pytest_bdd_apply_tag(tag, function):
    if tag == 'todo':
        marker = pytest.mark.skip(reason="Not implemented yet")
        marker(function)
        return True
    else:
        # Fall back to the default behavior of pytest-bdd
        return None

Test setup

Test setup is implemented within the Given section. Even though these steps are executed imperatively to apply possible side-effects, pytest-bdd is trying to benefit of the PyTest fixtures which is based on the dependency injection and makes the setup more declarative style.

@given("I have a beautiful article", target_fixture="article")
def article():
    return Article(is_beautiful=True)

The target PyTest fixture "article" gets the return value and any other step can depend on it.

Feature: The power of PyTest
    Scenario: Symbolic name across steps
        Given I have a beautiful article
        When I publish this article

The When step is referencing the article to publish it.

@when("I publish this article")
def publish_article(article):
    article.publish()

Many other BDD toolkits operate on a global context and put the side effects there. This makes it very difficult to implement the steps, because the dependencies appear only as the side-effects during run-time and not declared in the code. The "publish article" step has to trust that the article is already in the context, has to know the name of the attribute it is stored there, the type etc.

In pytest-bdd you just declare an argument of the step function that it depends on and the PyTest will make sure to provide it.

Still side effects can be applied in the imperative style by design of the BDD.

Feature: News website
    Scenario: Publishing an article
        Given I have a beautiful article
        And my article is published

Functional tests can reuse your fixture libraries created for the unit-tests and upgrade them by applying the side effects.

@pytest.fixture
def article():
    return Article(is_beautiful=True)


@given("I have a beautiful article")
def i_have_a_beautiful_article(article):
    pass


@given("my article is published")
def published_article(article):
    article.publish()
    return article

This way side-effects were applied to our article and PyTest makes sure that all steps that require the "article" fixture will receive the same object. The value of the "published_article" and the "article" fixtures is the same object.

Fixtures are evaluated only once within the PyTest scope and their values are cached.

Backgrounds

It's often the case that to cover certain feature, you'll need multiple scenarios. And it's logical that the setup for those scenarios will have some common parts (if not equal). For this, there are backgrounds. pytest-bdd implements Gherkin backgrounds for features.

Feature: Multiple site support

  Background:
    Given a global administrator named "Greg"
    And a blog named "Greg's anti-tax rants"
    And a customer named "Wilson"
    And a blog named "Expensive Therapy" owned by "Wilson"

  Scenario: Wilson posts to his own blog
    Given I am logged in as Wilson
    When I try to post to "Expensive Therapy"
    Then I should see "Your article was published."

  Scenario: Greg posts to a client's blog
    Given I am logged in as Greg
    When I try to post to "Expensive Therapy"
    Then I should see "Your article was published."

In this example, all steps from the background will be executed before all the scenario's own given steps, adding a possibility to prepare some common setup for multiple scenarios in a single feature. About best practices for Background, please read Gherkin's Tips for using Background.

Reusing fixtures

Sometimes scenarios define new names for an existing fixture that can be inherited (reused). For example, if we have the pytest fixture:

@pytest.fixture
def article():
   """Test article."""
   return Article()

Then this fixture can be reused with other names using given():

@given('I have a beautiful article')
def i_have_an_article(article):
   """I have an article."""

Reusing steps

It is possible to define some common steps in the parent conftest.py and simply expect them in the child test file.

# content of common_steps.feature

Scenario: All steps are declared in the conftest
    Given I have a bar
    Then bar should have value "bar"

# content of conftest.py

from pytest_bdd import given, then


@given("I have a bar", target_fixture="bar")
def bar():
    return "bar"


@then('bar should have value "bar"')
def bar_is_bar(bar):
    assert bar == "bar"

# content of test_common.py

@scenario("common_steps.feature", "All steps are declared in the conftest")
def test_conftest():
    pass

There are no definitions of steps in the test file. They were collected from the parent conftest.py.

Default steps

Here is the list of steps that are implemented inside pytest-bdd:

given

• trace - enters the pdb debugger via pytest.set_trace()

when

• trace - enters the pdb debugger via pytest.set_trace()

then

• trace - enters the pdb debugger via pytest.set_trace()

Feature file paths

By default, pytest-bdd will use the current module's path as the base path for finding feature files, but this behaviour can be changed in the pytest configuration file (i.e. pytest.ini, tox.ini or setup.cfg) by declaring the new base path in the bdd_features_base_dir key. The path is interpreted as relative to the pytest root directory. You can also override the features base path on a per-scenario basis, in order to override the path for specific tests.

pytest.ini:

[pytest]
bdd_features_base_dir = features/

tests/test_publish_article.py:

from pytest_bdd import scenario


@scenario("foo.feature", "Foo feature in features/foo.feature")
def test_foo():
    pass


@scenario(
    "foo.feature",
    "Foo feature in tests/local-features/foo.feature",
    features_base_dir="./local-features/",
)
def test_foo_local():
    pass

The features_base_dir parameter can also be passed to the @scenario decorator.

Avoid retyping the feature file name

If you want to avoid retyping the feature file name when defining your scenarios in a test file, use functools.partial. This will make your life much easier when defining multiple scenarios in a test file. For example:

# content of test_publish_article.py

from functools import partial

import pytest_bdd


scenario = partial(pytest_bdd.scenario, "/path/to/publish_article.feature")


@scenario("Publishing the article")
def test_publish():
    pass


@scenario("Publishing the article as unprivileged user")
def test_publish_unprivileged():
    pass

You can learn more about functools.partial in the Python docs.

Programmatic step generation

Sometimes you have step definitions that would be much easier to automate rather than writing them manually over and over again. This is common, for example, when using libraries like pytest-factoryboy that automatically creates fixtures. Writing step definitions for every model can become a tedious task.

For this reason, pytest-bdd provides a way to generate step definitions automatically.

The trick is to pass the stacklevel parameter to the given, when, then, step decorators. This will instruct them to inject the step fixtures in the appropriate module, rather than just injecting them in the caller frame.

Let's look at a concrete example; let's say you have a class Wallet that has some amount of each currency:

# contents of wallet.py

import dataclass

@dataclass
class Wallet:
    verified: bool

    amount_eur: int
    amount_usd: int
    amount_gbp: int
    amount_jpy: int

You can use pytest-factoryboy to automatically create model fixtures for this class:

# contents of wallet_factory.py

from wallet import Wallet

import factory
from pytest_factoryboy import register

class WalletFactory(factory.Factory):
    class Meta:
        model = Wallet

    amount_eur = 0
    amount_usd = 0
    amount_gbp = 0
    amount_jpy = 0

register(Wallet)  # creates the "wallet" fixture
register(Wallet, "second_wallet")  # creates the "second_wallet" fixture

Now we can define a function generate_wallet_steps(...) that creates the steps for any wallet fixture (in our case, it will be wallet and second_wallet):

# contents of wallet_steps.py

import re
from dataclasses import fields

import factory
import pytest
from pytest_bdd import given, when, then, scenarios, parsers


def generate_wallet_steps(model_name="wallet", stacklevel=1):
    stacklevel += 1

    human_name = model_name.replace("_", " ")  # "second_wallet" -> "second wallet"

    @given(f"I have a {human_name}", target_fixture=model_name, stacklevel=stacklevel)
    def _(request):
        return request.getfixturevalue(model_name)

    # Generate steps for currency fields:
    for field in fields(Wallet):
        match = re.fullmatch(r"amount_(?P[a-z]{3})", field.name)
        if not match:
            continue
        currency = match["currency"]

        @given(
            parsers.parse(f"I have {{value:d}} {currency.upper()} in my {human_name}"),
            target_fixture=f"{model_name}__amount_{currency}",
            stacklevel=stacklevel,
        )
        def _(value: int) -> int:
            return value

        @then(
            parsers.parse(f"I should have {{value:d}} {currency.upper()} in my {human_name}"),
            stacklevel=stacklevel,
        )
        def _(value: int, _currency=currency, _model_name=model_name) -> None:
            wallet = request.getfixturevalue(_model_name)
            assert getattr(wallet, f"amount_{_currency}") == value

# Inject the steps into the current module
generate_wallet_steps("wallet")
generate_wallet_steps("second_wallet")

This last file, wallet_steps.py, now contains all the step definitions for our "wallet" and "second_wallet" fixtures.

We can now define a scenario like this:

# contents of wallet.feature
Feature: A feature

    Scenario: Wallet EUR amount stays constant
        Given I have 10 EUR in my wallet
        And I have a wallet
        Then I should have 10 EUR in my wallet

    Scenario: Second wallet JPY amount stays constant
        Given I have 100 JPY in my second wallet
        And I have a second wallet
        Then I should have 100 JPY in my second wallet

and finally a test file that puts it all together and run the scenarios:

# contents of test_wallet.py

from pytest_factoryboy import scenarios

from wallet_factory import *  # import the registered fixtures "wallet" and "second_wallet"
from wallet_steps import *  # import all the step definitions into this test file

scenarios("wallet.feature")

Hooks

pytest-bdd exposes several pytest hooks which might be helpful building useful reporting, visualization, etc. on top of it:

• pytest_bdd_before_scenario(request, feature, scenario) - Called before scenario is executed
• pytest_bdd_after_scenario(request, feature, scenario) - Called after scenario is executed (even if one of steps has failed)
• pytest_bdd_before_step(request, feature, scenario, step, step_func) - Called before step function is executed and it's arguments evaluated
• pytest_bdd_before_step_call(request, feature, scenario, step, step_func, step_func_args) - Called before step function is executed with evaluated arguments
• pytest_bdd_after_step(request, feature, scenario, step, step_func, step_func_args) - Called after step function is successfully executed
• pytest_bdd_step_error(request, feature, scenario, step, step_func, step_func_args, exception) - Called when step function failed to execute
• pytest_bdd_step_func_lookup_error(request, feature, scenario, step, exception) - Called when step lookup failed

Browser testing

Tools recommended to use for browser testing:

• pytest-splinter - pytest splinter integration for the real browser testing

Reporting

It's important to have nice reporting out of your bdd tests. Cucumber introduced some kind of standard for json format which can be used for, for example, by this Jenkins plugin.

To have an output in json format:

pytest --cucumberjson=