[FIXED] pytest new test for every iteration | for loop | parametrize fixture

Issue

Basically, I'm trying to do a test for each iteration of a list of routes to check that the web pages associated with a particular function return a valid status code.

I want something along the lines of this:

import pytest
from flask import url_for
from myflaskapp import get_app

@pytest.yield_fixture
def app():
    # App context and specific overrides for test env
    yield get_app()

@pytest.yield_fixture
def client(app):
    yield app.test_client()

@pytest.yield_fixture
def routes(app):
    routes = [
    'foo',
    'bar',
    # There's quite a lot of function names here
    ]
    with app.app_context():
        for n, route in enumerate(routes):
            routes[n] = url_for(route)
            # yield url_for(route) #NOTE: This would be ideal, but not allowed.
            # convert the routes from func names to actual routes
    yield routes

@pytest.mark.parametrize('route', routes)
def test_page_load(client, route):
    assert client.get(route.endpoint).status_code == 200

I read up that you can't mix parametrize with a fixture as an argument due to something along the lines of interpretation/load/execution order, although, how is this solved in terms of 'best practice'?

I saw a solution where you can generate tests from a function directly, and that seems extremely flexible and might be along the lines of what I want Passing pytest fixture in parametrize (Although I can't use call a fixture decorated function directly, so probably not)

Although, I'm new to pytest and I'd love to see more examples of how to generate tests or perform multiple tests in an iteration with little-to-no restrictions while adhering to proper pytest styling and the DRY principle. (I know about conftest.py)

I'd prioritize versatility/practicality over proper styling if that matters. (within reason, maintainability is a high priority too)

I want to be able to reference the solution to this problem to help guide how I tackle structuring my tests in the future, but I seem to keep hitting roadblocks/limitations or being told by pytest I can't do X solution the way I would expect/want too.

Relevant Posts:

1. DRY: pytest: parameterize fixtures in a DRY way
2. Generate tests from a function: Passing pytest fixture in parametrize
3. Very simply solution (doesn't apply to this case): Parametrize pytest fixture
4. Flask app context in PyTest: Testing code that requires a Flask app or request context
5. Avoiding edge-cases with multiple list fixtures: Why does Pytest perform a nested loop over fixture parameters

Solution

Pytest fixtures themselves can be parameterized, though not with pytest.mark.parametrize. (It looks like this type of question was also answered here.) So:

import pytest
from flask import url_for
from myflaskapp import get_app

@pytest.fixture
def app():
    app = get_app()
    # app context stuff trimmed out here
    return app

@pytest.fixture
def client(app):
    client = app.test_client()
    return client

@pytest.fixture(params=[
    'foo', 
    'bar'
])
def route(request, app):
    '''GET method urls that we want to perform mass testing on'''
    with app.app_context():
        return url_for(request.param)


def test_page_load(client, route):
    assert client.get(route.endpoint).status_code == 200

The documentation explains it this way:

Fixture functions can be parametrized in which case they will be called multiple times, each time executing the set of dependent tests, i. e. the tests that depend on this fixture. Test functions usually do not need to be aware of their re-running. Fixture parametrization helps to write exhaustive functional tests for components which themselves can be configured in multiple ways.

Extending the previous example, we can flag the fixture to create two smtp_connection fixture instances which will cause all tests using the fixture to run twice. The fixture function gets access to each parameter through the special request object:

# content of conftest.py
import pytest import smtplib


@pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"])
def smtp_connection(request):
    smtp_connection = smtplib.SMTP(request.param, 587, timeout=5)
    yield smtp_connection
    print("finalizing {}".format(smtp_connection))
    smtp_connection.close()

The main change is the declaration of params with @pytest.fixture, a list of values for each of which the fixture function will execute and can access a value via request.param. No test function code needs to change.

Answered By - pydsigner