[FIXED] How to use pytest fixture with fixture factory?

Issue

In scikit-learn, there is a function parametrize_with_checks() that is used as a pytest fixture factory--it returns a pytest.mark.parametrize fixture, and is called as a decorator with an iterable of estimators, e.g.

@parameterize_with_checks(list_of_estimators)

My issue is that my list of estimators may change (or I may have multiple lists), and each list I am setting up in a fixture.

Here is a M(N)WE:

import pytest
from sklearn.linear_model import LinearRegression
from sklearn.utils.estimator_checks import parametrize_with_checks


@pytest.fixture
def models():
    return (LinearRegression(fit_intercept=flag) for flag in (False, True))


class TestModels:
    @parametrize_with_checks(models)
    def test_1(self, estimator, check):
        check(estimator)
        print("Do other stuff...")

    def test_2(self, models):
        print("Do even more stuff...")

However, models is still a function object when it is passed to parametrize_with_checks, so it throws an error. How can I get around this?

Solution

parametrize_with_checks() can not work with the values set by fixtures. It handles models just as a function, not a fixture. You can access fixtures only from test functions.
So, looks like you need to set the models as a list or make a call to the model function in parametrize_with_checks()

import pytest
from sklearn.linear_model import LinearRegression
from sklearn.utils.estimator_checks import parametrize_with_checks


MODELS = [LinearRegression(fit_intercept=flag) for flag in (False, True)]


@pytest.fixture()
def models():
    return [LinearRegression(fit_intercept=flag) for flag in (False, True)]


class TestModels:
    @parametrize_with_checks(MODELS)
    # OR
    # @parametrize_with_checks(estimators=models())
    def test_1(self, estimator, check):
        check(estimator)
        print("Do other stuff...")

    def test_2(self, models):
        print("Do even more stuff...")

Answered By - Eugeny Okulik