Workaround for pytest issue #3542

[barnabasJ]

There seems to be a bug in pytest (#3542) where metafunc.parametrize breaks
fixture scope. By creating a fixture and parameterizing it indirectly
scope can be maintained

[philpep]

Hi, thanks for the PR.
I remember to dig into this issue and made some similar tests with indirect = True several years ago. But IIRC this was failing due to another issue.

Do you have some tests / example alongside your PR ?

[barnabasJ]

I used this to test the workaround

test_fixtures.py:

import pytest


@pytest.fixture(scope='module', params=[1, 2])
def fixture1(request):
    print "fixture1, round: %i" % request.param
    return request.param


@pytest.fixture(scope='module', params=[1, 2])
def fixture2(fixture1,  request):
    print "fixture3, round: %i" % request.param
    return request.param + fixture1


@pytest.fixture(scope='module', params=[1, 2])
def fixture3(request):
    print "fixture4, round: %i" % request.param
    return request.param


class TestScope:

    def test_host1(self, fixture2, host):
        assert 1

    def test_host2(self, fixture3, host):
        assert 1

Output of pytest --setup-show test_fixtures.py
Without the workaround:

====================================================================================================== test session starts ======================================================================================================
platform linux2 -- Python 2.7.13, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /home/inetser/infrastructure-tests/testinfra, inifile:
plugins: xdist-1.22.2, forked-0.2, gitignore-1.3, testinfra-1.14.0
collected 6 items

test_fixtures.py
  SETUP    M fixture1[1]
  SETUP    M fixture2 (fixtures used: fixture1)[1]
  SETUP    M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M host (fixtures used: _testinfra_host)
        test_fixtures.py::TestScope::()::test_host1[1-1-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  SETUP    M fixture3[1]
        test_fixtures.py::TestScope::()::test_host2[1-local] (fixtures used: _testinfra_host, fixture3, host).
  TEARDOWN M fixture2[1]
  SETUP    M fixture2 (fixtures used: fixture1)[2]
  TEARDOWN M host
  TEARDOWN M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M host (fixtures used: _testinfra_host)
        test_fixtures.py::TestScope::()::test_host1[2-1-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  TEARDOWN M fixture2[2]
  TEARDOWN M fixture1[1]
  SETUP    M fixture1[2]
  SETUP    M fixture2 (fixtures used: fixture1)[2]
  TEARDOWN M host
  TEARDOWN M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M host (fixtures used: _testinfra_host)
        test_fixtures.py::TestScope::()::test_host1[2-2-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  TEARDOWN M fixture2[2]
  SETUP    M fixture2 (fixtures used: fixture1)[1]
  TEARDOWN M host
  TEARDOWN M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  SETUP    M host (fixtures used: _testinfra_host)
        test_fixtures.py::TestScope::()::test_host1[1-2-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  TEARDOWN M fixture3[1]
  SETUP    M fixture3[2]
        test_fixtures.py::TestScope::()::test_host2[2-local] (fixtures used: _testinfra_host, fixture3, host).
  TEARDOWN M host
  TEARDOWN M _testinfra_host[<testinfra.host.Host object at 0x7fd42095ee90>]
  TEARDOWN M fixture3[2]
  TEARDOWN M fixture2[1]
  TEARDOWN M fixture1[2]

=================================================================================================== 6 passed in 0.04 seconds ====================================================================================================

With the workaround:

====================================================================================================== test session starts ======================================================================================================
platform linux2 -- Python 2.7.13, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /home/inetser/infrastructure-tests/testinfra, inifile:
plugins: xdist-1.22.2, forked-0.2, gitignore-1.3, testinfra-1.14.0
collected 6 items

test_fixtures.py
  SETUP    M fixture1[1]
  SETUP    M fixture2 (fixtures used: fixture1)[1]
  SETUP    M _testinfra_host[<testinfra.host.Host object at 0x7efd76121e50>]
  SETUP    M host (fixtures used: _testinfra_host)
        test_fixtures.py::TestScope::()::test_host1[1-1-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  TEARDOWN M fixture2[1]
  SETUP    M fixture2 (fixtures used: fixture1)[2]
        test_fixtures.py::TestScope::()::test_host1[2-1-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  TEARDOWN M fixture2[2]
  TEARDOWN M fixture1[1]
  SETUP    M fixture1[2]
  SETUP    M fixture2 (fixtures used: fixture1)[2]
        test_fixtures.py::TestScope::()::test_host1[2-2-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  TEARDOWN M fixture2[2]
  SETUP    M fixture2 (fixtures used: fixture1)[1]
        test_fixtures.py::TestScope::()::test_host1[1-2-local] (fixtures used: _testinfra_host, fixture1, fixture2, host).
  SETUP    M fixture3[1]
        test_fixtures.py::TestScope::()::test_host2[1-local] (fixtures used: _testinfra_host, fixture3, host).
  TEARDOWN M fixture3[1]
  SETUP    M fixture3[2]
        test_fixtures.py::TestScope::()::test_host2[2-local] (fixtures used: _testinfra_host, fixture3, host).
  TEARDOWN M host
  TEARDOWN M _testinfra_host[<testinfra.host.Host object at 0x7efd76121e50>]
  TEARDOWN M fixture3[2]
  TEARDOWN M fixture2[1]
  TEARDOWN M fixture1[2]

=================================================================================================== 6 passed in 0.03 seconds ====================================================================================================

[philpep]

Hi, sorry for having a long time before merging this. I wanted to be sure to not break something...

Here's a sample test which reproduce the issue adressed in this PR (unfortunatelly, it cannot be added to the test suite since our host fixture is override to a function scoped in test/conftest.py

import pytest

V = 0


@pytest.fixture(scope="module")
def f(host):
    global V
    V += 1
    assert V == 1


@pytest.mark.parametrize("p", ["a", "b"])
def test_foo(host, f, p):
    pass

Merged, thanks for the PR !