List which integration tests can easily be converted to unit tests

[betodealmeida]

Currently most of our Python tests are implemented as "integration" tests under tests/integration_tests, and require setting up a database and populating with data in order to run. This is overkill for many of the tests, and makes it hard to develop Superset since a local test run takes minutes.

@ofekisr created a new directory for proper unit tests. In addition to encouraging people to write proper unit tests under the new tests/unit_tests directory we should also convert the old tests. @betodealmeida started this process, but there's still a lot to do. Fortunately, there are a lot of low hanging fruits.

Straightforward

• commands_test.py
• email_tests.py
• form_tests.py
• logging_configurator_tests.py
• pandas_postprocessing_tests.py
• result_set_tests.py
• viz_tests.py
• databases/schema_tests.py
• db_engine_specs/ascend_tests.py
• db_engine_specs/clickhouse_tests.py
• db_engine_specs/dremio_tests.py
• db_engine_specs/drill_tests.py
• db_engine_specs/druid_tests.py
• db_engine_specs/elasticsearch_tests.py
• db_engine_specs/firebird_tests.py
• db_engine_specs/firebolt_tests.py
• db_engine_specs/gsheets_tests.py
• db_engine_specs/hana_tests.py
• db_engine_specs/impala_tests.py
• db_engine_specs/kylin_tests.py
• db_engine_specs/mssql_tests.py
• db_engine_specs/mysql_tests.py
• db_engine_specs/oracle_tests.py
• db_engine_specs/pinot_tests.py
• db_engine_specs/presto_tests.py
• db_engine_specs/redshift_tests.py
• db_engine_specs/snowflake_tests.py
• db_engine_specs/sqlite_tests.py
• db_engine_specs/trino_tests.py
• security/analytics_db_safety_tests.py
• utils/core_tests.py
• utils/csv_tests.py
• utils/date_parser_tests.py
• utils/decorators_tests.py
• utils/hashing_tests.py
• utils/public_interfaces_test.py

Require fixture(s)

• cache_tests.py
• cli_tests.py
• jinja_context_tests.py
• db_engine_specs/crate_tests.py
• db_engine_specs/hive_tests.py
• db_engine_specs/postgres_tests.py
• utils/encrypt_tests.py

[villebro]

Thanks for bringing this up @betodealmeida ! I agree this is a really valuable initiative taken on by @ofekisr and will have a great impact on backend test quality going forward.

For the existing db_engine_specs integration tests that require the self.get_dttm() fixture, I've created the dttm fixture in tests/unit_tests/common.py which is used in e.g.test_mssql.py.

Converting these are actually a good exercise and a great way to get into the codebase (see #18251 for an example of a db engine spec test migration), so I warmly encourage new backend developers to take a stab at converting a few of these!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[rusackas]

@betodealmeida This one's approaching the 1 year mark since any activity... and it doesn't seem like a bug. I don't quite want to close it as stale, but I'm not sure quite what to do with it. Should we make a project board, and try to get people involved in auditing/tackling the individual items?

[rusackas]

Closing this as stale since it's been silent for so long, and we're trying to steer toward a more actionable Issues backlog. If anyone still wants to take this on, please re-open this issue, open a new Issue with updated context, or raise a PR to address the problems. Thanks!