META - testing improvements

[dcmcand]

Context

This is a meta issue to track improvements that need to be made to conda-store's testing.

Value and/or benefit

Our test suite should provide the following:

1. Rapid feedback to developers - we want failures to occur as early as possible in the pipeline so developers can get feedback on their changes rapidly.
2. Relevance - We should test things that we control. In practice this means not testing other people's code. We should be using pinned versions of dependencies for integration tests, mocked responses for end to end tests, and only testing our own code for unit tests.
3. Consistency - We need to ensure that all of our tests are consistent and deterministic. Flaky tests are annoying, slow down feedback loops, and undermine confidence in the system.
4. Confidence - By the end of our test pipeline we should have high confidence that what we release will not break things for users and that our code works as we intend. This includes both for functional requirements, and nonfunctional requirements such as security and accessibility.
5. Comprehensive coverage - all important user journeys and code paths should be comprehensively tested.

Anything else?

No response

Tasks

Beta Give feedback

1. Define and prioritize critical user journeys #619
   area: testing ✅ needs: discussion 💬 type: question 🤔 +3
   
2. Integration Test: Login and create environment #668
   area: CI 👷🏽‍♀️ area: api area: testing ✅ status: in progress 🏗 type: enhancement 💅🏼 +5
3. Integration Test: Login and delete shared environment #669
   area: CI 👷🏽‍♀️ area: testing ✅ +2
   
4. Integration Test: Admin Login and delete old environment #670
   area: CI 👷🏽‍♀️ area: testing ✅ needs: investigation 🔎 +3
   
5. Integration Test: Mark build as active #671
   area: CI 👷🏽‍♀️ area: testing ✅ needs: investigation 🔎 +3
   
6. Integration Test: access logs from failed buld #672
   area: CI 👷🏽‍♀️ area: testing ✅ needs: investigation 🔎 +3
   
7. Integration Test: Developer get lockfile for environment #673
   area: CI 👷🏽‍♀️ area: testing ✅ status: in progress 🏗 +3
8. Integration Test: Cancel a build #674
   area: CI 👷🏽‍♀️ area: testing ✅ needs: review 👀 +3