False alarms from pip's dependency resolver caused by conflicts with local packages

[millatello]

briefcase create android

To Reproduce
Steps to reproduce the behavior:

1. Go to cmd
2. execute the command: briefcase create android
3. See error:
   pip's dependency resolver does not currently take into account the
   packages that are installed. This behaviour is the source of the
   following dependency conflicts.
   toga-winforms 0.3.0.dev33 requieres toga-core==0.3.0.dev33, but you
   have toga-core 0.3.0.dev34 which is incompatible.

Expected behavior
Lastly, Android emulator not working

Screenshots

Environment:

• Operating System: Windows 8.1 Pro
• Python version: 3.10.2
• Software versions:
  • Briefcase: 0.3.7
  • Toga: I think so
    [tool.briefcase.app.helloworld.windows]
    requires = [
    'toga-winforms>=0.3.0.dev32'
  • ...

Additional context
Add any other context about the problem here.

[freakboy3742]

The errors you've reported here don't actually indicate a problem - they're warnings generated because your virtual environment has a different version of packages than the ones being installed, but they shouldn't stop anything from running.

The only other indication from this report that anything actually is wrong is the throwaway comment "Lastly, Android emulator not working" - but you haven't provided any details about what that means - like, for example, the logs from when it didn't work. The fact that you're on Windows 8.1 suggests to me that you're on very old hardware, so the issue is almost certainly that the Android toolchain doesn't support your hardware.

I'm going to close this on the basis that the report as provided doesn't indicate a problem; if you are still having problems that you believe indicate a bug in Briefcase, please open a new ticket with the full output of briefcase run android -v.

[mhsmith]

This came up again a few days ago (beeware/toga#1533). I think we should consider it a bug if Briefcase is showing misleading warnings which the user will waste time trying to understand and solve.

pip has a --no-warn-conflicts option which might solve this, but we'll need to check it doesn't disable any other warnings which might be important.

[freakboy3742]

My concern with "--no-warn-conflicts" is that we'd be silencing legitimate conflict warnings - e.g., if you actually defined incompatible toga versions in your pyproject, or some other version inconsistency existed in requirements.

Ultimately, the issue we're facing is that we're using a slightly "off label" feature of pip by using the --target argument to create an isolated environment. pip expects to be invoked by the version of Python that will use the packages that are installed - something that isn't true in for Briefcase.

There are two potential solutions I can think of:

1. Include python as part of the app support package, and use that executable to invoke pip, rather than sys.executable.
2. Run pip inside a virtual environment that is localised to the app.

Approach (1) could be especially interesting, as it provides a potential to fix to some related problems:

1. Support the use of pip in packaged apps #419 - if we have access to a python interpreter in the app, we can invoke python -m pip in an app.
2. There's a known issue with using multiprocessing, as it invokes fork, and the stub app doesn't support the command line arguments needed to support these fork calls.

Historically, I would have been opposed to including a python interpreter due to the size overhead. However, if we use dynamic linking, the executable is potentially a lot smaller; combine that with some tree pruning functionality in Briefcase, and we could have the best of both worlds.

[mhsmith]

I think running pip in a separate virtual environment is definitely the way to go. pip has very strong support for virtual environments, and Python itself now comes with standard library support for creating them. Whereas --target is a rarely-used option which the pip developers don't give much priority to, and as a result it comes with a whole host of bugs and poorly-defined behavior, as I've discovered from using it in Chaquopy.

[mhsmith]

Could you reopen this issue, and rename it to something clearer?