Fixes for new gt4py default backend

[ninaburg]

This PR fixes the multiple tests failing after recent commits in gt4py:

1. The backend is set explicitly for the advection and one diffusion tests to the previous default (roundtrip) one.
2. Several built-in functions were replaced:
   2.1 np.asarray(<field>) are replaced by <field>.asnumpy()
   2.2 as_1D_sparse_field() are replaced by numpy_to_1D_sparse_field()
   2.3 StridedNeighborOffsetProvider() are replaced by grid.get_offset_provider()
   2.4 np_as_located_field() are replaced by as_field(.., data=...)

[ninaburg]

cscs-ci run

[ninaburg]

launch jenkins spack

[edopao]

This change will make it difficult to test and integrate different backends (for example the dace backend, which I am working on a daily basis). A better solution, in my opinion, is to checkout the last working commit from gt4py, by setting explicitly the SHA1 of the commit (or a git tag) in base-requirements.txt

@havogt Do we expect the embedded backend to be working again anytime soon?

Another solution is to change the default backend for all tests (currently embedded) in pytest_config.py

[halungge]

This change will make it difficult to test and integrate different backends (for example the dace backend, which I am working on a daily basis). A better solution, in my opinion, is to checkout the last working commit from gt4py, by setting explicitly the SHA1 of the commit (or a git tag) in base-requirements.txt

@havogt Do we expect the embedded backend to be working again anytime soon?

Another solution is to change the default backend for all tests (currently embedded) in pytest_config.py

The backend in the pytest_config.py is called embedded, but it actually is the roundtrip. So nothing has changed there. It is picked up for the StencilTest but not for the traditional numpy test format, that we still use in the advection package, changing all of them turned out to be a very big task... But you are right it would be a better solution to reuse the fixture in the advection tests instead of importing the backend directly in all of them.

Otherwise fixing the to gt4py hash might be the best option for a quick solution.

[ninaburg]

Ok I'll submit the change of the gt4py hash to 67a618856331a97530560b1607a7d11c3c3ef802. Thanks for the input.

Update: the problem of the spack tests would still remain because spack retrieves its own gt4py version in its environment file. Thus this still won't unlock icon4py main. Moreover the cluster used for the testing is also included in the maintenance day and no test run.

[ninaburg]

cscs-ci run

[ninaburg]

launch jenkins spack

[halungge]

cscs-ci run

[halungge]

cscs-ci run

[halungge]

cscs-ci run

[halungge]

launch jenkins spack

[ninaburg]

cscs-ci run

[ninaburg]

launch jenkins spack

[halungge]

cscs-ci run

[halungge]

launch jenkins spack

[github-actions]

Mandatory Tests

Please make sure you run these tests via comment before you merge!

• cscs-ci run
• launch jenkins spack

Optional Tests

In case your change might affect downstream icon-exclaim, please consider running

• launch jenkins icon

For more detailed information please look at CI in the EXCLAIM universe.

[ninaburg]

cscs-ci run

[ninaburg]

launch jenkins spack

[samkellerhals]

The changes look good to me. Although all these .asnumpy calls are a bit ugly.. Will we also use the StencilTest infrastructure here at some point? In that way we could hide the asnumpy calls in the future.