Enable DEV=y build support on HV=kubevirt #4074

andrewd-zededa · 2024-07-09T22:29:09Z

Use correct build yml for pillar so that image is
built with -dev suffix.

andrewd-zededa · 2024-07-10T20:29:12Z

build failure fixed by recently merged #4073

#41 [build 26/56] RUN gpg --verify openssh-9.8p1.tar.gz.asc openssh-9.8p1.tar.gz
#0 0.099 gpg: can't open 'openssh-9.8p1.tar.gz.asc': No such file or directory
#0 0.099 gpg: verify signatures failed: No such file or directory
Error: error building "lfedge/eve-debug:dfd4456a0f41cfea7577a264976b6cf55e6882a4": error building for arch arm64: failed to solve: process "/bin/sh -c gpg --verify openssh-${OPENSSH_VERSION}.tar.gz.asc openssh-${OPENSSH_VERSION}.tar.gz" did not complete successfully: exit code: 2
2024/07/09 22:36:32 error during command execution: error building "lfedge/eve-debug:dfd4456a0f41cfea7577a264976b6cf55e6882a4": error building for arch arm64: failed to solve: process "/bin/sh -c gpg --verify openssh-${OPENSSH_VERSION}.tar.gz.asc openssh-${OPENSSH_VERSION}.tar.gz" did not complete successfully: exit code: 2
make: *** [Makefile:1035: eve-debug] Error 1
Error: Process completed with exit code 2.

eriknordmark

LGTM

milan-zededa · 2024-07-12T08:06:24Z

build failure fixed by recently merged #4073

#41 [build 26/56] RUN gpg --verify openssh-9.8p1.tar.gz.asc openssh-9.8p1.tar.gz
#0 0.099 gpg: can't open 'openssh-9.8p1.tar.gz.asc': No such file or directory
#0 0.099 gpg: verify signatures failed: No such file or directory
Error: error building "lfedge/eve-debug:dfd4456a0f41cfea7577a264976b6cf55e6882a4": error building for arch arm64: failed to solve: process "/bin/sh -c gpg --verify openssh-${OPENSSH_VERSION}.tar.gz.asc openssh-${OPENSSH_VERSION}.tar.gz" did not complete successfully: exit code: 2
2024/07/09 22:36:32 error during command execution: error building "lfedge/eve-debug:dfd4456a0f41cfea7577a264976b6cf55e6882a4": error building for arch arm64: failed to solve: process "/bin/sh -c gpg --verify openssh-${OPENSSH_VERSION}.tar.gz.asc openssh-${OPENSSH_VERSION}.tar.gz" did not complete successfully: exit code: 2
make: *** [Makefile:1035: eve-debug] Error 1
Error: Process completed with exit code 2.

Then why not to rebase this PR so that we can check that it does not break anything?

andrewd-zededa · 2024-07-12T19:37:32Z

latest push is a rebase, we'll see how it does

andrewd-zededa · 2024-07-12T22:05:28Z

Only arm64 builds failed and I think all identically like this:

#23 1.445 5 errors; 55 MiB in 117 packages
#23 ERROR: process "/dev/.buildkit_qemu_emulator /bin/sh -c eve-alpine-deploy.sh" did not complete successfully: exit code: 5

#18 [build-cross 1/2] FROM docker.io/lfedge/eve-alpine:1f7685f95a475c6bbe682f0b976f12180b6c8726@sha256:fa826e4955485301f44075ba1d9a47bcf7242ec0c1fdf372e5b7e3b208adf45a
#18 sha256:82db86f77d3e39b3f6fdce689f98d1f39f1a20f7953c14aa809e2047320f2d5a 797.97MB / 869.90MB 17.7s
Error: error building "lfedge/eve-pillar:7bd2b408a6eedf63e9f39a30fe92d9e73f646e42": error building for arch arm64: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c eve-alpine-deploy.sh" did not complete successfully: exit code: 5
2024/07/12 19:51:14 error during command execution: error building "lfedge/eve-pillar:7bd2b408a6eedf63e9f39a30fe92d9e73f646e42": error building for arch arm64: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c eve-alpine-deploy.sh" did not complete successfully: exit code: 5
make: *** [Makefile:1041: eve-pillar] Error 1
Error: Process completed with exit code 2.

I'm not sure what to make of this at the moment.

rene · 2024-07-15T09:05:41Z

@eriknordmark , @andrewd-zededa , I've started to see this error recently, @jsfakian also observed on his PR... some Alpine packages are failing from time to time, it needs investigation. In this case iproute2 and radvd packages installation failed, causing the job to fail:

https://github.com/lf-edge/eve/actions/runs/9913215972/job/27390017159?pr=4074#step:9:860

rene · 2024-07-15T09:07:35Z

Triggering failed jobs again, let's see if it works now...

deitch · 2024-07-15T09:18:56Z

In this case iproute2 and radvd packages installation failed, causing the job to fail:

It isn't just those 2 packages, I see bash, crony, iprotue2, radvd, even busybox and ca-certificates.

It is strange, because none of this goes to the Internet; it all uses stuff cached in lfedge/eve-alpine. And it isn't like you added new packages that are missing.

Can you rebase this on latest master? It will simplify the logs and make things quicker, since we got rid of the unnecessary eve-build-ubuntu image on each make command in #4081

andrewd-zededa · 2024-07-15T16:06:43Z

pushed up a rebase on latest master

andrewd-zededa · 2024-07-16T16:32:43Z

Still build failures in apk package installs and all error 127 and I can't reproduce this on my local build environment. My arm builds succeed.

#22 [cross-compile-libs 2/3] RUN eve-alpine-deploy.sh
...
#22 0.494 Executing busybox-1.35.0-r17.trigger
#22 0.500 ERROR: busybox-1.35.0-r17.trigger: script exited with error 127

#23 [collector 2/15] RUN eve-alpine-deploy.sh
...
#23 0.566 (2/115) Installing alpine-baselayout (3.2.0-r23)
#23 0.566 Executing alpine-baselayout-3.2.0-r23.pre-install
#23 0.573 ERROR: alpine-baselayout-3.2.0-r23.pre-install: script exited with error 127
#23 0.582 Executing alpine-baselayout-3.2.0-r23.post-install
#23 0.587 ERROR: alpine-baselayout-3.2.0-r23.post-install: script exited with error 127
...
#23 0.764 (12/115) Installing bash (5.1.16-r2)
#23 0.798 Executing bash-5.1.16-r2.post-install
#23 0.803 ERROR: bash-5.1.16-r2.post-install: script exited with error 127
...
#23 0.969 (22/115) Installing chrony (4.2-r0)
#23 0.972 Executing chrony-4.2-r0.pre-install
#23 0.989 ERROR: chrony-4.2-r0.pre-install: script exited with error 127
...
#23 1.218 (54/115) Installing iproute2 (5.17.0-r0)
#23 1.227 Executing iproute2-5.17.0-r0.post-install
#23 1.232 ERROR: iproute2-5.17.0-r0.post-install: script exited with error 127
...
#23 1.522 (81/115) Installing radvd (2.19-r2)
#23 1.524 Executing radvd-2.19-r2.pre-install
#23 1.530 ERROR: radvd-2.19-r2.pre-install: script exited with error 127

The first failure though is in eve-alpine-deploy.sh and I'm curious if the rest are symptoms of that:

#22 [cross-compile-libs 2/3] RUN eve-alpine-deploy.sh
#22 0.098 ALPINE_VERSION='3.16'
#22 0.098 BB_ASH_VERSION='1.35.0'
#22 0.098 BUILD_PKGS=' '
#22 0.098 EVE_BUILD_ARCH='aarch64'
#22 0.098 EVE_TARGET_ARCH='aarch64'
#22 0.098 FUNCNAME=''
#22 0.098 HOME='/root'
#22 0.098 HOSTNAME='buildkitsandbox'
#22 0.098 IFS='
#22 0.098 '
#22 0.098 LINENO=''
#22 0.098 OPTIND='1'
#22 0.098 OTEL_EXPORTER_OTLP_TRACES_ENDPOINT='unix:///dev/otel-grpc.sock'
#22 0.098 OTEL_EXPORTER_OTLP_TRACES_PROTOCOL='grpc'
#22 0.098 OTEL_TRACES_EXPORTER='otlp'
#22 0.098 OTEL_TRACE_PARENT='00-ee73f16e0530ff9542e1fb7554031bb6-9456862a206c0ace-01'
#22 0.098 PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
#22 0.098 PKGS='musl-dev libgcc libintl libuuid libtirpc libblkid '
#22 0.098 PPID='0'
#22 0.098 PS1='\w $ '
#22 0.098 PS2='> '
#22 0.098 PS4='+ '
#22 0.098 PWD='/'
#22 0.098 SHLVL='2'
#22 0.098 TRACEPARENT='00-ee73f16e0530ff9542e1fb7554031bb6-9456862a206c0ace-01'
#22 0.238 (1/18) Installing ca-certificates-bundle (20220614-r0)
#22 0.255 (2/18) Installing libcrypto1.1 (1.1.1w-r1)
#22 0.305 (3/18) Installing libssl1.1 (1.1.1w-r1)
#22 0.313 (4/18) Installing zlib (1.2.12-r3)
#22 0.318 (5/18) Installing apk-tools (2.12.9-r3)
#22 0.321 (6/18) Installing ssl_client (1.35.0-r17)
#22 0.322 (7/18) Installing libblkid (2.38-r1)
#22 0.327 (8/18) Installing libgcc (11.2.1_git20220219-r2)
#22 0.329 (9/18) Installing libintl (0.21-r2)
#22 0.330 (10/18) Installing libtirpc-conf (1.3.2-r1)
#22 0.331 (11/18) Installing krb5-conf (1.0-r2)
#22 0.332 (12/18) Installing libcom_err (1.46.6-r0)
#22 0.333 (13/18) Installing keyutils-libs (1.6.3-r1)
#22 0.334 (14/18) Installing libverto (0.3.2-r0)
#22 0.335 (15/18) Installing krb5-libs (1.19.4-r0)
#22 0.366 (16/18) Installing libtirpc (1.3.2-r1)
#22 0.370 (17/18) Installing libuuid (2.38-r1)
#22 0.371 (18/18) Installing musl-dev (1.2.3-r0)
#22 0.494 Executing busybox-1.35.0-r17.trigger
#22 0.500 ERROR: busybox-1.35.0-r17.trigger: script exited with error 127
#22 0.505 OK: 19 MiB in 20 packages
#22 DONE 0.9s

deitch · 2024-07-16T16:51:22Z

The alpine-baselayout pre-install script is here. Although that is the latest, and ours might be slightly different, I don't think it is material. The bash post-install script is here.

They are as simple as it gets. What could possibly be failing with so many of them with 127? Normally that means command not found, but what could not be found there?

deitch · 2024-07-16T16:54:25Z

And you aren't seeing it locally, correct? I would worry about full filesystems, but given that other packages are being installed successfully - it is only the scripts - that doesn't seem logical to me.

pkg/pillar/build-kubevirt-dev.yml

Use correct build yml for pillar so that image is built with -dev suffix. Signed-off-by: Andrew Durbin <andrewd@zededa.com>

andrewd-zededa · 2024-07-19T20:55:23Z

latest push was just a rebase on latest

eriknordmark

Run eden

github-actions bot requested a review from eriknordmark July 9, 2024 22:29

eriknordmark approved these changes Jul 11, 2024

View reviewed changes

andrewd-zededa force-pushed the andrewd-kubevirt-dev-support branch from 758e1ec to 10d9d11 Compare July 12, 2024 19:36

github-actions bot requested a review from eriknordmark July 12, 2024 19:37

andrewd-zededa force-pushed the andrewd-kubevirt-dev-support branch from 10d9d11 to d62c9fe Compare July 15, 2024 16:06

europaul reviewed Jul 17, 2024

View reviewed changes

pkg/pillar/build-kubevirt-dev.yml Outdated Show resolved Hide resolved

Enable DEV=y build support on HV=kubevirt

f8f5812

Use correct build yml for pillar so that image is built with -dev suffix. Signed-off-by: Andrew Durbin <andrewd@zededa.com>

andrewd-zededa force-pushed the andrewd-kubevirt-dev-support branch from d62c9fe to f8f5812 Compare July 19, 2024 15:17

andrewd-zededa marked this pull request as ready for review July 19, 2024 20:54

eriknordmark approved these changes Jul 20, 2024

View reviewed changes

andrewd-zededa mentioned this pull request Jul 22, 2024

collect-info: collect longhorn support bundle #4095

Merged

eriknordmark merged commit af68942 into lf-edge:master Jul 23, 2024
39 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable DEV=y build support on HV=kubevirt #4074

Enable DEV=y build support on HV=kubevirt #4074

andrewd-zededa commented Jul 9, 2024

andrewd-zededa commented Jul 10, 2024

eriknordmark left a comment

milan-zededa commented Jul 12, 2024

andrewd-zededa commented Jul 12, 2024

andrewd-zededa commented Jul 12, 2024

rene commented Jul 15, 2024 •

edited

Loading

rene commented Jul 15, 2024

deitch commented Jul 15, 2024

andrewd-zededa commented Jul 15, 2024

andrewd-zededa commented Jul 16, 2024

deitch commented Jul 16, 2024

deitch commented Jul 16, 2024

andrewd-zededa commented Jul 19, 2024

eriknordmark left a comment

Enable DEV=y build support on HV=kubevirt #4074

Enable DEV=y build support on HV=kubevirt #4074

Conversation

andrewd-zededa commented Jul 9, 2024

andrewd-zededa commented Jul 10, 2024

eriknordmark left a comment

Choose a reason for hiding this comment

milan-zededa commented Jul 12, 2024

andrewd-zededa commented Jul 12, 2024

andrewd-zededa commented Jul 12, 2024

rene commented Jul 15, 2024 • edited Loading

rene commented Jul 15, 2024

deitch commented Jul 15, 2024

andrewd-zededa commented Jul 15, 2024

andrewd-zededa commented Jul 16, 2024

deitch commented Jul 16, 2024

deitch commented Jul 16, 2024

andrewd-zededa commented Jul 19, 2024

eriknordmark left a comment

Choose a reason for hiding this comment

rene commented Jul 15, 2024 •

edited

Loading