Cypress signtool unable to locate imgtool during Post-Build

[LDong-Arm]

Description of defect

When compiling any application (e.g. mbed-os-example-blinky) for CYTFM_064B0S2_4343W, we get the following Python-dependency error:

mbed compile -t GCC_ARM -m CYTFM_064B0S2_4343W -v

Post-Build: mbed-os-example-blinky
M0 core image file found: ./mbed-os/targets/TARGET_Cypress/TARGET_PSOC6/TARGET_CYTFM_064B0S2_4343W/COMPONENT_TFM_S_FW/tfm_s.hex.
Policy file found: /Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/targets/TARGET_Cypress/TARGET_PSOC6/TARGET_CYTFM_064B0S2_4343W/policy/policy_multi_CM0_CM4_tfm.json.
[PSOC6.sign_image] Using policy file: /Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/targets/TARGET_Cypress/TARGET_PSOC6/TARGET_CYTFM_064B0S2_4343W/policy/policy_multi_CM0_CM4_tfm.json
Traceback (most recent call last):
  File "/Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/tools/make.py", line 74, in wrapped_build_project
    bin_file, update_file = build_project(
  File "/Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/tools/build_api.py", line 621, in build_project
    res = toolchain.link_program(resources, build_path, name)
  File "/Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/tools/toolchains/mbed_toolchain.py", line 804, in link_program
    post_build_hook(self, r, elf, full_path)
  File "/Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/tools/targets/__init__.py", line 681, in sign_image
    psoc6_sign_image(t_self, resources, elf, binf, m0hexf)
  File "/Users/lindon01/Documents/mbed-os-example-blinky/mbed-os/tools/targets/PSOC6.py", line 180, in sign_image
    import cysecuretools
  File "/usr/local/lib/python3.8/site-packages/cysecuretools/__init__.py", line 16, in <module>
    from cysecuretools.main import CySecureTools
  File "/usr/local/lib/python3.8/site-packages/cysecuretools/main.py", line 47, in <module>
    from cysecuretools.execute.signtool import SignTool
  File "/usr/local/lib/python3.8/site-packages/cysecuretools/execute/signtool.py", line 19, in <module>
    import imgtool.main as imgtool
ModuleNotFoundError: No module named 'imgtool.main'; 'imgtool' is not a package

We have both cysecuretools and imgtool installed through pip. @jainvikas8 and I tried the compilation on multiple host OSs (Windows, Linux, macOS) and Python versions (3.6, 3.7 and 3.8) and had the same issue in all cases.

@ARMmbed/team-cypress

Target(s) affected by this defect ?

CYTFM_064B0S2_4343W

Toolchain(s) (name and version) displaying this defect ?

Not toolchain related.

What version of Mbed-os are you using (tag or sha) ?

mbed-os-6.5.0

What version(s) of tools are you using. List all that apply (E.g. mbed-cli)

• mbed-cli 1.10.4
• cysecuretools 3.0.0
• imgtool 1.7.0a1

I tried downgrading cysecuretools and imgtool but it didn't solve the problem.

How is this defect reproduced ?

Compile any Mbed application for CYTFM_064B0S2_4343W.

[ciarmcom]

Thank you for raising this detailed GitHub issue. I am now notifying our internal issue triagers.
Internal Jira reference: https://jira.arm.com/browse/IOTOSM-2956

[0xc0170]

cc @ARMmbed/team-cypress Please review

[evedon]

I can reproduce the same issue on my MacBook but there are no failures in CI. See a recent PR
http://mbed-os-ci-public.s3-website-eu-west-1.amazonaws.com/?prefix=jenkins-ci/ARMmbed/mbed-os/mbed-os-ci/PR-14144/artifacts/14144/3/build-example-GCC_ARM/PASS/CYTFM_064B0S2_4343W/

[LDong-Arm]

@jainvikas8 and I can also reproduce this issue on Windows and Linux. Maybe the issue is specific to newer versions of Python or packages, but we are not sure.

[RaymondNgun]

This problem is related to this issue: #14153

It seems imgtool is included in mbed-os/tools/psa/tfm/bin_utils. Any reason why it's included in this way? Possible we can delete this version and rely on pip install.

[0xc0170]

@RaymondNgun As we are now porting post build hooks (cypress part almost ready to be merged to master) and TFM (soon), this should be fixed there. Would anyone be able to help ?

[RaymondNgun]

@0xc0170, I don't quite follow. Do you expect this to be fixed in the post build integration stream?

[LDong-Arm]

Fix is here: https://github.com/ARMmbed/mbed-os/pull/14329/files
The idea is that the latest 3.0.0 of cysecuretools isn't compatible with the script in Mbed OS, but 2.x.0 work fine.

[RaymondNgun]

@LDong-Arm, we will likely want to move to newer tools at some point. Correct? Fix it when we get there?

[LDong-Arm]

@RaymondNgun I'll upload a fix soon.
Closing this as the root cause has been identified in #14153, continuing discussions there. (A different target is mentioned there, so reopening this to avoid confusion.)

[LDong-Arm]

See this fix: LDong-Arm@e5901ec
We have imgtool.py in Mbed OS, because Arm Musca targets require a modified/wrapped version of the image tool copied from trusted-firmware-m + MCUboot. But this copy should not be added to the python path, when building for other targets (e.g. Cypress targets). My patch fixes that.

Note that we may not be able to merge that fix as of this week, because we're waiting for some other work to be merged first, to avoid code conflicts.

[RaymondNgun]

@LDong-Arm - That looks great! Thanks!

[LDong-Arm]

PR created: #14362
Closing this one as duplicated and let's continue the conversation in #14153