fabtests: Add extra configure options for efa and lpp tests

[zachdworkin]

add --enable-efa=yes/no --enable-lpp=yes/no, defaulted to yes to turn off these providers if they are not needed to be built.

[shijin-aws]

@zachdworkin @j-xiong Sigh, I think 6901162 should be a better fix for this issue for EFA provider. Sorry that I haven't pushed in that direction earlier

[shijin-aws]

I believe you may have trouble with -lefa?

[zachdworkin]

#10411 broke intel CI because our nodes don't have an updated rdma-core package with the EFADV_DEVICE_ATTR_CAPS_RDMA_WRITE flag. I need some solution to completely disable efa in fabtests. Can we have both options? One will conditionally disable if the header is missing and the other can force disable it.

[shijin-aws]

@zachdworkin sure we can do both

[aingerson]

@shijin-aws I think having an option to disable it is fine, but I also think the efa config needs to fully check for that support by default and be smarter like your fix is doing so our CI shouldn't need the disabling option - I tried your proposed fix and it does not fix the issue for me

[shijin-aws]

@aingerson it needs one more diff, let me open a PR and let you try

[zachdworkin]

@aingerson does your efadv.h file have the added flag? Even if the header is detected it might fail to compile because the package is too old

[shijin-aws]

@zachdworkin I can add AC_CHECK_DECL for EFADV_DEVICE_ATTR_CAPS_RDMA_WRITE

[shijin-aws]

@zachdworkin @aingerson can you test this #10420 ?

[aingerson]

I don't necessarily have a problem with including these changes but I don't think they are really solving an issue and I don't think we should disable the efa and lpp builds on our CI. Our CI should probably try building as many things as possible to catch all build issues we can catch.
The thing I would change about these patches is the name of the config define (EFA and LPP) to make them more descriptive and avoid any potential duplicate defines

[zachdworkin]

I don't necessarily have a problem with including these changes but I don't think they are really solving an issue and I don't think we should disable the efa and lpp builds on our CI. Our CI should probably try building as many things as possible to catch all build issues we can catch. The thing I would change about these patches is the name of the config define (EFA and LPP) to make them more descriptive and avoid any potential duplicate defines

I agree that the CI should build as many things as possible. I can change the names but we need this option until we do our cluster upgrade next quarter. We can remove the command line --enable-efa=no and --enable-lpp=no when that upgrade is complete.

[aingerson]

Why do we need this option? Doesn't Shi's configure change fix it?

[zachdworkin]

I guess we don't need it but it could be nice to have for specific types of builds

[j-xiong]

In libfabric we have per provider configure script that check if a provider can be built and global option to enable/disable providers. We can have the same options in fabtests, so I am fine with both this and #10420 in.

[aingerson]

Yeah, I'm totally fine with it being included - just wanted to clarify that it wasn't actually needed

[zachdworkin]

@shijin-aws whats the aws failure?

[darrylabbate]

bot:aws:retest

[zachdworkin]

@shijin-aws @darrylabbate whats the AWS failure? Rerun doesn't seem to let it pass

[shijin-aws]

@zachdworkin you need to rebase on the latest main branch, there was a bug in efa provider which was fixed by #10421. It's not blocking your PR though

[zachdworkin]

@shijin-aws can you share the AWS CI failure?

[shijin-aws]

It's socket provider failure again

server_command: ssh -n -o StrictHostKeyChecking=no -o ConnectTimeout=30 -o BatchMode=yes 172.31.41.41 'timeout 360 /bin/bash --login -c '"'"'FI_LOG_LEVEL=warn /home/ec2-user/PortaFiducia/build/libraries/libfabric/pr10419-debug/install/fabtests/bin/fi_rdm_tagged_peek -p sockets -s 172.31.41.41'"'"''

client_command: ssh -n -o StrictHostKeyChecking=no -o ConnectTimeout=30 -o BatchMode=yes 172.31.41.41 'timeout 360 /bin/bash --login -c '"'"'FI_LOG_LEVEL=warn /home/ec2-user/PortaFiducia/build/libraries/libfabric/pr10419-debug/install/fabtests/bin/fi_rdm_tagged_peek -p sockets -s 172.31.41.41 172.31.41.41'"'"''
client_stdout:
timeout: the monitored command dumped core

client returncode: 255
server_stdout:
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
fi_cq_sread/fi_cq_read(): functional/rdm_tagged_peek.c:56, ret=-259 (Error available)
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_tx_entry():1959<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
libfabric:130006:1727803785::sockets:ep_data:sock_pe_progress_rx_pe_entry():2037<warn> Peer disconnected: removing fd from pollset: fi_sockaddr_in://172.31.41.41:41875
Sending 10 tagged messages
Waiting for messages to complete

server returncode: 1

[shijin-aws]

bot:aws:retest