NEWS: News update for v1.12.0 rc1 #7741

brminich · 2021-12-02T16:10:21Z

What

News update for v.1.12.0 release
Porting v.1.11.x news from v.1.11.x branch

Why ?

Release preparation

How ?

Comparing v1.11.x and v1.12.x branches

brminich · 2021-12-02T16:36:18Z

@Akshay-Venkatesh, can you please add CUDA related features/bug fixes added in ucx 1.12? (if something is missed)

shamisp · 2021-12-02T18:38:00Z

NEWS

+#### Core
+* Added initial support for Go language bindings
+* Added memory invalidation on error detection
+* Added threshold for ep connection matching in UCP


ep-> endpoint

shamisp · 2021-12-02T18:38:48Z

NEWS

+* Added memory invalidation on error detection
+* Added threshold for ep connection matching in UCP
+* Added new objects to VFS (md, component, log_level, etc)
+* Added config variable to specify what loadable modules are needed


config -> configuration

shamisp · 2021-12-02T18:39:29Z

NEWS

 #### UCP
-* Added API for querying UCP library attributes
+* Added new sockaddr private data format
+* Enabled rendezvous and tag sync for all cases with error handling


all cases -> all protocols ?

no, previously RNDV and sync were disabled if user used connect to worker address with error handling. Now this restriction is removed

Enabled rendezvous and tag sync protocols when error handling is enabled on the endpoint

shamisp · 2021-12-02T18:42:50Z

NEWS

+* Added usage of mpool set for unexpected eager message to reduce memory consumption
+* Added client_id to ucp_worker_create() and ucp_conn_request_query() APIs
+* Added support for modifying UCT and UCS configs by ucp_config_modify() API
+* Added address versioning to correctly preserve wire compatibility


This is the big one. Probably should be moved to the top. Also, we should explicitly specify from what version we are wire protocol backward compatible.

shamisp · 2021-12-02T19:59:26Z

NEWS

+* Added address versioning to correctly preserve wire compatibility
+* Optimized unpacked rkeys memory consumption
+* Added request flag to influence latency vs. bandwidth protocol
+* Added ucp_worker_address_query() API


Can you please group all API changes together ?

shamisp · 2021-12-02T20:03:53Z

NEWS

+#### CUDA
+* Added option to set cuda_copy bandwidth
+* Added profiling of CUDA runtime function calls
+* Added stub for memory invalidation support


I'm not sure if we need to list "stub" functions - seems like does not contribute to anything (you have few assurances here).

this is important change, because it allows these transports to be selected for RNDV protocol.
Remove?

I would rephrase this: Added stub for memory invalidation support inorder to enable CUDA transport selection for rendezvous protocols

wdyt Added stub for memory invalidation support to enable CUDA-IPC transport selection for rendezvous protocols in case of error handling enabled?

Actually, I would make it a bit shorter Added stub for memory invalidation support to enable CUDA-IPC transport selection for rendezvous protocols

i would remove this altogether, it's part of memory invalidation err flows fix

shamisp · 2021-12-02T20:04:40Z

NEWS

+* Added process placement option for ucx_info
+* Extended parameters correctness check in ucx_perftest
+#### CI
+* Replaced gtest 1.7 with gtest 1.10


Updated gtest 1.7 to 1.10

shamisp · 2021-12-02T20:05:07Z

NEWS

+
+### Bugfixes
+#### Core
+* Fixed simultaneous ep close with ucp_hello_world


ep -> endpoint

shamisp · 2021-12-02T20:05:50Z

NEWS

+* Suppressed EHOSTUNREACH error in TCP sockcm
+* Restricted connecting loop-back to other devices in TCP
+#### RDMA CORE (IB, ROCE, etc.)
+* Added pkey_index initialization when creating RC QP with DEVX


s/Added/Fixed

shamisp · 2021-12-02T20:07:04Z

NEWS

+* Fixes in UCP, UCT, UCS, FAQ and README documentation
+#### Tests
+* Fixed memory leak in io_demo
+* More fixes in io_demo


remove...sounds duplicated. Maybe replace the both lines with Multiple fixes

brminich · 2021-12-03T17:40:36Z

@shamisp, fixed

Akshay-Venkatesh · 2021-12-03T21:48:35Z

@Akshay-Venkatesh, can you please add CUDA related features/bug fixes added in ucx 1.12? (if something is missed)

@brminich I'm copying this from Yossi's talk this week. These were the main additions:

added global memtype cache to allow UCT transports to query memory attributes
added capability to select cuda stream based on source and destination memory type required for device-memory-based pipelining
Auto-register cuda whole allocations to avoid repeated registration costs
Select Cuda-IPC capabilities based on NVLINK topology to prefer writes vs reads for specific platforms using NVML
Generalized rendezvous fragment pool per mem-type device

There were some recent bug-fixes but they're part of master and not v1.12.x

NEWS

yosefe · 2021-12-04T13:34:35Z

NEWS

+#### CUDA
+* Added option to set cuda_copy bandwidth
+* Added profiling of CUDA runtime function calls
+* Added stub for memory invalidation support


i would remove this altogether, it's part of memory invalidation err flows fix

NEWS

yosefe · 2021-12-04T13:37:15Z

@petro-rudenko can you pls check didn't miss any news for Java

petro-rudenko · 2021-12-04T16:33:56Z

JUCX:

Set worker id and query it from the connection request.
ucp_listener_reject functionality.
Support setting port 0 for UcpListener to bind on a free port.

yosefe

seems few comments missed from prev review

yosefe · 2021-12-07T15:07:52Z

@shamisp can you pls take a look?

shamisp · 2021-12-07T15:34:03Z

NEWS

 #### UCP
-* Added API for querying UCP library attributes
+* Added address versioning to correctly preserve wire compatibility since v1.11.0


since -> starting from the version v1.11.0

shamisp · 2021-12-07T15:34:19Z

NEWS

 ### Features:
+#### Core
+* Added beta-level support for Go language bindings
+* Added new objects to VFS (md, component, log_level, etc)


etc -> etc.

shamisp · 2021-12-07T15:35:58Z

NEWS

+* Added support for user-defined alignment in Active Messages
+* Added support for offload tag sync in new protocols
+* Updated ucp_atomic_post() to use NBX flow
+##### API


Please move APIs section before UCP. These are the most visible updates.

but this is UCP API changes, it has nested level (extra #)

or you mean move it right after ####UCP?

moved to the top, but removed #####API line, because:

We do not have such section for other parts (UCT, UCS), so it is consistent with other parts - that API changes are just on the top

Otherwise would need to introduce some other caption which would separate API and plain features

shamisp · 2021-12-07T15:41:33Z

NEWS

+* Improved accuracy of the topology distance estimation
+* Added thread-safe put to ptr_map
+* Added prints of leaked callbacks from the callback queue
+* Added new ptr_array API for bulk allocation


Please move API changes to the top of UCS section

shamisp · 2021-12-07T15:41:45Z

NEWS

+* Added ucs_ffs32()
+* Removed a diagnostic message when fuse thread is stopped
+* Added ucs_vsnprintf_safe() which always adds '\0'
+* Added API for a per-process aggregate-sum statistics report


Please move API changes to the top of UCS section

shamisp · 2021-12-07T15:43:58Z

NEWS

+* Added support for setting worker id and querying it from the connection request
+* Added support to bind on a free port in UcpListener
+#### Packaging
+* Added cmake support


Can you please elaborate on this one, we still use auto tools.

afair #7096

Added cmake config files for better integration with external cmake based projects

brminich · 2021-12-08T12:24:05Z

bot:pipe:retest

bureddy · 2021-12-08T20:53:55Z

NEWS

+* Added selection of CUDA-IPC capabilities based on NVLINK topology
+  (to prefer writes vs reads for specific platforms using NVML)
+* Added option to set cuda_copy bandwidth
+* Added profiling of CUDA runtime function calls


Can you add the following in CUDA section to reflect #7772
"Added option to limit GPUDirectRDMA size in rendezvous protocol"

brminich · 2021-12-09T07:34:38Z

@shamisp, @yosefe let's keep it open to be able to update if something else gets into v1.12

brminich · 2021-12-09T15:11:38Z

@tonycurtis, can you please take a look?

tonycurtis · 2021-12-09T15:13:22Z

NEWS

-* Added API for querying UCP library attributes
+* Added client_id to ucp_worker_create() and ucp_conn_request_query() APIs
+* Added ucp_worker_address_query() API
+* Updated ucp_ep_query() API with getting local and remote addresses


for getting?

tonycurtis · 2021-12-09T15:14:11Z

NEWS

+* Added client_id to ucp_worker_create() and ucp_conn_request_query() APIs
+* Added ucp_worker_address_query() API
+* Updated ucp_ep_query() API with getting local and remote addresses
+* Added address versioning to correctly preserve wire compatibility starting from the version v1.11.0


remove "the", and the "v" in front of the number seems redundant.

tonycurtis · 2021-12-09T15:16:18Z

NEWS

+* Added memory limit support to memtrack
+#### CUDA
+* Added global memtype cache to allow UCT transports to query memory attributes
+* Auto-register cuda whole allocations to avoid repeated registration costs


cuda -> CUDA

tonycurtis · 2021-12-09T15:16:41Z

NEWS

+* Added capability to select CUDA stream based on source and destination memory type
+  (required for device memory based pipelining)
+* Added selection of CUDA-IPC capabilities based on NVLINK topology
+  (to prefer writes vs reads for specific platforms using NVML)


vs. (period on end)

tonycurtis · 2021-12-09T15:19:40Z

NEWS

 ### Bugfixes:
+* Fixes in Cuda memory hooks


Cuda -> CUDA

tonycurtis · 2021-12-09T15:20:19Z

NEWS

@@ -68,7 +238,7 @@
 #### RDMA CORE (IB, ROCE, etc.)
 * Added report of QP info in case of completion with error
 * Refactored of FC send operations


ah, not added by this issue, but looks like a missing word here

shamisp · 2021-12-09T18:38:04Z

@shamisp, @yosefe let's keep it open to be able to update if something else gets into v1.12

makes sense. You also want to update Authors files (in separate PR)

brminich · 2021-12-09T18:47:45Z

@shamisp, @yosefe let's keep it open to be able to update if something else gets into v1.12

makes sense. You also want to update Authors files (in separate PR)

Probably I was wrong. Would be good to merge it to not block rc1.
@tonycurtis, your comments applied. If you do not have any new comments I'd squash the changes

shamisp · 2021-12-09T22:30:54Z

@shamisp, @yosefe let's keep it open to be able to update if something else gets into v1.12

makes sense. You also want to update Authors files (in separate PR)

Probably I was wrong. Would be good to merge it to not block rc1. @tonycurtis, your comments applied. If you do not have any new comments I'd squash the changes

You and @tonycurtis have to decide. In the past we have tried to push NEWs on the fist RC but sometimes those were delayed.

tonycurtis · 2021-12-09T22:38:04Z

On Dec 9, 2021, at 5:31 PM, Pavel Shamis (Pasha) ***@***.***> wrote: @shamisp <https://github.com/shamisp>, @yosefe <https://github.com/yosefe> let's keep it open to be able to update if something else gets into v1.12 makes sense. You also want to update Authors files (in separate PR) Probably I was wrong. Would be good to merge it to not block rc1. @tonycurtis <https://github.com/tonycurtis>, your comments applied. If you do not have any new comments I'd squash the changes You and @tonycurtis <https://github.com/tonycurtis> have to decide. In the past we have tried to push NEWs on the fist RC but sometimes those were delayed.

I could convince myself either way. Keep a running update of NEWS; or accumulate a separate PR until the release. I’d go for the latter, if pushed. I think people understand an RC is not a release, so the meta-documentation can wait for a final update. Tony

brminich · 2021-12-10T14:01:35Z

Imo, better have actual NEWs for rc release, because users may/will want to try it and check its content

shamisp · 2021-12-10T14:06:59Z

@brminich agree. If you happen to have ready to go before RC I don't see a good reason not to include. I have seen few some linux distress using our RC.

brminich · 2021-12-10T14:20:36Z

@yosefe, @tonycurtis, so I'm going to merge it unless you have some objections

tonycurtis · 2021-12-10T14:28:45Z

On Dec 10, 2021, at 9:20 AM, Mikhail Brinskiy ***@***.***> wrote: @yosefe <https://github.com/yosefe>, @tonycurtis <https://github.com/tonycurtis>, so I'm going to merge it unless you have some objections

No, please go ahead. Tony

shamisp requested changes Dec 2, 2021

View reviewed changes

shamisp requested a review from tonycurtis December 4, 2021 01:21

yosefe reviewed Dec 4, 2021

View reviewed changes

yosefe reviewed Dec 6, 2021

View reviewed changes

yosefe approved these changes Dec 6, 2021

View reviewed changes

shamisp requested changes Dec 7, 2021

View reviewed changes

shamisp approved these changes Dec 8, 2021

View reviewed changes

yosefe approved these changes Dec 8, 2021

View reviewed changes

bureddy reviewed Dec 8, 2021

View reviewed changes

tonycurtis suggested changes Dec 9, 2021

View reviewed changes

tonycurtis approved these changes Dec 9, 2021

View reviewed changes

NEWS: News update for v1.12.0 rc1

203c100

brminich force-pushed the news/v.1.12 branch from 4c3193b to 203c100 Compare December 9, 2021 18:59

yosefe merged commit 8ab494b into openucx:v1.12.x Dec 10, 2021

NEWS: News update for v1.12.0 rc1 #7741

NEWS: News update for v1.12.0 rc1 #7741

Conversation

brminich commented Dec 2, 2021

What

Why ?

How ?

brminich commented Dec 2, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shamisp Dec 3, 2021 • edited Loading

Choose a reason for hiding this comment

dmitrygx Dec 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brminich commented Dec 3, 2021

Akshay-Venkatesh commented Dec 3, 2021

Choose a reason for hiding this comment

yosefe commented Dec 4, 2021

petro-rudenko commented Dec 4, 2021

yosefe left a comment

Choose a reason for hiding this comment

yosefe commented Dec 7, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brminich commented Dec 8, 2021

bureddy Dec 8, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brminich commented Dec 9, 2021

brminich commented Dec 9, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shamisp commented Dec 9, 2021

brminich commented Dec 9, 2021

shamisp commented Dec 9, 2021

tonycurtis commented Dec 9, 2021 via email

brminich commented Dec 10, 2021

shamisp commented Dec 10, 2021

brminich commented Dec 10, 2021

tonycurtis commented Dec 10, 2021 via email

shamisp Dec 3, 2021 •

edited

Loading

dmitrygx Dec 3, 2021 •

edited

Loading

bureddy Dec 8, 2021 •

edited

Loading