Build on macOS (UCS TODO List)

[hiroyuki-sato]

tweets

TODO

• UCS:
  • debug/memtrack.c: missing malloc_usable_size
  • sys/event_set.c: EPOLLET alternative in kqueue
  • sys/sock.c: MSG_NOSIGNAL
• UCT:
  • tcp/tcp_base.c: TCP_SYNCNT in setsockopt
• UCP:
  • core/ucp_worker.c: missing sys/eventfd.h
• UCM:
  • util/sys.c: missing linux/mman.h
  • event/event.c: missing SHM_REMAP
  • util/reloc.c: missing link.h

TODO(src/ucs) only

• librt doesn't exists.
• strdupa, CPU_ZERO, CPU_ISSET
  • strdupa ([ucx:3917] UCS/STATS: Stop using strdupa(). It is not available on non-Linux pl)
  • CPU_ZERO
• pthread_spinlock_t not supported on Mac
  • OpenCV way
  • os_unfair_lock
• timer_t missing on Mac OSX
  • timer_t in macOS / iOS
  • timer_t alternative
• shmctl(0, SHM_INFO,.. doesn't available
• link.h doesn't exist
• memalign not available (OpenUCX#4000)
• unknown type name pthread_spinlock_t
• fatal error: 'linux/futex.h' file not found
• implicit declaration of function 'basename' is invalid

Branches.

• event_set_functions: This branch split epoll functions into 'ucs/sys/event_set.{c|h}'.
  Created PR [WIP] UCS/API event_set
  After this branch merged, Next step is to modify src/ucs/async/thread.c for using ucs/sys/event_set.{c,h}
  Add kqueue code in ucs/sys/event_set.{c,h}
• osx_fixes: Fix macOS specific issues.
• kqueue_test: fored osx_fiexed for testing kqueue system call.

Tips

• Build specific code: cd src/ucs ; make V=1 sys/libucs_la-event_set.lo
• Build specific test: GTEST_FILTER='*test_event_set*' make -C test/gtest test

[keisukefukuda]

Just memo:

From @shamisp 's info:

• librt is only for debugging purpose, so it can be disabled
• libucm can be disabled.

[hiroyuki-sato]

Thanks!, @keisukefukuda

I add his tweet

[hiroyuki-sato]

Yossi Itigin comment about async mode in e-mail.

Probably need to wrap this struct def with some #ifdef 's

-----Original Message-----
From: Hiroyuki Sato hiroysato@xxx
Sent: Monday, April 22, 2019 4:11 PM
To: Yossi Itigin yosefe@xxx
Subject: Re: [ucx-group] [Q] async mode (For porting macOS)

Hello, Yossi.

Thank you for your reply.

• spinlock - need some alternative for macos since spinlock is used a lot in the code, not just for async
  OK. We'll find another way.

• signal - we can make it unsupported on macos for now
  It's a good news for us.

• thread - seems kqueue patch should resolve the missing epoll support, right?

Yes, I use kqueue instead of epoll.
https://github.com/hiroyuki-sato/ucx/blob/kqueue_test/src/ucs/async/thread.c
Maybe, this modification almost good, But I have another problem.

I got the following error. timer_t doesn't support macOS. :-(
#10

Best regards.

2019年4月22日(月) 21:57 Yossi Itigin yosefe@xxx:

• spinlock - need some alternative for macos since spinlock is used a lot in the code, not just for async
• signal - we can make it unsupported on macos for now
• thread - seems kqueue patch should resolve the missing epoll support, right?

-----Original Message-----
From: Hiroyuki Sato hiroysato@xxx
Sent: Monday, April 22, 2019 1:10 PM
To: Yossi Itigin yosefe@xxxx
Subject: Re: [ucx-group] [Q] async mode (For porting macOS)

Hello, Yossi.
Thank you for your reply.

There are async mode issues.

• spinlock
  • pthread_spinlock_t not supported on macOS. (It use in ucs/async/thread.c)
  • We made dummy
    function(ba54755))
• signal
  • timer_create not support on macOS (It use in ucs/async/signal.c)
  • I can't find alternative solution yet. (dispatch(3)?)
• thread
  • epoll not support on macOS(It use in ucs/async/thread.c)
  • Now I'm porting thread mode to use kqueue instead of epoll.

Maybe I can port thread mode at first.
If other modes can disable, we focus on other things.

Note.
I update all of the issues about macOS in this issue.
#1

Best regards.

2019年4月22日(月) 18:37 Yossi Itigin yosefe@xxx:

Hi,

Currently async infra has to support all modes, even though 'thread' is actually used.
What are the macOS porting issues? Maybe we can disable 'signal' mode on it.

--Yossi

-----Original Message-----
From: Hiroyuki Sato via ucx-group ucx-group@elist.ornl.gov
Sent: Sunday, April 21, 2019 11:41 AM
To: ucx-group@elist.ornl.gov
Subject: [ucx-group] [Q] async mode (For porting macOS)

Hello, members.

Could you tell me about async mode?

OpenUCX supports multiple async modes(signal, mutex, spinlock, and poll).
For porting macOS, I think It requires one mode only.
(For example, porting signal mode first) Is this correct?
Or, Does it require all modes?

Best regards.

--
Hiroyuki Sato

[Torrekie]

epoll-shim is now ported to Darwin so we can use this to provide epoll and eventfd

[shamisp]

@Torrekie nice !

[hiroyuki-sato]

@Torrekie Thanks!