Implement support for Darwin ppc, fix Darwin ppc64 #7

barracuda156 · 2022-09-17T06:56:37Z

This PR implements initial support for Darwin PPC: adds ppc32 and fixes minor issues with ppc64 (clang should not be used for PPC on MacOS at all).

Feedback welcomed.

Some fixes to the master branch will also be needed, as it seems. I got up to here:

:info:build /opt/local/bin/gcc-mp-12 -o bin/nim c_code/1_1/stdlib_assertions.nim.o c_code/1_1/stdlib_dollars.nim.o c_code/1_1/stdlib_formatfloat.nim.o c_code/3_1/stdlib_io.nim.o c_code/3_1/stdlib_system.nim.o c_code/2_1/stdlib_parseutils.nim.o c_code/2_1/stdlib_math.nim.o c_code/1_1/stdlib_algorithm.nim.o c_code/2_1/stdlib_unicode.nim.o c_code/3_1/stdlib_strutils.nim.o c_code/2_1/stdlib_pathnorm.nim.o c_code/3_1/stdlib_posix.nim.o c_code/3_1/stdlib_times.nim.o c_code/3_1/stdlib_os.nim.o c_code/1_1/stdlib_hashes.nim.o c_code/2_1/stdlib_strtabs.nim.o c_code/1_1/stdlib_sets.nim.o c_code/2_1/@mpathutils.nim.o c_code/2_1/@mropes.nim.o c_code/3_1/stdlib_tables.nim.o c_code/2_1/@mlineinfos.nim.o c_code/3_7/@mplatform.nim.o c_code/1_1/@mprefixmatches.nim.o c_code/2_1/stdlib_strformat.nim.o c_code/2_1/stdlib_terminal.nim.o c_code/3_1/@moptions.nim.o c_code/3_1/@mmsgs.nim.o c_code/1_1/@mcondsyms.nim.o c_code/3_1/stdlib_streams.nim.o c_code/3_1/stdlib_cpuinfo.nim.o c_code/3_1/stdlib_osproc.nim.o c_code/2_3/stdlib_sha1.nim.o c_code/2_1/stdlib_lexbase.nim.o c_code/3_1/stdlib_parsejson.nim.o c_code/3_1/stdlib_json.nim.o c_code/3_1/@mextccomp.nim.o c_code/1_1/@mwordrecg.nim.o c_code/3_1/@mnimblecmd.nim.o c_code/2_3/stdlib_parseopt.nim.o c_code/1_1/@mincremental.nim.o c_code/3_1/@mcommands.nim.o c_code/2_1/@mllstream.nim.o c_code/1_1/@midents.nim.o c_code/1_1/@midgen.nim.o c_code/1_1/@mint128.nim.o c_code/1_1/@mast.nim.o c_code/1_1/@mnimlexbase.nim.o c_code/3_1/@mlexer.nim.o c_code/3_1/@mparser.nim.o c_code/1_1/@mrenderer.nim.o c_code/1_1/@mfilters.nim.o c_code/1_1/@mfilter_tmpl.nim.o c_code/1_1/@msyntaxes.nim.o c_code/1_1/stdlib_intsets.nim.o c_code/3_1/@mrodutils.nim.o c_code/3_1/@mastalgo.nim.o c_code/1_1/@mtrees.nim.o c_code/3_1/@mtypes.nim.o c_code/1_1/@mbtrees.nim.o c_code/1_1/stdlib_md5.nim.o c_code/1_1/@mmodulegraphs.nim.o c_code/1_1/@mmagicsys.nim.o c_code/1_1/@mbitsets.nim.o c_code/1_1/@mnimsets.nim.o c_code/3_1/@msemfold.nim.o c_code/3_1/@mmodulepaths.nim.o c_code/1_1/@mvmdef.nim.o c_code/3_1/@msemdata.nim.o c_code/1_1/@mlinter.nim.o c_code/1_1/@mnimfix@sprettybase.nim.o c_code/1_1/@mlookups.nim.o c_code/3_1/@msemtypinst.nim.o c_code/1_1/@mparampatterns.nim.o c_code/1_1/@mlowerings.nim.o c_code/1_1/@m..@slib@spackages@sdocutils@srstast.nim.o c_code/3_1/@m..@slib@spackages@sdocutils@srst.nim.o c_code/1_1/@m..@slib@spackages@sdocutils@shighlite.nim.o c_code/3_1/@m..@slib@spackages@sdocutils@srstgen.nim.o c_code/1_1/stdlib_xmltree.nim.o c_code/1_1/stdlib_uri.nim.o c_code/1_1/stdlib_cgi.nim.o c_code/1_1/@mtypesrenderer.nim.o c_code/3_1/@mdocgen.nim.o c_code/3_1/@msigmatch.nim.o c_code/1_1/@mimporter.nim.o c_code/1_1/@mprocfind.nim.o c_code/3_1/@mpragmas.nim.o c_code/3_1/@mreorder.nim.o c_code/1_1/@mpasses.nim.o c_code/1_1/@msaturate.nim.o c_code/1_1/@mguards.nim.o c_code/3_1/@msighashes.nim.o c_code/3_1/@mliftdestructors.nim.o c_code/1_1/@msempass2.nim.o c_code/1_1/@mcgmeth.nim.o c_code/1_1/@maliases.nim.o c_code/1_1/@mpatterns.nim.o c_code/3_1/@mdfa.nim.o c_code/3_1/@minjectdestructors.nim.o c_code/1_1/@mliftlocals.nim.o c_code/1_1/@mlambdalifting.nim.o c_code/1_1/@mclosureiters.nim.o c_code/1_1/@mtransf.nim.o c_code/1_1/@mvmgen.nim.o c_code/3_1/@mvmdeps.nim.o c_code/1_1/@mvmmarshal.nim.o c_code/3_1/@mgorgeimpl.nim.o c_code/1_1/@mmacrocacheimpl.nim.o c_code/3_1/@mevaltempl.nim.o c_code/3_1/@mvm.nim.o c_code/1_1/@msemmacrosanity.nim.o c_code/1_1/@mpluginsupport.nim.o c_code/1_1/@mplugins@slocals.nim.o c_code/1_1/@mplugins@sitersgen.nim.o c_code/1_1/@mplugins@sactive.nim.o c_code/1_1/@mspawn.nim.o c_code/1_1/@msemparallel.nim.o c_code/3_1/@msem.nim.o c_code/1_1/@mccgutils.nim.o c_code/1_1/@mtreetab.nim.o c_code/1_1/@mndi.nim.o c_code/1_1/@mcgendata.nim.o c_code/3_1/@mccgmerge.nim.o c_code/1_1/@menumtostr.nim.o c_code/2_1/stdlib_dynlib.nim.o c_code/3_1/@mcgen.nim.o c_code/3_1/@mnimconf.nim.o c_code/1_1/@mpassaux.nim.o c_code/1_1/@mdepends.nim.o c_code/1_1/@mmodules.nim.o c_code/1_1/@mjsgen.nim.o c_code/3_1/@mdocgen2.nim.o c_code/1_1/@mmain.nim.o c_code/3_1/@mscriptconfig.nim.o c_code/3_1/@mcmdlinehelper.nim.o c_code/3_1/@mnim.nim.o -ldl -lm -arch ppc
:info:build : SUCCESS

But then the main code again invokes Clang, which fails, of course. I will fix that in another PR to Nim master branch.

barracuda156 · 2022-09-17T07:53:21Z

P. S. For Rosetta, this part poses a problem and breaks the build:

    if [ "$HOSTTYPE" = "x86_64" ] ; then
      ucpu="amd64"
    fi

I do not know how to fix it in a generally acceptable way at the moment. Locally I just did this:

    if [ "$HOSTTYPE" = "x86_64" ] ; then
      ucpu="ppc"
    fi

This works, but obviously cannot be merged, since it gonna break all Intel builds. If anyone can suggest how to write a special case for Rosetta, it would be greatly appreciated. (On Rosetta the physical cpu is x86*, but the triple is powerpc-apple-darwin10, since ppc32 is emulated by the OS itself.)

barracuda156 · 2022-09-17T13:28:04Z

@Araq When you have time, please take a look. I would like to bring Nim to MacOS PPC, and that should be feasible, provided ppc64 exists (at least potentially – it certainly could not have been built on MacOS with Clang).

(Rosetta issue I mention in the comment above may be safely ignored: I can add a patch for it in Macports when I update Nim port.)

Varriount · 2022-09-20T20:26:50Z

build.sh

@@ -6968,7 +6991,279 @@ c_code/3_2/@mnim.nim.o $LINK_FLAGS
    ;;
  powerpc)
    set -x
-    $CC -o $binDir/nim  $LINK_FLAGS
+    $CC $COMP_FLAGS -Ic_code -c c_code/1_1/stdlib_assertions.nim.c -o c_code/1_1/stdlib_assertions.nim.o


Why were all these added?

@Varriount Otherwise there is nothing to build: ppc32 was just not there for Darwin.

Araq · 2022-09-21T07:03:39Z

Sorry for this rather late reply, but how was this done? I need to replicate your steps.

barracuda156 · 2022-09-21T08:24:21Z

@Araq Thanks for replying, and I will greatly appreciate your involvement here.

To the question: I started from assumption that existing code base is coherent and tested (ppc64 on Mac might not have been, but that is not essential in this context), extracted lists of sources used from build.sh for combinations of platforms and archs, diffed them, turned out there are not many files which are changed within a platform depending on an arch, and most of those are shared within an arch across platforms (example stdlib_sha1.nim.c). The only one unique was @mplatform.nim.c, but diffing that, only code number of processor is changed for a given OS (compare amd64 and ppc64 for Mac), so that file I had to make, and put it into 3_7 (existing folders already have some version of that file).

--- /Users/svacchanda/Desktop/ppc64_mac.sh	2022-09-15 21:42:25.000000000 +0700
+++ /Users/svacchanda/Desktop/amd64_mac.sh	2022-09-15 21:51:54.000000000 +0700
@@ -1,10 +1,10 @@
-  powerpc64)
+  amd64)
     set -x
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_assertions.nim.c -o c_code/1_2/stdlib_assertions.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_dollars.nim.c -o c_code/1_2/stdlib_dollars.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_formatfloat.nim.c -o c_code/1_2/stdlib_formatfloat.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_io.nim.c -o c_code/3_2/stdlib_io.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/3_6/stdlib_system.nim.c -o c_code/3_6/stdlib_system.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_system.nim.c -o c_code/3_2/stdlib_system.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_parseutils.nim.c -o c_code/2_2/stdlib_parseutils.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_math.nim.c -o c_code/2_2/stdlib_math.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/stdlib_algorithm.nim.c -o c_code/1_2/stdlib_algorithm.nim.o
@@ -21,7 +21,7 @@
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/@mropes.nim.c -o c_code/2_2/@mropes.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_tables.nim.c -o c_code/3_2/stdlib_tables.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_4/@mlineinfos.nim.c -o c_code/2_4/@mlineinfos.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/3_6/@mplatform.nim.c -o c_code/3_6/@mplatform.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mplatform.nim.c -o c_code/3_2/@mplatform.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/@mprefixmatches.nim.c -o c_code/1_2/@mprefixmatches.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_4/stdlib_strformat.nim.c -o c_code/2_4/stdlib_strformat.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_terminal.nim.c -o c_code/2_2/stdlib_terminal.nim.o
@@ -31,14 +31,14 @@
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_streams.nim.c -o c_code/3_2/stdlib_streams.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_cpuinfo.nim.c -o c_code/3_2/stdlib_cpuinfo.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_osproc.nim.c -o c_code/3_2/stdlib_osproc.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/2_6/stdlib_sha1.nim.c -o c_code/2_6/stdlib_sha1.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_sha1.nim.c -o c_code/3_2/stdlib_sha1.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_lexbase.nim.c -o c_code/2_2/stdlib_lexbase.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_parsejson.nim.c -o c_code/3_2/stdlib_parsejson.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/stdlib_json.nim.c -o c_code/3_2/stdlib_json.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mextccomp.nim.c -o c_code/3_2/@mextccomp.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/@mwordrecg.nim.c -o c_code/1_2/@mwordrecg.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mnimblecmd.nim.c -o c_code/3_2/@mnimblecmd.nim.o
-    $CC $COMP_FLAGS -Ic_code -c c_code/2_4/stdlib_parseopt.nim.c -o c_code/2_4/stdlib_parseopt.nim.o
+    $CC $COMP_FLAGS -Ic_code -c c_code/2_2/stdlib_parseopt.nim.c -o c_code/2_2/stdlib_parseopt.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/1_2/@mincremental.nim.c -o c_code/1_2/@mincremental.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/3_2/@mcommands.nim.c -o c_code/3_2/@mcommands.nim.o
     $CC $COMP_FLAGS -Ic_code -c c_code/2_2/@mllstream.nim.c -o c_code/2_2/@mllstream.nim.o

Of these, stdlib_system.nim.c differ for 64-bit versions but not for 32-bit versions of OS (for a given platform). So I used i386 for Mac as a template and replaced arch-specific files with those for ppc.

(Apparently organization of files into folders was changed between older csources packed within Nim 1.6.6 distro and these ones and few extra files been added, so my patch will not work with the former. I am using the latest-commit version of csources_v1 with my patches on top.)

Then, Clang should not be used for any PPC on Mac, at least until someone has time and motivation to fix it eventually. So Clang, if hardcoded for w/e reason, should not be used for PPC. GCC works.
st_mtim patch borrowed from here: marlam/mpop#9 (comment)

At this point, bootstrapping from csources_v1 appears to work fine, while subsequent compilation of Nim source fails on static asserts here: nim-lang/Nim#19914 (comment) (notice the same issue occurred on other Unix).
That could be some bug in the code (I assume a less likely alternative), Rosetta-specific problem (Rosetta gets tricky to fix sometimes), or I still need to fix something in the source that was missed (this is not relevant to this PR).

barracuda156 · 2022-09-21T08:41:33Z

@Araq There are few other changes needed to replicate the process on 10.6.8 Rosetta and use a newer GCC which are installed in Macports prefix. That we can handle locally in Macports, since I do not know how to fix it in a generally correct way.

This should only be used on Rosetta (it is a hack, since build system [mis]detects host arch as x86):

--- a/build.sh	2022-05-05 07:55:19.000000000 +0700
+++ b/build.sh	2022-09-16 00:34:38.000000000 +0700
@@ -127,7 +127,7 @@
     CC="clang"
     LINK_FLAGS="$LINK_FLAGS -ldl -lm"
     if [ "$HOSTTYPE" = "x86_64" ] ; then
-      ucpu="amd64"
+      ucpu="ppc"
     fi
     ;;
   *aix* )

--- a/makefile	2022-05-05 07:55:19.000000000 +0700
+++ b/makefile	2022-09-16 23:26:53.000000000 +0700
@@ -47,7 +47,7 @@
   CC = clang
   LDFLAGS += -ldl -lm
   ifeq ($(HOSTTYPE),x86_64)
-    ucpu = amd64
+    ucpu = ppc
   endif
 endif
 ifeq ($(uos),aix)

--- a/tools/niminst/buildsh.nimf	2022-05-04 23:15:31.000000000 +0700
+++ b/tools/niminst/buildsh.nimf	2022-09-17 20:29:34.000000000 +0700
@@ -128,7 +128,7 @@
     CC="clang"
     LINK_FLAGS="$LINK_FLAGS -ldl -lm"
     if [ "$HOSTTYPE" = "x86_64" ] ; then
-      ucpu="amd64"
+      ucpu="ppc"
     fi
     ;;
   *aix* )

--- a/tools/niminst/makefile.nimf	2022-05-04 23:15:31.000000000 +0700
+++ b/tools/niminst/makefile.nimf	2022-09-17 20:30:49.000000000 +0700
@@ -50,7 +50,7 @@
   CC = clang
   LDFLAGS += -ldl -lm
   ifeq ($(HOSTTYPE),x86_64)
-    ucpu = amd64
+    ucpu = ppc
   endif
 endif
 ifeq ($(uos),aix)

And this must be patched to avoid invoking gcc-4.2 which does not accept fmax-errors=:

--- a/config/nim.cfg	2022-05-04 23:15:31.000000000 +0700
+++ b/config/nim.cfg	2022-09-19 23:03:47.000000000 +0700
@@ -9,6 +9,8 @@
 #  gcc.path %= "$CC_PATH"
 
 cc = gcc
+gcc.exe %= "@gcc_v@"
+gcc.path %= "@prefix@"
 
 # additional options always passed to the compiler:
 --parallel_build: "0" # 0 to auto-detect number of processors

In all patches from both PRs gcc changed to @GCC@, then reinplace used in portfile to set correct paths to /opt/local/bin/gcc-mp-12 (or whichever chosen version).
Something like:

    post-patch {
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/build.sh
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/makefile
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/tools/niminst/buildsh.nimf
        reinplace "s|@GCC@|${configure.cc}|g" ${worksrcpath}/tools/niminst/makefile.nimf
        
        if {[regexp {macports-gcc-(.*)} ${configure.compiler} -> gcc_v]} {
            reinplace "s|@gcc_v@|gcc-mp-${gcc_v}|g" ${worksrcpath}/config/nim.cfg
            reinplace "s|@prefix@|${prefix}/bin|g" ${worksrcpath}/config/nim.cfg
        } else {
            reinplace "s|@gcc_v@|gcc|g" ${worksrcpath}/config/nim.cfg
            reinplace "s|@prefix@|usr/bin|g" ${worksrcpath}/config/nim.cfg
        }
    }

Araq · 2022-09-21T17:24:41Z

Uh, but you're supposed to use the "niminst" tool and adapt the configuration in compiler/installer.ini :-)

Varriount · 2022-09-21T18:18:23Z

@Araq Is that documented anywhere? And if it isn't, @barracuda156, could you add documentation (wherever is sensible) to modify niminst instead?

barracuda156 · 2022-09-23T17:46:38Z

Uh, but you're supposed to use the "niminst" tool and adapt the configuration in compiler/installer.ini :-)

@Araq Thank you, I will check that.

(Please allow me a few days, I am in a mid of a travel until the end of month – and I also get back to my PPC machines by then.)

barracuda156 · 2022-10-10T23:59:54Z

Uh, but you're supposed to use the "niminst" tool and adapt the configuration in compiler/installer.ini :-)

@Araq Do I get it right that to use niminst one has to have working Nim to begin with? :)

I do not have it yet, I need to build it first (and there is an assert error on PPC after csources are built and koch initialized).

Araq · 2022-10-11T17:47:57Z

You only need a working Nim on some machine, it doesn't have to be the PPC thing.

barracuda156 · 2022-10-19T10:20:57Z

You only need a working Nim on some machine, it doesn't have to be the PPC thing.

@Araq Ok, I will try it.

barracuda156 added 5 commits September 15, 2022 22:21

Fix build.sh for Darwin ppc

0131114

Implement Darwin ppc32 support

3da3401

stdlib_os.nim: add st_mtim define for Apple

2a7f1f0

Fix makefile for Darwin ppc32

4de213a

build.sh: use gcc for Darwin PPC

15d1798

barracuda156 mentioned this pull request Sep 17, 2022

Implement support for Darwin PPC nim-lang/Nim#20373

Open

Varriount reviewed Sep 20, 2022

View reviewed changes

Varriount approved these changes Sep 20, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement support for Darwin ppc, fix Darwin ppc64 #7

Implement support for Darwin ppc, fix Darwin ppc64 #7

barracuda156 commented Sep 17, 2022

barracuda156 commented Sep 17, 2022

barracuda156 commented Sep 17, 2022

Varriount Sep 20, 2022

barracuda156 Sep 20, 2022

Araq commented Sep 21, 2022

barracuda156 commented Sep 21, 2022 •

edited

Loading

barracuda156 commented Sep 21, 2022

Araq commented Sep 21, 2022

Varriount commented Sep 21, 2022

barracuda156 commented Sep 23, 2022

barracuda156 commented Oct 10, 2022

Araq commented Oct 11, 2022

barracuda156 commented Oct 19, 2022

Implement support for Darwin ppc, fix Darwin ppc64 #7

Are you sure you want to change the base?

Implement support for Darwin ppc, fix Darwin ppc64 #7

Conversation

barracuda156 commented Sep 17, 2022

barracuda156 commented Sep 17, 2022

barracuda156 commented Sep 17, 2022

Varriount Sep 20, 2022

Choose a reason for hiding this comment

barracuda156 Sep 20, 2022

Choose a reason for hiding this comment

Araq commented Sep 21, 2022

barracuda156 commented Sep 21, 2022 • edited Loading

barracuda156 commented Sep 21, 2022

Araq commented Sep 21, 2022

Varriount commented Sep 21, 2022

barracuda156 commented Sep 23, 2022

barracuda156 commented Oct 10, 2022

Araq commented Oct 11, 2022

barracuda156 commented Oct 19, 2022

barracuda156 commented Sep 21, 2022 •

edited

Loading