[klibc] Klibc 2.0.2 build vs -j 30-ish

[Maciej Żenczykowski]

I've run into a problem with building klibc with high parallelism
It's not quite predictable, but at a -j of approx 25-30 or more
(on a 24 hyperthreaded machine), I often get something like
the following error messages:

...
 KLIBCCC usr/klibc/sigsuspend.o
 KLIBCCC usr/klibc/sigprocmask.o
 KLIBCCC usr/klibc/pselect.o
 KLIBCCC usr/klibc/ppoll.o
/git/KLIBC/usr/klibc/sigsuspend.c:8:31: fatal error:
klibc/havesyscall.h: No such file or directory
compilation terminated.
 KLIBCCC usr/klibc/sbrk.o
 KLIBCCC usr/klibc/malloc.o
make[2]: *** [usr/klibc/sigsuspend.o] Error 1
make[2]: *** Waiting for unfinished jobs....
 KLIBCCC usr/klibc/brk.o
...
 KLIBCAS usr/klibc/syscalls/getsockopt.o
 LIST    usr/klibc/syscalls/klib.list
make[1]: *** [all] Error 2
make: *** [klibc] Error 2


The problem stems from within usr/klibc:

sigsuspend.c includes <klibc/havesyscall.h> which is effectively
$(objtree)/$(KLIBCINC)/klibc/havesyscall.h, which comes about
as a side effect of building $(objtree)/$(obj)/syscalls/syscalls.mk,
this happens relatively early while recursively building the 'syscalls'
subdirectory (of usr/klibc).

However, we have no dependency on this.

With "make -j 1" klib-y ordering makes it work (syscalls/ before sigsuspend.o).

It's easy to break even a -j 1 make, by (within usr/klibc/Kbuild) adding:

klib-y += sigsuspend.o

before

# Generate syscall stubs
klib-y += syscalls/

and removing it (sigsuspend.o) from the later long list.

With a sufficiently large -j (and a fast enough/parallel enough machine)
this reordering change is not needed, since it can happen naturally:
we can start compiling sigsuspend.o before the first parallel make
(which is recursing into the [usr/klibc/]syscalls/ subdirectory) finishes
doing the half dozen things it has to do before the havesyscall.h file is ready.

The error might be about the file missing, or rarer about mismatched #ifdefs
(file was read to EOF by compiler before it was fully written out)

A potential solution involves adding one line to usr/klibc/Kbuild:

$(obj)/sigsuspend.o: $(obj)/syscalls

This appears to resolve the problem.

There is unfortunately no way I can find to be more specific than relying
on the recursive make for the entire subdirectory to have already completed
before we attempt to compile sigsuspend.o.

Is this a reasonable fix?  Or is there something better?

However I am unsure if this is the resolution.

- Maciej

----

diff --git a/usr/klibc/Kbuild b/usr/klibc/Kbuild
--- a/usr/klibc/Kbuild
+++ b/usr/klibc/Kbuild
@@ -72,6 +72,15 @@ klib-y += vsnprintf.o snprintf.o vsprintf.o sprintf.o \
          stdio/fseek.o stdio/ftell.o stdio/rewind.o \
          stdio/fileno.o stdio/feof.o stdio/ferror.o

+# sigsuspend.c includes <klibc/havesyscall.h> which is effectively
+# $(objtree)/$(KLIBCINC)/klibc/havesyscall.h, which comes about
+# as a side effect of building $(objtree)/$(obj)/syscalls/syscalls.mk,
+# this happens relatively early while recursively building the 'syscalls'
+# subdirectory.  There is unfortunately no way I can find to be more
+# specific than relying on the recursive make for the entire subdirectory
+# to have already completed before we attempt to compile sigsuspend.o.
+$(obj)/sigsuspend.o: $(obj)/syscalls
+
 klib-$(CONFIG_KLIBC_ERRLIST) += errlist.o

 ifeq ($(CONFIG_KLIBC_ERRLIST),y)