Build fails with massively unhelpful guile vomit

[Craig Lennox]

A clue about what's going on in this incoherent mess would be outstanding.

Making all in guile
GUILE_AUTO_COMPILE=0 GUILE_LOAD_PATH='/Users/blipvert/devel/sortsmill-tools/guile:/Users/blipvert/devel/sortsmill-tools/guile:/opt/local/share/guile/2.0:/opt/local/share/guile/site/2.0:/opt/local/share/guile/site:/opt/local/share/guile' GUILE_LOAD_COMPILED_PATH='/Users/blipvert/devel/sortsmill-tools/guile:/opt/local/lib/guile/2.0/ccache:/opt/local/lib/guile/2.0/site-ccache' LTDL_LIBRARY_PATH='/Users/blipvert/devel/sortsmill-tools/guile' /opt/local/bin/guile -L ../guile -L ../guile -s ./generate-guile-api.scm   \
            '(sortsmill fontforge-api)' ../fontforge/fontforge.types.apii > sortsmill/fontforge-api.scm-tmp
Backtrace:
In ice-9/boot-9.scm:
2786: 19 [try-module-autoload (sortsmill alloc) ()]
2131: 18 [save-module-excursion #<procedure 10240a840 at ice-9/boot-9.scm:2787:17 ()>]
2797: 17 [#<procedure 10240a840 at ice-9/boot-9.scm:2787:17 ()>]
In unknown file:
   ?: 16 [primitive-load-path "sortsmill/alloc" #f]
In ice-9/eval.scm:
 494: 15 [#<procedure 1021da400 at ice-9/eval.scm:488:4 (exp)> (library # # # ...)]
In ice-9/psyntax.scm:
1091: 14 [expand-top-sequence ((library # # # ...)) () ((top)) ...]
 974: 13 [scan ((library (sortsmill alloc) (export c:zalloc c:free ...) ...)) () ...]
1198: 12 [syntax-type (# # # # ...) () (()) ...]
1407: 11 [expand-macro #<procedure 1022fc540 at ice-9/r6rs-libraries.scm:126:2 (stx)> ...]
In ice-9/r6rs-libraries.scm:
 180: 10 [#<procedure 1022fc600 (name name* version espec ispec body)> # # () ...]
In ice-9/boot-9.scm:
 623: 9 [map #<procedure 1022fca60 at ice-9/r6rs-libraries.scm:181:19 (im)> #]
2594: 8 [resolve-interface (sortsmill dynlink) #:select ...]
2519: 7 [#<procedure 1022ee240 at ice-9/boot-9.scm:2507:4 (name #:optional autoload version #:key ensure)> # ...]
2786: 6 [try-module-autoload (sortsmill dynlink) ()]
2131: 5 [save-module-excursion #<procedure 1027d30c0 at ice-9/boot-9.scm:2787:17 ()>]
2797: 4 [#<procedure 1027d30c0 at ice-9/boot-9.scm:2787:17 ()>]
In unknown file:
   ?: 3 [primitive-load-path "sortsmill/dynlink" #f]
In ice-9/eval.scm:
 421: 2 [eval # ()]
 442: 1 [eval # ()]
In unknown file:
   ?: 0 [dynamic-link "libguile-sortsmill_symbols"]

ERROR: In procedure dynamic-link:
ERROR: In procedure dynamic-link: file: "libguile-sortsmill_symbols", message: "file not found"
make[1]: *** [sortsmill/fontforge-api.scm] Error 1
make: *** [all-recursive] Error 1

Built from 5046d472ef3a2bd59d3162aaf0142611d716dfc6.

[Barry Schwartz]

It’s called a backtrace. :) It’s actually complaining about some C code. I think there may still be Makefile problems that I haven’t treated appropriately yet; my scheme for ensuring dynamic linkage still creates as many problems as it solves. It’s a ‘make depend’ situation, where I’m still doing it manually instead while I think about it.

I will study ‘make -C guile view-guile-modules’ to see if there are necessary dependencies left out.

[Barry Schwartz]

If you want, you can try setting ‘GENERATED_GO_DEPS = anything-you-like’ in guile/Makefile.am and see if that lets you build. It’s a dependency tracking mechanism that doesn’t work quite right in all ways.

[Barry Schwartz]

Maybe I should just make the developer-generated libguile_sortsmill_symbols.c part of the distribution, rather than leaving it purely generated on the fly by the user -- at least for now.

No, that won’t work unless I split it into pieces depending on configuration options. :/

[Craig Lennox]

Okay, I tried your GENERATED_GO_DEPS suggestion and got this instead.

Making all in guile
Makefile:2852: sortsmill/fontforge-api.Dscm: No such file or directory
Makefile:2852: sortsmill/gdraw-api.Dscm: No such file or directory
GUILE_AUTO_COMPILE=0 GUILE_LOAD_PATH='/Users/blipvert/devel/sortsmill-tools/guile:/Users/blipvert/devel/sortsmill-tools/guile:/opt/local/share/guile/2.0:/opt/local/share/guile/site/2.0:/opt/local/share/guile/site:/opt/local/share/guile' GUILE_LOAD_COMPILED_PATH='/Users/blipvert/devel/sortsmill-tools/guile:/opt/local/lib/guile/2.0/ccache:/opt/local/lib/guile/2.0/site-ccache' LTDL_LIBRARY_PATH='/Users/blipvert/devel/sortsmill-tools/guile' /opt/local/bin/guile-tools compile -Wunbound-variable -Warity-mismatch -Wduplicate-case-datum -Wbad-case-datum -Wformat -L ../guile -L ../guile sortsmill/dynlink.scm -o sortsmill/dynlink.go
Backtrace:
In srfi/srfi-1.scm:
 619: 19 [for-each #<procedure 108c16340 at scripts/compile.scm:179:14 (file)> #]
In scripts/compile.scm:
 182: 18 [#<procedure 108c16340 at scripts/compile.scm:179:14 (file)> "sortsmill/dynlink.scm"]
In system/base/target.scm:
  59: 17 [with-target "x86_64-apple-darwin12.2.0" ...]
In system/base/compile.scm:
 147: 16 [compile-file "sortsmill/dynlink.scm" #:output-file ...]
  43: 15 [call-once #<procedure 108ef03c0 at system/base/compile.scm:56:5 ()>]
In ice-9/boot-9.scm:
 171: 14 [with-throw-handler #t ...]
In system/base/compile.scm:
  59: 13 [#<procedure 108ef0380 at system/base/compile.scm:58:9 ()>]
 150: 12 [#<procedure 108ef0400 at system/base/compile.scm:148:8 (port)> #<input-output: sortsmill/dynlink.go.MZ3DAO 6>]
 199: 11 [read-and-compile #<input: sortsmill/dynlink.scm 5> #:from ...]
 211: 10 [lp () #f #<module (#{ g42}#) 108f7df30>]
 177: 9 [lp (#<procedure compile-tree-il (x e opts)>) (library # # ...) ...]
In ice-9/boot-9.scm:
2131: 8 [save-module-excursion #<procedure 1091a2540 at language/scheme/compile-tree-il.scm:29:3 ()>]
In language/scheme/compile-tree-il.scm:
  31: 7 [#<procedure 1091a2540 at language/scheme/compile-tree-il.scm:29:3 ()>]
In ice-9/psyntax.scm:
1091: 6 [expand-top-sequence ((library # # # ...)) () ((top)) ...]
 976: 5 [scan ((library # # # ...)) () ((top)) ...]
 976: 4 [scan ((# # # #) (# # # #) (# # # #) (# # # #) ...) () (()) ...]
 270: 3 [scan ((define sortsmill-dynlink-dll #)) () ((top)) ...]
In ice-9/eval.scm:
 442: 2 [eval # ()]
In unknown file:
   ?: 1 [dynamic-link "libguile-sortsmill_symbols"]
In ice-9/boot-9.scm:
 106: 0 [#<procedure 108ef0340 at ice-9/boot-9.scm:97:6 (thrown-k . args)> misc-error ...]

ice-9/boot-9.scm:106:20: In procedure #<procedure 108ef0340 at ice-9/boot-9.scm:97:6 (thrown-k . args)>:
ice-9/boot-9.scm:106:20: In procedure dynamic-link: file: "libguile-sortsmill_symbols", message: "file not found"
make[1]: *** [sortsmill/dynlink.go] Error 1
make: *** [all-recursive] Error 1

They're like snowflakes; each one slightly different but beautiful in its own way.

[Barry Schwartz]

Isn’t it nice the way you get messages about Guile internals so you have to be one of the developers to know what they mean?

Sometimes stuff like this also means I forgot to mark a symbol visible in the C code. I think I may have to do the fifteenth overhaul of how the build is set up. I’m sure if I were more experienced in Autotools I’d have made this work long ago, but I’m still on the learning curve.

[Barry Schwartz]

But, anyway, the problem here almost surely is not actually in Guile as such. It’s complaining it can’t find C code to link with.

[Barry Schwartz]

Just to be double sure you could also try exporting LTDL_LIBRARY_PATH='/Users/blipvert/devel/sortsmill-tools/guile' in your environment, though the messages above imply this is getting set as necessary.

[Craig Lennox]

Same result, unfortunately.

[Barry Schwartz]

It would be useful to see the surrounding context to see if build order for some reason is different on your system. Yours is the second or third report like this but I cannot, for the life of me, reproduce it. BTW I’m using GNU Make 3.82.

[Craig Lennox]

I'm using Make 3.81 !!!

I will upgrade my make and get back to you.

[Barry Schwartz]

I think it may simply be that I’ve been very lucky about the build order GNU make chooses, but the dependency rules are actually woefully incomplete. It’s a problem whenever you have dependencies the build system doesn’t know about already! (That’s part of what I hate about CMake; you practically have to write MIXAL code if you use something other than C or C++.)

[Barry Schwartz]

SCons is better about this stuff than Make is, but of course, because it is writing Python code instead of Makefiles, it has other, very severe problems. It makes difficult things very easy, but makes practically every easy thing too difficult for me to comprehend.

[Barry Schwartz]

You might try it now. I set up a manual ‘make depend’ to be run by me with the results being part of the distribution, as at least a temporary measure. It’s something I was trying to avoid.

[Craig Lennox]

Hate to say it but it still doesn't work. Here's a complete make V=1 trace. Hopefully you can glean some clues from it.

[Barry Schwartz]

The other people resolved the problem, and so theirs was different. Also AFAICT your libguile-sortsmill_symbols.la does get created. Now I suspect this is a Mac-specific issue related to libguile-sortsmill_symbols not working as a dynamically loadable module. If that’s the case, it is your problem to solve. :)

You could try adding the directory with libguile-sortsmill_symbols.la in it to LTDL_LIBRARY_PATH, start up guile, and try entering (dynamic-link "libguile-sortsmill_symbols") which should result in a message like

$some-number = #<dynamic-object "libguile-sortsmill_symbols">

and see if that works.

[Barry Schwartz]

Otherwise maybe the libtool options are wrong when the module gets linked, or something like that, which can be hard to detect on Linux because of the near equivalence of ordinary shared libraries and dlopenable ones. (The one difference I know of is you can have a GNU ld script as a shared library but not as a dlopenable one.)

(Which incidentally is why George’s zany dlopen code eventually quit working correctly even on a straight-laced GNU system like Gentoo. :) )

[Craig Lennox]

Now I suspect this is a Mac-specific issue related to libguile-sortsmill_symbols not working as a dynamically loadable module. If that’s the case, it is your problem solve. :)

It did work before, though. The problem did not exist for me in a2e607e53ca507a35a67eb902853fc42ed40c903.

I will say that Guile is probably my least favorite Scheme implementation. It is brittle as hell and does not fail gracefully or with helpful diagnostics. The otherwise excellent gEDA suite uses it and it is just a constant source of headache there.

[Craig Lennox]

Okay, I finally made it work, although I am not entirely happy with how. In a fit of desperation I just typed make -k to see how much of the build I could get through. I then typed make -k install, and was surprised to see that it was successful.

If I then do

export GUILE_LOAD_PATH=$GUILE_LOAD_PATH:/opt/sortsmill-tools/share/guile/site/2.0
export GUILE_LOAD_COMPILED_PATH=$GUILE_LOAD_COMPILED_PATH:/opt/sortsmill-tools/lib/guile/2.0/site-ccache
export DYLD_LIBRARY_PATH=/opt/sortsmill-tools/lib

Then I am finally able to run /opt/sortsmill-tools/bin/smed and have it come up.

The fact that it was necessary to use -k suggests that you have a bootstrapping issue which is invisible to you because you probably always have an existing installation.

[Barry Schwartz]

I run make uninstall then run distcheck all the time now, and sometimes clean out the installation directories by hand. I also sometimes build from a fresh repository and sometimes on Ubuntu 32-bit in a virtual machine. It has to be related to what is in the build directories. It could be junk left behind before, I suppose. It is a good idea to clean out junk now and then, because I change things.

I have modified the libguile-sortsmill_symbols to go to comic lengths to make the toolchain treat it as useful code rather than what it is -- a trick to fool GNU ld with the oddball default settings that Debian and Ubuntu give it.

[Barry Schwartz]

If you are building in the source directories instead of a VPATH build, then some old junk might have been overwritten by generated code when you ran make -k.

I’m speculating. But sometimes distributed code gets replaced by generated code and vice versa, and if you build in the source directories then they would have the same path (but Khaled and I both build in a ./build subdirectory, though I do in-source builds for testing).