Link time optimization

trass3r avatartrass3r created an issue

I just tried out -flto but it doesn't work at all.

In my real-world code it ends in an ICE in one of the files:

foo.d:1:0: internal compiler error: in write_symbol, at lto-streamer-out.c:2457

A simple hello world results in:

/usr/bin/ld: _tlsstart: TLS reference in /opt/gdc/lib/gcc/x86_64-unknown-linux-gnu/4.6.1/../../../../lib64/libgphobos2.a(thread.o) mismatches non-TLS definition in /tmp/ccwbRKwh.o.ironly section .text
/opt/gdc/lib/gcc/x86_64-unknown-linux-gnu/4.6.1/../../../../lib64/libgphobos2.a: could not read symbols: Bad value

Comments (12)

  1. trass3r

    The ICE also happens when trying to compile phobos2 with -flto. btw, is configure in d/phobos2 necessary? I modified configure.in for LTO and a new configure was generated in libphobos.

  2. Iain Buclaw

    Yes configure is necessary (not everyone has autoconf/automake installed on their workstation - and you don't *require* them to build gdc). Will try building with -flto and see where I get - though forewarning, LTO is not wondrously supported by many of the GCC frontends (I think just C/C++).

  3. trass3r

    I think I read the Ada frontend doesn't support it, don't know about other non-C ones.

    Note that I use 4.6.1 though, reportedly LTO is crap in 4.5.

  4. trass3r

    btw, libphobos fails with the following:

    gdc/gcc-4.6.1/objdir/./gcc/gdc -B/home/trass3r/coding/gdc/gcc-4.6.1/objdir/./gcc/ -B/opt/gdc/x86_64-unknown-linux-gnu/bin/ -B/opt/gdc/x86_64-unknown-linux-gnu/lib/ -isystem /opt/gdc/x86_64-unknown-linux-gnu/include -isystem /opt/gdc/x86_64-unknown-linux-gnu/sys-include -m32 -o rt/fpmath.o -Wall -g -frelease -O2 -flto -fversion=GC_Use_Alloc_MMap -fversion=GC_Use_Stack_GLibC -fversion=GC_Use_Data_Fixed -nostdinc -pipe -fdeprecated -I ../../../../libphobos -I ./x86_64-unknown-linux-gnu -m32 -c ../../../../libphobos/rt/fpmath.d
    ../../../../libphobos/rt/fpmath.d:1:0: internal compiler error: in write_symbol, at lto-streamer-out.c:2457
    
  5. trass3r
    • changed status to open

    Well phobos compiles with -flto now, but when I try to compile a hello world afterwards it fails. btw, the path shown (coding/gdc/gcc-4.6.1...) is the directory where I built gdc and /opt/gdc is the install dir.

    $ /opt/gdc/bin/gdc -flto test.d 
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:91:0: warning: type of '_D13TypeInfo_Enum7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:333:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:322:0: warning: type of '_D9Exception7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1327:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:304:0: warning: type of '_D6object9Throwable9TraceInfo11__InterfaceZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1227:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:301:0: warning: type of '_D6object9Throwable7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1224:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:112:0: warning: type of '_D25TypeInfo_AssociativeArray7__ClassZ' does not match original declaration [enabled by default]
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:112:0: warning: type of '_D25TypeInfo_AssociativeArray7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:589:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:32:0: warning: type of '_D6Object7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:54:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:101:0: warning: type of '_D14TypeInfo_Array7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:388:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:106:0: warning: type of '_D20TypeInfo_StaticArray7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:472:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:165:0: warning: type of '_D15TypeInfo_Struct7__ClassZ' does not match original declaration [enabled by default]
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:165:0: warning: type of '_D15TypeInfo_Struct7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:899:0: note: previously declared here
    lto1: warning: type of '_Dmodule_ref' does not match original declaration [enabled by default]
    ......
    lto1: warning: type of '_Dmodule_ref' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1777:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:329:0: warning: type of '_D6object5Error7__ClassZ' does not match original declaration [enabled by default]
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:329:0: warning: type of '_D6object5Error7__ClassZ' does not match original declaration [enabled by default]
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:329:0: warning: type of '_D6object5Error7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1369:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:40:0: warning: type of '_D6Object7Monitor11__InterfaceZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:111:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:205:0: warning: type of '_D15TypeInfo_Shared7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1158:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:200:0: warning: type of '_D18TypeInfo_Invariant7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1150:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:84:0: warning: type of '_D16TypeInfo_Typedef7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:299:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:195:0: warning: type of '_D14TypeInfo_Const7__ClassZ' does not match original declaration [enabled by default]
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:195:0: warning: type of '_D14TypeInfo_Const7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:1107:0: note: previously declared here
    /home/trass3r/coding/gdc/gcc-4.6.1/libphobos/object.di:67:0: warning: type of '_D8TypeInfo7__ClassZ' does not match original declaration [enabled by default]
    ../../../libphobos/object_.d:214:0: note: previously declared here
    In file included from <built-in>:51:0,
                     from :204:
    ../../../libphobos/core/thread.d: In function 'core.thread.thread_suspendHandler':
    ../../../libphobos/core/thread.d:622:0: error: bp cannot be used in asm here
    lto-wrapper: /opt/gdc/bin/gdc returned 1 exit status
    /usr/bin/ld: lto-wrapper failed
    collect2: ld returned 1 exit status
    
  6. trass3r

    I made another test by compiling gtkD with -flto.

    $ gdmd -flto demos/gtkD/TestWindow/TestImage.o demos/gtkD/TestWindow/TestScales.o demos/gtkD/TestWindow/TestWindow.o demos/gtkD/TestWindow/TestTreeView1.o demos/gtkD/TestWindow/app.o demos/gtkD/TestWindow/TestEntries.o demos/gtkD/TestWindow/TestTreeView.o demos/gtkD/TestWindow/TTextView.o demos/gtkD/TestWindow/TestStock.o demos/gtkD/TestWindow/TestAspectFrame.o demos/gtkD/TestWindow/TestThemes.o demos/gtkD/TestWindow/TEditableCells.o demos/gtkD/TestWindow/TestDrawingArea.o demos/gtkD/TestWindow/TestIdle.o demos/gtkD/TestWindow/TestText.o  -ofTestWindow -L-ldl -m64 -L-L. -L-lgtkd
    /opt/gdc/lib/gcc/x86_64-unknown-linux-gnu/4.6.1/../../../../lib64/libgphobos2.a: could not read symbols: Bad value
    

    Not sure why it complains about libphobos though. It was built without -flto this time.

  7. trass3r

    Ok I tested cl4d (with -m32 due to #254) and LTO finally worked for some code, yay :)

    There are some things to be noted though. The library contains a set of structs that provide automatic reference counting. So e.g. the destructor merely contains a release() call which in turn calls some extern(C) release() and does some error handling with exceptions.

    • Without the exceptions gdc manages to inline everything and the destructor calls are directly turned into the C release calls at the correct places in _Dmain. Yet some destructor functions (with inlined D release call) still exist and are referenced in some obscure code section.
    • With these exceptions release() seems to be too big to be inlined, but the destructor isn't, so it's inlined. Some destructors again exist as described above.
    • Then there's another destructor whose release() method is empty. The codegen resulted in 'push ebp; mov ebp, esp; pop ebp', so it does nothing and doesn't seem to be referenced anywhere either, yet it's in the executable.
  8. trass3r

    Edit: Never mind. This is the correct code and it works fine. With LTO, everything is inlined into main, then it even inlines the write call, fantastic! Nothing is left of this nor release().

    import std.stdio;
    
    struct F
    {
    	~this()
    	{
    		release();
    	}
    
    	void release()
    	{
    		write("release()\n");
    	}
    }
    
    void main()
    {
    	F f;
    }
    
  9. trass3r

    Biggest issue is still the phobos one though. Tried again to compile a Hello World with -flto (phobos been built with -flto too):

    Still all those warnings about not matching original declaration and now it ends with

    lto1: internal compiler error: in ipcp_iterate_stage, at ipa-cp.c:891
    
  10. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.