waf cannot find ifort when building under Windows 7

Issue #4 resolved
Bob Apthorpe
created an issue

Hi,

I tried building Aotus using haraldkl-aotus-8a30017cd4e9 with Intel Composer XE 2013 SP1 on Windows 7. waf could not find ifort.exe before or after the Intel compiler environment variables have been set. It appears that the correct paths were searched but ifort.exe was not found; it's not clear how to manually set the path for the Fortran compiler.

The specific commands used were:

python .\waf configure

and

python ..\waf --check-fortran-compiler=ifort configure build

The explicit invocation of python is due to Windows' inability to invoke interpreters via the #! line at the top of the script. waf seems to run fine when called this way.

The Intel compiler needs environment variables set using the ifortvars.bat script; running 'ifortvars.bat ia32' sets ifort to build for x86 platforms.

The attached zip file contains the python version, path and environment variables before and after invoking 'ifortvars ia32', and waf output to stderr, stdout, and to the build/config.log file for both attempts to configure & build Aotus.

I understand I'm pretty far outside your normal gcc/gfortran/linux environment so if there's any additional information you need, please let me know. I'm caught between implementing my own DSL in Fortran (a very, very bad idea) and embedding a sane, supported language in our code under Windows (and for my benefit, Linux) so I have some pretty strong motivation to help you debug this. I just don't know enough about waf to understand how it is searching for ifort. -- Viele Danken!

Comments (42)

  1. Harald Klimach repo owner

    Thanks for reporting this. There should be a recent version of waf in Aotus, I now pulled the changes from apesteam/aotus (https://bitbucket.org/apesteam/aotus) into here, which includes an update in waf. Maybe this would help? I will try to look deeper into this, but do not have a Windows easily available to me.

    Can you set environment variables in Windows? Try setting the environment variable FC to the correct path of the ifort executable.

  2. Bob Apthorpe reporter

    I'm heading to work in a few minutes and I'll grab the update and test it. I'll try it with and without explicitly setting FC and will send results in a few hours.

    As always, thanks for your quick response :)

  3. Bob Apthorpe reporter

    I spent a few hours getting my head around waf, constructed a minimal example of the bug, and opened a ticket with the waf developers: https://github.com/waf-project/waf/issues/1672 The problem seems to exist in their most recent version (1.8.17) so you may want to watch that ticket and update waf once they resolve the problem. Then I'll keep working on getting aotus to build on Windows & help with any extra documentation, etc. to simplify the process for others.

  4. Harald Klimach repo owner

    I now had a look into your logs. The issue is not with ifort, but with xiar, which is not found. The xiar is only required if you do interprocedural optimization, otherwise any archiver should be fine. Please try to set the AR variable to the available archiver.

  5. Bob Apthorpe reporter

    I found xiar under "c:\Program Files (x86)\Intel/Composer XE 2013 SP1/bin/intel64_mic/xiar.exe" but I'm building under ia32 (x86) so this is probably the wrong archiver. From https://software.intel.com/en-us/node/525274 it looks like I need to set AR to "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\lib.exe" or equivalent. I'll let you know how that works.

  6. Harald Klimach repo owner

    I'm sorry, I didn't realize the waf issue any earlier. I ran into trouble with the compiler detection myself, this might also play a role. It should be fixed now, and I pushed an updated waf into aotus. Please give fbcdadc a try. We also should change the lookup for the archiver in waf itself, but it is a little hard for me to figure this out, as I do not have an ifort available on Windows. So please let me know how it should look like.

  7. Bob Apthorpe reporter

    So here's the status of testing fbcdadc so far:

    • gfortran + gcc + Linux: works
    • gfortran + gcc + Windows: works (using TDM gcc/gfortran from http://tdm-gcc.tdragon.net/download)
    • ifort + gcc + Linux: works
    • ifort + gcc + Windows: fails
    • ifort + msvc + Windows: fails

    For Windows + ifort, I set FC=ifort.exe.

    For msvc, I've tried AR=lib.exe and AR=ifort.exe; neither work. msvc fails during waf configure; it can't find the expected functions and libraries (mkstemp, popen, srandom, m)

    For gcc, I set AR=ar.exe. gcc fails to find srandom but I don't think that's fatal; I believe the same problem crops up with gfortran/gcc on Windows. waf --check-c-compiler=gcc --check-fortran-compiler=ifort configure completes without error but waf --check-c-compiler=gcc --check-fortran-compiler=ifort build throws a number of messages like

    [73/81] Linking build\aotus_sample.exe
    Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 16.0.1.146 Build 20151021
    Copyright (C) 1985-2015 Intel Corporation.  All rights reserved.
    ifort: command line warning #10006: ignoring unknown option '/xHOST'
    ifort: command line warning #10006: ignoring unknown option '/ipo'
    ifort: command line warning #10006: ignoring unknown option '/no-prec-div'
    
    ifort: NOTE: The evaluation period for this product ends on 27-jan-2016 UTC.
    ifort: warning #10145: no action performed for file 'all'
    

    and

    Microsoft (R) Incremental Linker Version 14.00.23506.0                                                            
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    -out:C:\Users\apthorpe\Documents\Projects\aotus\wintel\haraldkl-aotus-fbcdadc36b5b\build\aotus_sample.exe         
    -subsystem:console                                                                                                
    all                                                                                                               
    sample\aotus_sample.f90.6.o                                                                                       
    LINK : fatal error LNK1181: cannot open input file 'all.obj'
    

    The ifort + gcc combination seems the closest to working. The 'unknown options' often need to be converted from (say) -xHost to /QxHOST, -ipo to /Qipo, etc. The error for all.obj erros is probably because --warn all needs to be --warn=all on Windows for some reason known only to Intel.

    I've installed an evaluation copy of ifort on my gaming PC at home because it's possible that my work machine doesn't have the Windows SDK installed, or it's not installed in the right location, or something (ifort builds code fine from within Visual Studio.) This is incredibly frustrating because it seems like Intel has gone out of their way to make ifort nonfunctional on Windows outside of Visual Studio. Setting up the evaluation version at home was a multihour headache so I'm not surprised this configuration hasn't been extensively tested. The whole Windows experience is unnecessarily awful. Apologies for the rant; I just don't understand how with a Microsoft OS on an Intel CPU, the combined MS and Intel dev tools cost a fortune and don't work nearly as well as gfortran/gcc. It makes absolutely no sense to me since ifort works great on Linux.

    Anyway, let me know what information might help or which specific combination of ifort and gcc or msvc you'd like me to test. I think we're close; the project works on 3 of 5 tool combinations tested and I've successfully built aotus_sample manually for each of the working combinations. Provided the Windows ifort issues can be resolved, I'll be able to link Aotus to our code using our existing build process without too much effort. The problem is getting ifort to behave.

  8. Bob Apthorpe reporter

    I posted an archive containing the build results and -vvv log entries for building fbcdadc using TDM gcc and both ifort and gfortran on Windows at http://apthorpe.cynistar.net/code/aotus/fbcd_results_for_win_ifort_and_tdm_gfortran.zip (it's 4MB, too large for uploading here). Hopefully there's something useful in either config.log or the *_out.txt and *_err.txt from stdout & stderr, respectively.

    Using TDM gfortran/gcc works, ifort/gcc does not. I really don't know how to debug this other than manually translate the working build steps from the TDM stdout logs into a batch file that executes the equivalent with ifort instead of gfortran. Again, waf is giving bad options to ifort and it's not clear to me what needs to be changed either in waf or in wscript to fix the problem.

    If I can reverse-engineer a working build process, I'll post it here.

  9. Harald Klimach repo owner

    To me it looks like your failing runs are basically due to wrong compiler options on windows. Due to my definition of the compiler flags to choose. If you look into the wscript you will find these lines:

        # Flags for the default (production) variant
        conf.env['FCFLAGS'] = ( fcopts[conf.env.FC_NAME, 'optimize']
                              + fcopts[conf.env.FC_NAME, 'warn'] )
        conf.env['LINKFLAGS_fcprogram'] = conf.env['FCFLAGS']
    
        # Set flags for the debugging variant
        # DEBUG Variant
        conf.setenv('debug',conf.env)
        conf.env['FCFLAGS'] = ( fcopts[conf.env.FC_NAME, 'standard']
                              + fcopts[conf.env.FC_NAME, 'warn']
                              + fcopts[conf.env.FC_NAME, 'w2e']
                              + fcopts[conf.env.FC_NAME, 'debug'] )
        conf.env['LINKFLAGS_fcprogram'] = conf.env['FCFLAGS']
    

    The fcopts are taken from fc_flags.py, can you try to modify the options there to fit to the windows requirements? I will change this to be dependent on the OS.

    As you mentioned missing mkstemp, srandom and the like should not be a big issue, Lua can be built without those.

  10. Harald Klimach repo owner

    I added options for ifort on windows in 404e607. Please check whether this helps, waf should now automatically use the windows options for ifort when windows is detected.

    Would it help your build setup if I create a Makefile for Aotus to build it against an existing installation of Lua?

  11. Bob Apthorpe reporter

    I was able to build Aotus from scratch on Windows with MSVC and ifort but had some issues with reading Lua tables. Attached is a three part tutorial I wrote on the manual build process which gives detailed instructions and diagnostic steps.

    I believe the remaining problem is either due to bad compiler/linker options or the way I manually compiled Aotus. I had problems with the real extdouble and quadruple routines so I compiled with the dummy routines instead (this is explained in part 3 of the tutorial). I have an 80% solution and I think with a little work, the table issue can be resolved and the build process automated with waf. Diagnosis was somewhat painful but hopefully we only have to do it once :)

  12. Harald Klimach repo owner

    Thanks for your detailed feedback. Some comments:

    • Dig through the logs to find the literal commands used to build the sample application

    There is a module available for waf called print-commands, I will put it into the waf of Aotus to enable easier access to the actual commands.

    • Migrate the literal build commands and dependencies (static libraries, module and object files, etc.) into your existing build process

    There are the waftools, which allow the export of C/C++ projects to other build systems, I would hope Fortran could be added to those, but up to now did not get around looking into it.

    • Python: I used Python 2.7 on both Windows and Linux; I didn't test Python 3.x.

    Python 3 should also work.

    I don't own a machine that runs OSX and would appreciate confirmation that this method works for Mac users too.

    Our working machines are MacBooks, so development of Aotus basically is done on MacOS, building Aotus on them out of the box with a gfortran 5 installed via macports and clang is no issue, and maybe one of the best tested builds besides Linux.

    This doesn't remove the dependency of waf from the Aotus library build process but that's a secondary concern. Since we're linking with a static Aotus library rather than compiling the project source each time we build our code

    There is also an install option in waf, though I never use it, so it is barely tested...

    Regarding MSVC+Ifort: this should be possible with waf itself, could you give it a try and let me know the resulting errors in the logs?

    I could never find a way to manually compile aot_top_module.f90 due to interface signature conflicts with aot_extdouble_top_module.f90.

    As far as I know extdouble is GNU specific and for Ifort the corresponding selected_real_kind results in quadruple or double precision. Thus, you end up with duplicated interfaces for Ifort.

    which is as expected; this shows that Lua tables can be written from Fortran; table support isn't completely broken.

    The output does not make use of Lua at all, it just eases the output of Lua tables via Fortran write statements, so this is unfortunately not a sign of anything working in the Lua API. However, we are looking at a nested table here and the first table seems to be recognized with the correct number of entries (2). So at least this one level seems to work. The code then goes on and tries to open the entries by position, which seems to fail.

    You don't see the table issue with the TDM compilers, right? It might be, that there is some issue with handing over the arguments from Fortran to C, but the calling of a Lua function seems to work properly, which might hint some other issue yet. But I don't have a clue. Can you use Ifort/Icc or Ifort/gcc for a comparison?

    Are you sure the config.lua is not changed? For example if the tables would be defined like

    stl_files = { a={'filename', 123, 'binary'},
                  b={'geomfile', 456, fileformat='ascii'} }
    

    instead of

    stl_files = { {'filename', 123, 'binary'},
                  {'geomfile', 456, fileformat='ascii'} }
    

    the reading in the sample file would fail as described.

  13. Bob Apthorpe reporter

    The tutorial was created over the past two weeks and doesn't reflect your recent changes in 404e607; I will retest at home tonight. The work machine has issues with an old version of ifort and no compatible installation of MSVC. Thanks for the tip on checking stl_files. That reminds me that I need to file a bug report against FortLua's test case; math.tanh was removed from Lua 5.3 so their test case fails quietly. I've learned a lot on this project...

  14. Harald Klimach repo owner

    OK, I did not include the print_commands now, instead, I added a small variant. The print_commands still only logs the commands to the log on debug level and prepends some string. More useful for your purpose would be a pure list of all commands executed along the build process, if I got that correctly. Thus, I created the option --command_sequence that will track all issued commands and put them into a single file. To create that file you should build a clean project with --command_sequence -j1. This is a very crude and limited hack, but it could help at least to create the basis for an integration in other build workflows. With

    ./waf --target=aotus --command_sequence -j1
    

    the executed commands can be limited to those necessary to create the Aotus library. Note, that the generated command_sequence.txt file is not deleted before another build, and all commands will be appended to the existing file.

  15. Bob Apthorpe reporter

    Looking much better; found a few new compiler/linker option issues:

    • /SUBSYSTEM:POSIX is not supported by MSVC (throws warnings)
    • ARFLAGS has rcs set on Windows (in ar, the rcs options tell the archiver to replace files without warning and to generate an index)
    • The parameters to the archiver include ['lib.exe', '', '/OUT:', 'libflu.a',...; I believe that should be ['lib.exe', '', '/OUT:libflu.a',...

    For the second problem, conf.env.ARFLAGS='rcs' is hardcoded in waflib/Tools/ar.py and waflib/Tools/ifort.py; setting ARFLAGS to an empty string or to /nologo doesn't fix this.

    Getting closer!

  16. Harald Klimach repo owner

    Regarding the first point. This flag is set in msvc.py by this line:

            v['CFLAGS_POSIX']     = v['CXXFLAGS_POSIX']     = ['/SUBSYSTEM:POSIX']
    

    The CFLAGS_POSIX are used, in the wscript by:

        bld(
            features = 'c',
            source = core_sources + lib_sources,
            defines = ['LUA_ANSI'],
            use = ['POSIX'],
            target = 'luaobjs')
    

    This is used to identify whether mkstemp, popen and srandom are available:

        if conf.env.DEFINES_POPEN and conf.env.DEFINES_MKSTEMP and conf.env.DEFINES_SRANDOM:
          conf.env.DEFINES_POSIX = ['LUA_USE_POSIX']
    

    I changed this to LUA_POSIX in cd229745a251, now the msvc should not set this CFLAGS anymore (this changeset also includes the above mentioned change in waf, hopefully this resolves the first two points).

  17. Harald Klimach repo owner

    For the third issue I am not quite sure, what is going on. I guess we need to ask Thomas about it. I think, it might be related to the execution mode in waf, it might help to have the linking as a shell task instead of a non-shell task. I attached a waf, which, I believe, has this changed accordingly and might work. I'll open an issue on this in waf.

  18. Harald Klimach repo owner

    There seem to be -ipo issues, can you please try to build the debug variant

    ./waf debug
    

    This will not use the optimization flags. Alternatively remove the /Qipo from fc_flags.py, change

    fcopts['IFORTwin', 'optimize'] = '/QxHost /O3 /Qipo /Qprec-div-'.split()
    

    to

    fcopts['IFORTwin', 'optimize'] = '/QxHost /O3 /Qprec-div-'.split()
    
  19. Bob Apthorpe reporter

    This morning I had a chance to test the new build and I made pretty good progress.

    I believe the /SUBSYSTEM:POSIX issue is settled.

    There is still a lingering ARFLAGS=rcs in waflib\Tools\ar.py which I manually commented out to keep debugging.

    I modified command_sequence.txt so it was runnable and found 2 instances where -Wl,--enable-auto-import /NOLOGO /MANIFEST was still being passed to ifort; I removed those and continued.

    It looks like there's an issue where libflu.a is created where symbols defined by the LuaFortran object code have already been resolved by other members of the library and won't be linked against external code. I believe that is what causes the build of flu_sample to fail. The specific warnings and errors are listed in the cmd.out log in the attached zip file.

    Otherwise, I tested lua.exe and I could get it to print Hello World! so I think things are working in general; there are just a few small issues to address as we find them.

  20. Harald Klimach repo owner

    There is still a lingering ARFLAGS=rcs in waflib\Tools\ar.py which I manually commented out to keep debugging.

    This should not be applied if you set ARFLAGS as an environment variable.

  21. Harald Klimach repo owner

    After some more thinking, I really believe, the remaining obstacle is the /Qipo option, the lib.exe linker is not capable to deal with the objects properly here. If we want to use the interprocedural compilation, we need to use the xiar linker from Intel. This is why there are no symbols in all the Fortran objects from the LuaFortran directory and the linking of the executables fails. Maybe, there are further issues beyond this step, but a successful build is very close now, I think.

  22. Bob Apthorpe reporter

    I can get further compilation if I remove /Qipo and (I believe) adjust -lflu to flu.lib. I'm still hazy about whether ifort respects -l or needs the actual name of the library. I'll send another updated version of command_sequence.txt soon. I was able to get flu_sample working so we're not too far away now.

  23. Harald Klimach repo owner

    Thanks, I changed the linking in 069f889df757 to inlcude /LINK before the /LIBPATHS for linking with ifort and changed the naming of the libraries to have .lib appended. Also I removed /Qipo from the optimization flags. I am not sure if I got the /LINK and PATHs stuff correct. But the LIBPATH should anyway be just the local path so we could leave this away completely, I think. Let me know whether this causes trouble and I should remove the LINK and LIBPATH options.

  24. Bob Apthorpe reporter

    I'm still waiting for my home DSL to get fixed which is why I've been so quiet lately. Anyway, I did a quick test at work of 069f889df757 and ran into a minor issue during initial creation of the test program.

    From config.log:

    ['ifort.exe', '-IC:\\Program Files (x86)\\Intel\\Composer XE 2013 SP1\\compiler\\include', '-IC:\\Program Files (x86)\\Intel\\Composer XE 2013 SP1\\compiler\\include\\ia32', '-IC:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\INCLUDE', '-IPlatformSDK\\include', '-IC:\\Program Files (x86)\\Intel\\Composer XE 2013 SP1\\mkl\\include', '-IC:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\INCLUDE', '-IC:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\INCLUDE', '-IC:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.10240.0\\ucrt', '-IC:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.6.1\\include\\um', '-IC:\\Program Files (x86)\\Windows Kits\\8.1\\include\\shared', '-IC:\\Program Files (x86)\\Windows Kits\\8.1\\include\\um', '-IC:\\Program Files (x86)\\Windows Kits\\8.1\\include\\winrt', '-c', '-oC:\\Users\\Apthorpe\\Documents\\My Projects\\Infra\\IT\\aotus\\haraldkl-aotus-069f889df757\\build\\conf_check_efe0ee4dbaa2d6087e4c73dde47496e8\\testbuild\\test.f.1.o', 'C:\\Users\\Apthorpe\\Documents\\My Projects\\Infra\\IT\\aotus\\haraldkl-aotus-069f889df757\\build\\conf_check_efe0ee4dbaa2d6087e4c73dde47496e8\\test.f']
    

    The element:

    '-oC:\\<long path removed>\\testbuild\\test.f.1.o'
    

    should be

    '-o', 'C:\\<long path removed>\\testbuild\\test.f.1.o'
    

    It's not always clear when the MS/Intel tools want 'cuddled' options and when they want them separated. I believe this was working before so it should be pretty easy to isolate.

  25. Harald Klimach repo owner

    Actually I got finally hands on a Windows machine with ifort and msvc on it today. I got aotus configuring and compiling fine (besides unit tests, which fail due a wrong expectation on the executable name), but I am struggling a little to get this correctly set in waf, such that the setup could be used in other cases as well. I will later on push a waf, that I think should work for aotus along with comments on how I built the aotus_sample for intel64 with it as a comment here.

  26. Harald Klimach repo owner

    Here we go, 28346b816ae5 should now have the correct flags for the output names. Though I changed them according to the compiler option documentation of Intel and need to test this on the Windows machin myself tomorrow. Here is the procedure that I'd expect to produce the aotus_sample.exe:

    Set the compiler flags for intel by running compilervars.bat, which for me was:

     C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\compilervars.bat intel64
    

    Setting a define to properly deal with float.h:

    set DEFINES=__MS_VC_INSTALL_PATH="C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt"
    

    float.h is defined in the Intel compiler includes, but relies on the float.h from the msvc. However, on our installation the msvc somehow does not provide it. Instead, it is found in the kit for Windows 10? Not sure, what's going on here, if you did not have problems before, you probably do not need to set this DEFINE.

    Afterwards just run the configuration step:

    python waf configure
    

    For me, this now found msvc for the c compiler and ifort for the Fortran compiler. You could also use the options --msvc_version and --msvc_target to define the msvc version to use and the target architecture to compile for. --help should provide you with more information on these configure options.

    The configure step already compiles a small Fortran test. Thus, when the configure works fine, we have reached a good basis. Now compiling aotus_sample.exe can be achieved by

    python waf --target=aotus_sample
    

    Which should result in an executable in the generated build directory and which could be used in the sample directory.

    The library can be generated by running

    python waf --target=aotus
    

    The complete build currently does not work, due to the utest expectations on the filename. I'll have a look into this tomorrow.

  27. Bob Apthorpe reporter

    I made more progress; running

    python waf --command_sequence --target=aotus_sample
    

    does well for a while, then dies on

    ifort.exe /QxHost /O3 /Qprec-div- /warn:all '-IC:\\Program Files (x86)\\IntelSWTools\\<many_include_directories>\\include' /object:C:\Users\apthorpe\Documents\Projects\aotus\haraldkl-aotus-28346b816ae5\build\LuaFortran\dump_lua_fif_module.f90.4.o C:\Users\apthorpe\Documents\Projects\aotus\haraldkl-aotus-28346b816ae5\LuaFortran\dump_lua_fif_module.f90
    

    which need a little Windowsification to:

        ifort.exe /c /QxHost /O3 /Qprec-div- /warn:all -I"C:\Program Files (x86)\IntelSWTools\<many_include_directories>\include" /object:"C:\Users\apthorpe\Documents\Projects\aotus\haraldkl-aotus-28346b816ae5\build\LuaFortran\dump_lua_fif_module.f90.4.o" C:\Users\apthorpe\Documents\Projects\aotus\haraldkl-aotus-28346b816ae5\LuaFortran\dump_lua_fif_module.f90
    

    Several issues:

    • Need /c flag to compile-to-object-only; equivalent of gfortran's -c
    • Convert single to double quote, remove escaping on backslash
    • Change quote placement from "-IC:\<include_path>" to -I"C:\<include_path>"

    Once those changes were made to command_sequence.txt, the pieces compiled without incident.

  28. Harald Klimach repo owner

    Thanks to Thomas Nagy from waf, we now have a version, which runs kind of out of the box for me on windows with msvc and intel. Please try f08c7886c91b, this revision works for me on Windows in all respects, there are still some warnings due to some compiler flags I set, which are also passed to the linker, but besides that even the execution and checking of unit tests seems to work.

  29. Harald Klimach repo owner

    With c184a8a19cc9 we now have a slightly improved revision, which should not pass on the compiler flags to the linker anymore. I have attached the resulting command_sequence.txt on my box. There is also a waf.bat included now. Thus it should be possible to build Aotus with

    waf configure
    waf
    

    Please give this a new version a try. I will close this ticket as resolved if you have no objections.

  30. Log in to comment