1. Blaze Avatar Blaze
  2. Blaze
  3. blaze
  4. Issues

undefined behaviour due to LAPACK prototypes missing "hidden" variables required by Fortran

Issue #251 resolved
Conrad Sanderson created an issue

The prototypes for Fortran LAPACK functions appear to be incomplete in Blaze. This can lead to undefined behaviour with recent releases of gcc due to more aggressive compiler optimisations.

According to gfortran passing conventions (followed by other fortran compilers as well), every char* argument also has an associated "hidden" argument which specifies the number of characters. The type of the "hidden" argument may vary across compilers and/or compiler versions. The position of each "hidden" argument is also compiler dependent, though seems to be typically tacked onto the end of the function definition.
https://gcc.gnu.org/onlinedocs/gfortran/Argument-passing-conventions.html

As an example in Blaze, the current prototype for dsyev in
https://bitbucket.org/blaze-lib/blaze/src/master/blaze/math/lapack/clapack/syev.h
is defined as:
void dsyev_( char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info );

But it probably should be:
void dsyev_( char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info, blas_len jobz_len, blas_len uplo_len);

where blas_len is typically a 32 bit unsigned int on 32 bit platforms, and 64 bit unsigned int on 64 bit platforms. (This does not apply to gcc versions <= 7, where it is always int).

Avoiding the use of the blas_len arguments appeared to work, but recently gcc has started to optimise more heavily, leading to stack overwrites when these arguments are not used.

The same bug affects Armadillo, R, and a lot of other software in C and C++ that uses LAPACK:

Comments (8)

  1. Conrad Sanderson reporter
    • edited description

    The prototypes for Fortran LAPACK functions appear to be incomplete in Blaze. This can lead to undefined behaviour with recent releases of gcc due to more aggressive compiler optimisations.

    According to gfortran passing conventions (followed by other fortran compilers as well), every char* argument also has an associated "hidden" argument which specifies the number of characters. The type of the "hidden" argument may vary across compilers and/or compiler versions. The position of each "hidden" argument is also compiler dependent, though seems to be typically tacked onto the end of the function definition.
    https://gcc.gnu.org/onlinedocs/gfortran/Argument-passing-conventions.html

    As an example in Blaze, the current prototype for dsyev in
    https://bitbucket.org/blaze-lib/blaze/src/master/blaze/math/lapack/clapack/syev.h
    is defined as:
    void dsyev_( char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info );

    But it probably should be:
    void dsyev_( char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info, blas_len jobz_len, blas_len uplo_len);

    where blas_len is typically a 32 bit unsigned int on 32 bit platforms, and 64 bit unsigned int on 64 bit platforms. (This does not apply to gcc versions <= 7, where it is always int).

    The same bug affects Armadillo, R, and a lot of other software in C and C++ that uses LAPACK:

  2. Klaus Iglberger

    Hi Conrad!

    Thanks a lot for reporting this defect to us. We were unfortunately neither aware of the changes in gcc nor to these special Fortran conventions. Also thanks a lot for pointing out possible solutions.

    We will take some time to investigate the implications and potential differences on all supported platforms and think about reasonable changes. At this point, all test on all platforms and with all compilers (including gcc and in particular gcc-9.1) work perfectly, so we feel we have some time to properly investigate things. Thanks again,

    Best regards,

    Klaus!

  4. Klaus Iglberger

    The prototypes for Fortran LAPACK functions appear to be incomplete in Blaze. This can lead to undefined behaviour with recent releases of gcc due to more aggressive compiler optimisations.

    According to gfortran passing conventions (followed by other fortran compilers as well), every char* argument also has an associated &#34;hidden&#34; argument which specifies the number of characters. The type of the &#34;hidden&#34; argument may vary across compilers and/or compiler versions. The position of each &#34;hidden&#34; argument is also compiler dependent, though seems to be typically tacked onto the end of the function definition.
    https://gcc.gnu.org/onlinedocs/gfortran/Argument-passing-conventions.html

    As an example in Blaze, the current prototype for dsyev in
    https://bitbucket.org/blaze-lib/blaze/src/master/blaze/math/lapack/clapack/syev.h
    is defined as:
    void dsyev_( char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info );

    But it probably should be:
    void dsyev_( char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info, blas_len jobz_len, blas_len uplo_len);

    where blas_len is typically a 32 bit unsigned int on 32 bit platforms, and 64 bit unsigned int on 64 bit platforms. (This does not apply to gcc versions <= 7, where it is always int).

    Avoiding the use of the blas_len arguments appeared to work, but recently gcc has started to optimise more heavily, leading to stack overwrites when these arguments are not used.

    The same bug affects Armadillo, R, and a lot of other software in C and C++ that uses LAPACK:

  6. Klaus Iglberger

    Hi Conrad!

    Thanks again for reporting this defect. After some testing we have decided to follow the path you suggested and extend the all according Fortran forward declaration with hidden arguments. However, as soon as we do that the forward declarations of Armadillo and Blaze will be incompatible. Therefore we suggest to synchronize our efforts to enable user to include both Armadillo and Blaze headers.

    We will introduce a new type similar to the one suggested by GCC. All forward declarations will be extended by the according number of trailing hidden arguments. For instance:

    void dgetrs_(char* trans, int* n, int* nrhs, double* A, int* lda, int* ipiv, double* B, int* ldb, int* info, blaze::fortran_charlen_t ntrans);
void dsyev_(char* jobz, char* uplo, int* n, double* A, int* lda, double* w, double* work, int* lwork, int* info, blaze::fortran_charlen_t njobz, blaze::fortran_charlen_t nuplo);
void zgesvdx_(char* jobu, char* jobv, char* range, int* m, int* n, double* A, int* lda, double* vl, double* vu, int* il, int* iu, int* ns, double* s, double* U, int* ldu, double* V, int* ldv, double* work, int* lwork, double* rwork, int* iwork, int* info, blaze::fortran_charlen_t njobu, blaze::fortran_charlen_t njobv, blaze::fortran_charlen_t nrange);

    We hope this approach matches the one that you have selected.

    Best regards,

    Klaus!

  7. Conrad Sanderson reporter

    This is what I have so far in Armadillo for the Fortran hidden argument type:

    #if !defined(ARMA_FORTRAN_CHARLEN_TYPE)
  #if defined(__GNUC__) && !defined(__clang__)
    #if (__GNUC__ <= 7)
      #define ARMA_FORTRAN_CHARLEN_TYPE int
    #else
      #define ARMA_FORTRAN_CHARLEN_TYPE size_t
    #endif
  #else
    // TODO: determine the type for other compilers
    #define ARMA_FORTRAN_CHARLEN_TYPE size_t
  #endif
#endif

typedef ARMA_FORTRAN_CHARLEN_TYPE blas_len;

    ‌

    It’s a work in progress (relevant file in gitlab repo). So far only tested on Linux x86-64.

    size_t is an educated guess based on:

    I haven’t checked other Fortran compilers (eg. flang from clang/llvm). There is a list of compilers at:

  8. Klaus Iglberger

    Commit 08a5303 adds hidden arguments to all Fortran forward declarations with arguments of character type. The fix is immediately available via cloning the Blaze repository and will be officially released in Blaze 3.6.

  9. Log in to comment
Assignee
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!