Messed up locale and string - double conversion when reading from file

[Anders Logg (Chalmers)]

The following code gives an unexpected result:

from PySide import QtGui
from dolfin import File, Mesh, UnitSquareMesh, plot

# works if the following line is commented out
application = QtGui.QApplication([])

mesh = UnitSquareMesh(2, 2)
File("test.xml") << mesh
mesh = Mesh("test.xml")
plot(mesh, interactive=True)

The resulting plot shows a corrupted mesh:

This might be bug in the PySide package (from HomeBrew) but it's more likely (?) this is a bug in the XML reader triggered by loading something heavy before reading the mesh. If the line importing PySide is moved to after reading the mesh (but before plotting), the mesh looks good.

[Anders Logg (Chalmers)]

This also happens if I use PyQt4 in place of PySide.

[Chris Richardson]

Works normally for me on ubuntu. What is the contents of mesh.coordinates() and mesh.cells() after reading? And test.xml?

[Anders Logg (Chalmers)]

The XML fie is correct, but the coordinates seem to be messed up while reading.

Correct coordinates should be:

[[ 0.   0. ]
[ 0.5  0. ]
[ 1.   0. ]
[ 0.   0.5]
[ 0.5  0.5]
[ 1.   0.5]
[ 0.   1. ]
[ 0.5  1. ]
[ 1.   1. ]]

Instead I get this:

[[ 0.  0.]
[ 5.  0.]
[ 1.  0.]
[ 0.  5.]
[ 5.  5.]
[ 1.  5.]
[ 0.  1.]
[ 5.  1.]
[ 1.  1.]]

[Anders Logg (Chalmers)]

For some strange reason pugixml decides to interpret "5.000000000000000e-01" as the number 5.

Adding this inside the loop reading vertex coordinates in XMLMesh.cpp

std::cout << "x = " << it->attribute("x").as_string() << std::endl;
std::cout << "x = " << it->attribute("x").as_double() << std::endl;

gives the following output for the second vertex:

x = 5.000000000000000e-01
x = 5

[Anders Logg (Chalmers)]

Even stranger is that std::stod does the same thing. If I put the string value into a std::string and then convert into a double using std:stod, I still get 5...

[Chris Richardson]

how about boost::lexical_cast?

[Prof Garth Wells]

boost::lexical_cast is fine for testing, but it's too slow to be a solution.

[Anders Logg (Chalmers)]

as_string, std::stod and atoi all fail but boost::lexical_cast seems to work:

std::string s(it->attribute("x").as_string());
double d1(std::stod(s));
double d2(boost::lexical_cast<double>(s));
double d3(atof(s.c_str()));
std::cout << s << " " << d1 << " " << d2 << " " << d3 << std::endl;

Output:

5.000000000000000e-01 5 0.5 5

I'll experiment some more tomorrow.

[Benjamin Dam Kehlet]

Is something wrong with your locale? std::stod works as expected here.

[Anders Logg (Chalmers)]

How can I check/change that?

[Anders Logg (Chalmers)]

Update: This has nothing to do with the XML reader, and likely has nothing to do with DOLFIN. If I put the following code in begin() inside log.cpp (just to have a place to put C++ code to be called from the Python interface)

std::string s2("5.0e-01");
double d1(std::stod(s2));
double d2(boost::lexical_cast<double>(s2));
double d3(atof(s2.c_str()));
std::cout << s2 << " " << d1 << " " << d2 << " " << d3 << std::endl;

I get the following strange output:

5.0e-01 5 0.5 5

Any ideas for how to further debug this are welcome. It prevents me from running a GUI application using FEniCS and it shows up in reading the mesh from file, since that is where the conversion from string to double takes place.

[Martin Sandve Alnæs]

I get

5.0e-01 0.5 0.5 0.5

using

g++ (Ubuntu 4.9.1-16ubuntu6) 4.9.1

[Martin Sandve Alnæs]

Try looking into this:

http://www.cplusplus.com/reference/clocale/setlocale/

[Johannes Ring]

I get the same as @martinal on a OS X 10.10 machine.

[Benjamin Dam Kehlet]

What does locale (ie. on the command line) print?

[Anders Logg (Chalmers)]

locale gives

LANG="sv_SE.UTF-8"
LC_COLLATE="sv_SE.UTF-8"
LC_CTYPE="sv_SE.UTF-8"
LC_MESSAGES="sv_SE.UTF-8"
LC_MONETARY="sv_SE.UTF-8"
LC_NUMERIC="sv_SE.UTF-8"
LC_TIME="sv_SE.UTF-8"
LC_ALL=

[Anders Logg (Chalmers)]

Adding the following line (in C++) before calling atof, stod makes everything work:

setlocale(LC_ALL,"C");

[Anders Logg (Chalmers)]

So how should this be handled? Is my system messed up? Is PyQt messed up since it has the side-effect of making stod not work with my locale settings. Or could/should something be done on the DOLFIN side as a workaround? (But then DOLFIN might be harmful to other libraries/systems by messing up the locale settings.)

[Chris Richardson]

I have seen random weird behaviour in the past in dolfin when locale was unusual (sorry to be vague, but can't remember details). I don't see the harm of setting locale to "C" in dolfin - it shouldn't affect users in a bad way...

[Anders Logg (Chalmers)]

Perhaps a good place to put this would be in File::init?

[Anders Logg (Chalmers)]

Fix issue #498: Messed up locale and string - double conversion when reading from file

→ <<cset 1bdf81da42f2>>

[Anders Logg (Chalmers)]

And btw thanks for the comments pointing me to setlocale!

[Jan Blechta]

Other occurrence, maybe one Chris was thinking about, was some stringy parameter [Par]METIS or SCOTCH was passing in or out in some routine. I can't remember also whether it was direct solver reordering or mesh partitioning but I think that the bug is still present and the best is to use LC_ALL="C" with whole DOLFIN.

[Chris Richardson]

Here is one crash that was sent to me, which were resolved by setting the locale to "C". In this case, it is in File.cpp

terminate called after throwing an instance of 'std::runtime_error'
  what():  locale::facet::_S_create_c_locale name not valid

Program received signal SIGABRT, Aborted.
0x00007ffff5ee6f79 in __GI_raise (sig=sig@entry=6) at  ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff5ee6f79 in __GI_raise (sig=sig@entry=6) at  ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff5eea388 in __GI_abort () at abort.c:89
#2  0x00007ffff64ec6b5 in __gnu_cxx::__verbose_terminate_handler() ()  from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff64ea836 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff64ea863 in std::terminate() () from  /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff64eaaa2 in __cxa_throw () from  /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff653c677 in std::__throw_runtime_error(char const*) ()  from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007ffff65080c4 in  std::locale::facet::_S_create_c_locale(__locale_struct*&, char const*,  __locale_struct*) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#8  0x00007ffff64fd2e9 in std::locale::_Impl::_Impl(char const*,  unsigned long) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007ffff64fe41b in std::locale::locale(char const*) () from  /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007ffff59422c2 in boost::filesystem::path::codecvt() () from  /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0
#11 0x00007ffff594277f in boost::filesystem::path::begin() const ()  from /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0
#12 0x00007ffff5942fb5 in  boost::filesystem::path::compare(boost::filesystem::path const&) const  () from /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0
#13 0x00007ffff59431e5 in boost::filesystem::path::extension() const  () from /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.54.0
#14 0x00007ffff786b980 in extension (p=...) at  /usr/include/boost/filesystem/convenience.hpp:36
#15 dolfin::File::init (this=this@entry=0x7fffffffd400,  comm=comm@entry=0x7ffff7022ec0 <ompi_mpi_comm_world>, filename=...,  encoding=...) at /opt/source/fenics-dev/dolfin/dolfin/io/File.cpp:156
#16 0x00007ffff786c7f5 in dolfin::File::File (this=0x7fffffffd400,  filename=..., encoding=...) at  /opt/source/fenics-dev/dolfin/dolfin/io/File.cpp:51

[Jan Blechta]

It seems that it would be good to set LC_ALL="C" for whole DOLFIN.