UTF-8 characters in the path cause problems

[Barnabás Králik]

Trace:

[DEBUG][MainThread] on_fileopengerber()
Traceback (most recent call last):
  File "/home/kralikba/flatcam/FlatCAMApp.py", line 1417, in on_fileopengerber
    filename = str(filename)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 41: ordinal not in range(128)

[Juan Pablo Caram]

I don't know what this is about. Why would you have u'\xe1' in your path? Is your OS using some extended set of characters? What language are you using? Which OS?

[Barnabás Králik]

I'm using UTF-8, which is the most common encoding for Unicode characters. It is "extended" in the sense that it can be used to represent letters of languages other than English (and a few "relatives" thereof). UTF-8 is a variable length encoding. The first 4 bits of u'\xe1' signify that the upcoming character is represented on altogether 3 bytes. The ã letter of "João" is also encoded on more than one byte (u'\xc3a3'); its first three bits (110) signify that it is 2 bytes in length.

I'm using Ubuntu 14.10, my locale is set to hu_HU.UTF-8. I unfortunately do not know Python and how it handles locales, but generally I would expect it to have some simple to use facilities available that let string handling use the system-wide encoding. Some quick google searches suggest checking out the "locale" library, using it to acquire the preferred encoding and passing that to byte strings' .decode method.

[Juan Pablo Caram]

Increased priority as it's been confirmed by other users.

Does any one know a simple way to replicate this? I mean, without having to make major changes to my PC?

[Marius Stanciu]

You can always use a virtual machine software and use a virtual image already made so you don't have to install everything from scratch.

[Barnabás Králik]

1. (e.g.) Verify that on the machine (not over ssh!) 'locale' outputs all lines with the suffix UTF-8. (On modern systems, this would most probably the case)
2. Copy-paste in an empty directory: 'mkdir tükörfúrógép-árvíztűrő'
3. Create a file in this directory and open it with flatcam.

[Juan Pablo Caram]

Attempt to fix #138. Keeping open for verification.

→ <<cset 03c49f3afdfb>>

[Juan Pablo Caram]

Git closed it automatically. Please give it a try. I only did a quick test with a single Gerber using the procedure described by @kralikba .

[Juan Pablo Caram]

Reopened until formal unit test is added.

[Matthieu Berthomé]

Issue #233 was marked as a duplicate of this issue.

[Juan Pablo Caram]

Merged pull request #66.

I'm still going to keep this open since it might affect other file-system interactions.