Problem with decimal separator on Windows 7 and 8.1

[Felix Ulber]

Hey there, there seem to be a serious problem with decimal separator. I hope that is not a case of wrong setting/usage.

1) Wenn I draw a simple point at distance and angle, it almost always draws the line at a completely wrong angle. Editing the point still shows the correct angle, when I remove the fractional part of the angle value it draws correctly 2) typing in a fractional number in any formula produces an error.

By default I use a german windows version with US/international keyboard layout. I change the "with os options" setting, I switched my keyboard layout to german - anyway it won't accept neither a "." nor a "," as decimal separator making it practiacally impossible to use fractional numbers in any formula

Felix

[Felix Ulber]

Can reproduce that on Linux (ubuntu 14 64bit - did I mention I love Virtual Box :). Maybe the debugger will help me figure it out.

[Roman Telezhynskyi]

What language do you use in Valentina? There can be some problem with translation.

[Felix Ulber]

Yes, I can confirm it only happens in german. Still not sure if the 1) (above) has to do with it. Will try to find out more tomorow

[Roman Telezhynskyi]

This part always make troubles. I think we should think about creation unit tests for parser. Also it is can be error in translations. Some strings need very careful translate, parser depend on it.

[Former user]

I can confirm problems with the decimal separator. I use a German Windows 7, language and keyboard settings set to German. I have the latest download version of Valentina (from beginning of January) installed and it is set to German. When I have the check mark set for "with OS options (')" the following happens: I cannot enter any fractional number (neither with . nor with ,). If I draw a line the dialog that opens shows me the value of the angle as "270,171" in the textbox but the value shown as label next to the buttons shows "270.171". Maybe that helps a bit. Conceptually (I'm working as a software developer) usually you stick to "." for everything internally and then you have a convert function that converts every in/output to the UI into the correct format. Not sure how that works with Qt, but normally I would inherit from the standard textbox and add that conversion feature to it as a basic functionality (if your control does not have this setting anyhow). Then I use my special textbox everywhere and don't have to worry about converting issues with textboxes anymore. But as I said, it's a pain. I personally would not mind to enter values with a . instead of , I prefer to enter . than to worry about errors resulting from inconsistencies. Just saying maybe the time invested in getting "," to work properly would be better invested in other features or more serious bugs. But I don't know your code, so maybe it is just a small thing :-)

[Roman Telezhynskyi]

Hi @fuerchtnix, thanks for so good bug report.

When I have the check mark set for "with OS options (')" the following happens: I cannot enter any fractional number (neither with . nor with ,).
Because we have difficult expressions we can't allow user use in formula two decimal separators at the same time. After check mark was setted to "," all fractional numbers should be with ",". So this part i can confirm.

If I draw a line the dialog that opens shows me the value of the angle as "270,171" in the textbox but the value shown as label next to the buttons shows "270.171".
Can't confirm this. I have found that numbers like 270,171 works good, but 10,0 cause error. Parser convert 10,0 to 10 and ignore conversion for such numbers.

Conceptually (I'm working as a software developer) usually you stick to "." for everything internally and then you have a convert function that converts every in/output to the UI into the correct format.
We do exactly the same. All expressions in pattern file use "." for fractional numbers.

Not sure how that works with Qt, but normally I would inherit from the standard textbox and add that conversion feature to it as a basic functionality (if your control does not have this setting anyhow). Then I use my special textbox everywhere and don't have to worry about converting issues with textboxes anymore.
This is bug in Valentina code. Qt doesn't do any changes in textbox for us. And conversation we make in special methods that handles not only separators, but also variables names.

[Roman Telezhynskyi]

@kjaft , i have made changes. Can you test them?

[Felix Ulber]

Yeah, tomorrow I'll try. Unfortunately still my Windows build environment doesn't work for no reason (Qt 5.3). Do you use MS or Mingw?

[Roman Telezhynskyi]

Good.

[Felix Ulber]

Hi Roman, unfortunately from my first test now I cannot see any improvement - it seems rather worse. I now have to systematically test all combinations and see what goes wrong or not in each special case. That is a combination of three factors: Valentina language, system language and - very important: system keyboard layout, I think the latter one is just not detected in a correct way. Also, the resulting problems are of different kinds: one is the inital drawing in with wrong (in this case) angle, while the formula is displayed correctly in the tool dialog, the second is an error when editing an existing value - when the formula parsing goes wrong (and so wrong value can go to drawing). The third one is quite strange, when using the tools propery editor on the right, value is accepted but the temporarily draws it the wrong way again, with the unchanged line still drawn on screen. Sorry, I know this description may be just confusing and hard to follow. I have to do more tests and understand the parser better. I am not sure if the problem is on the input side only or also in the parser.

[Roman Telezhynskyi]

Did you tried begin with new file or opened compromised?

I still don't know expression that don't work for you. Can you publish some example?

Sorry, I know this description may be just confusing and hard to follow. I have to do more tests and understand the parser better. I am not sure if the problem is on the input side only or also in the parser.
Do you need more explanations about parser work?

[Felix Ulber]

I allways start with an empty pattern - and the simplest case of all, a point at distance and angle is sufficient to produce the problem. Maybe a screen recording would be a good idea? Do you know a good tool for that purpose?

Do you need more explanations about parser work?

That would be great. Still first I am gonna look after which values actually reach the parser from interface.

[Roman Telezhynskyi]

Thanks that show me that markdown support quotes.

Maybe a screen recording would be a good idea? Do you know a good tool for that purpose?

For which Os? For Linux i use Kazam.

That would be great. Still first I am gonna look after which values actually reach the parser from interface.

I will make documentation for parser where can. But i don't wait that first version will be perfect. So i need your help.

[Roman Telezhynskyi]

@kjaft , i made some documentation improvements. Hope it help.

[Felix Ulber]

Thanks Roman,

I still don't get it. Just need to ask some questions for clarification:

• The UI Language setting does not affect the decimal separator right? I just realized that, and I think a user might probably assume other.

• Thus, as long as "from os" setting is off, dot as decimal separator works just fine, independetly from valentina language, os language and keyboard layout setting

• does it mean it wouldn't alow me to use german keyboard layout on any non-german system, cause neither valentina UI language nor keyboard layout is considered?

• Debugging FormulaFromUser, this function seems to work correctly - if my assumption is correct, that here allways formula strings get converted to dot decimal separator noatation? Unfortunately problems still occur, some times for example a formula gets even displayed with "," in the value field, while typed in using "."...

• I just see, that the ",0"-thing works everytime - the reason i suspect in the FormulaFromUser, cause zero fractional part is thrown away during the conversion, so the value string does not contain any decimal separator at all after that.

I now gonna look into the FormulaToUser and the Parser, still I wasn't able to follow the comple data flow.

[Felix Ulber]

I hope I can manage to do a screen recording tomorrow

[Roman Telezhynskyi]

The UI Language setting does not affect the decimal separator right?

Yes.

I just realized that, and I think a user might probably assume other.

When you change language it is still not mean you change locale. And option clear say that it is OS separator.

Thus, as long as "from os" setting is off, dot as decimal separator works just fine, independetly from valentina language, os language and keyboard layout setting

Correct.

does it mean it wouldn't allow me to use german keyboard layout on any non-german system, cause neither valentina UI language nor keyboard layout is considered?

No, "from os" setting only for decimal separator only. It should allow you use any language layout you want.

if my assumption is correct, that here allways formula strings get converted to dot decimal separator noatation?

Yes, All formulas in pattern file use dot decimal separator.

I just see, that the ",0"-thing works everytime - the reason i suspect in the FormulaFromUser, cause zero fractional part is thrown away during the conversion, so the value string does not contain any decimal >separator at all after that.

Yes, it is my last fix. I think we shouldn't spend time to try save zero fractional part.

[Felix Ulber]

does it mean it wouldn't allow me to use german keyboard layout on any non-german system, cause neither valentina UI language nor keyboard layout is considered?

No, "from os" setting only for decimal separator only. It should allow you use any language layout you want.

Sorry, what I meant was, that in case for example I work on an english OS, but with german keyboard layout. For this is about my input device, I clearly want the decimal separator to work like the keyboard layout, not the os language. And that is not possible atm.

This is the one thing, more related to user-friendlyness. The other Problem is still the wrong internal dealing with numbers in case of german (os and "from os" settting). As I wrote, the FormulaFromUser and FormulaToUser methods seem to do their Job - but still after that something fails. Continuing debugging...

[Roman Telezhynskyi]

Sorry, what I meant was, that in case for example I work on an english OS, but with german keyboard layout. For this is about my input device, I clearly want the decimal separator to work like the keyboard layout, >not the os language. And that is not possible atm. This is the one thing, more related to user-friendlyness.

If you work on an english OS you should set correct regional standard that you want use. Windows, Mac and Linux have such options. This don't change language of the system, but will change regional standard for showing numbers, dates and currency. So, my opinion, this is not Valentina issue.

[Felix Ulber]

I don't agree completely, things are not that simple unfortunately. What I use very often is just 2 different keyboard layouts that I can quickly switch between. In case of windows for example, I set my "regional standard" to german (independently from if the os language itself). Nevertheless I can specify more than one Keyboard layout for that regional setting. Its the same on KDE (though on Linux things are a little bit more complex cause there are several configuration "layers").

So this is actually a matter of current keyboard layout vs. system language (or locale or regional standard) and thus directly related to the interpretation of the applications setting "From OS". Qt hast some measures to handle that, that exceed QLocale::system() - I mentioned keyboardInputLocale, but a quick test did not clear up things completely how this is to apply.

[Roman Telezhynskyi]

Ok, my conclusions based on my experience. If you show me way how handle this, we will implement this feature. Maybe i really know only about simple case.

[Felix Ulber]

Ok, I think about it and better start a discussion on the list or open up another feature request.

Sorry, my bad because this issue is actually about a different problem. I was able to break it down a bit further, but it is still strange. Problems definitely appear below the first Eval() call in Calculator::EvalFormula(). Here, the formula always is in dot decimal separator notation how it is supposed. Nevertheless strange things happen. There are two scenarios:

• That thing, where line angle is wrong after creation/drawing (simple point at distance and angle): This is caused by the decial separator just being removed from the expression, so an angle of 345,678 becoms an angle of 345678. There is no exception from the parser, just returns the wrong thing.

• When editing the value (in dialog or afterwars in options / toolpropertyeditor / wrongformula editor) the expression is throwing an "unexpected token .678 at position 3" error - although the formula is always correctly evaluated in dot-decimal notation. So this shouldn't be a problem about localisation but nevertheless only happens when using values converted from "," before.

Something goes nuts with the parser here. Strange: some times when debugging step-wise, it works ok. But most of the time not. Still have to learn more about the muParser.

I all tests I use only simple singe-value "formulas", no variables at all. It happens on Linux the same way as on Windows.

[Roman Telezhynskyi]

I still can't reproduce. Can you send me a file?

[Felix Ulber]

hm, have you got a system with "," decimal separator? I'll send you a file, but this is really the simplest of all test cases - just on the first drawn point in an new file.

Good point what I did not try yet is to open a file - in this case it is fine after opening - as soon as I select the point, it is drawn in red with the wrong angle again. Editing the point/formula produces the same behaviour as described.

I really gotta get on with those screenrecordings...

[Felix Ulber]

Simple test scene just containing one point at distance and angle

[Roman Telezhynskyi]

Here is my result. Still all works how need.

[Felix Ulber]

Very strange, too bad you cant reproduce it. The "only" difference I see is that my formula value is always displayed with dot - see attachment.

[Roman Telezhynskyi]

Hooray, i can reproduce. It is only appear if set German locale.

[Roman Telezhynskyi]

Now, i see why german user so passive.

[Roman Telezhynskyi]

int QmuParser::IsVal(const QString &a_szExpr, int *a_iPos, qreal *a_fVal)
{
    qreal fVal(0);

    std::wstring a_szExprStd = a_szExpr.toStdWString();
    stringstream_type stream(a_szExprStd);
    stream.seekg(0);        // todo:  check if this really is necessary
    stream.imbue(QmuParser::s_locale);
    stream >> fVal;
    stringstream_type::pos_type iEnd = stream.tellg(); // Position after reading

    if (iEnd==static_cast<stringstream_type::pos_type>(-1))
    {
        return 0;
    }

    *a_iPos += static_cast<int>(iEnd);
    *a_fVal = fVal;
    return 1;
}

Here this bustard. Seams like std::basic_stringstream ignore "." in string.

[Felix Ulber]

Ah, cool you already found it. Still doesn't make really sense why it only appears in a formula translated from "," to "." before. Or does it?

Now, i see why german user so passive

Lol

Hooray, i can reproduce. It is only appear if set German locale

you mean with other languages using "," it works?

[Roman Telezhynskyi]

Ah, cool you already found it. Still doesn't make really sense why it only appears in a formula translated from "," to "." before. Or does it?

Can't say exactly, need continue to dig.

you mean with other languages using "," it works?

For me yes. This is something more than just problem with separator.

This code doesn't work correct.

stringstream_type stream(a_szExprStd);
stream.seekg(0);        // todo:  check if this really is necessary
stream.imbue(QmuParser::s_locale);
stream >> fVal;

But i can't find better solution because need

stringstream_type::pos_type iEnd = stream.tellg(); // Position after reading

[Felix Ulber]

Ok, but this would probably only consider the first part of the problem? While still afterwards every input except non-fractional numbers is rejected.

I am really sorry, but here it also the whole thing appers in french and netherlands.

[Roman Telezhynskyi]

If you still here can you test this patch?

[Felix Ulber]

Yesssss.... almost.

The only one that doesnt wanna join in is the tooloptionspropertybrowser:

1. create anything
2. in the tooloptionspropertybrowser edit the formula
3. confirm-close the dialog
4. int the tooloptionspropertybrowser the number is show again with dot

Was this something static that was interfering with the two different ways Calculater is used?

[Roman Telezhynskyi]

Was this something static that was interfering with the two different ways Calculater is used?

Do you ask about s_locale? I think better i will make it local for each parser instance.

[Susan Spencer]

Yes it would make sense if this is why we haven't heard much from the German users.

[Roman Telezhynskyi]

Second try. Please, check.

[Felix Ulber]

hey Roman, unfortunately that does not change anything. The Issue mentioned above with edit formula/tooloptionspropertybrowser is stil there. Maybe this one is rather on the UI side? Some missing FormulaToUser or so?

Also discovered another one (already existing with patch 1):

1. Create an Arc, using fractional values for radius and first angle, fractional value + any variable on angle 2
2. Press Ok
3. Complaint about wrong formula, strangewise only on radius and angle 2
4. Chose edit/correct formula and just confirm without changing anything
5. values are accepted now

For this does not occur in other dialogs (though didnt try all), it also seems like something in UI domain.

All this tested on Windows and Linux.

cheers

[Felix Ulber]

I am sorry, yet another thing: In the Curve/Spile dialogs, spinboxes are used. Those all use a dot as decimal separator everytime. Might not be as urgent as the stuff above.

[Roman Telezhynskyi]

I am sorry, yet another thing: In the Curve/Spile dialogs, spinboxes are used. Those all use a dot as decimal separator everytime. Might not be as urgent as the stuff above.

It is because we don't set locale settings for sinboxes. I think we should change it.

[Roman Telezhynskyi]

hey Roman, unfortunately that does not change anything.

Sorry, i did not understand you correct. Also confirm. I already hate this bug. The biggest i ever saw.

[Felix Ulber]

Sorry, i did not understand you correct.

I meant I do not recognize an difference in behaviour between patch1 and patch 2.

Yeah, this one is a hard one. I am asking myself if this is actually an issue allready in muParser or just due to the way it is used in valentina.

[Roman Telezhynskyi]

I am asking myself if this is actually an issue allready in muParser or just due to the way it is used in valentina.

Both. I made muParser fork that now use Qt containers instead of STL. I think just because muParser doesn't perfect in some case he doesn't suit good for all our purposes.

[Roman Telezhynskyi]

Maybe this one is rather on the UI side? Some missing FormulaToUser or so?

Actually you are right.

[Roman Telezhynskyi]

You can try my last commits in feature branch.

[Felix Ulber]

This looks promising! Both problems (tooloptionspropertybrowser thing and arc dialog thing) are gone.

Though I had some probs with line/arc intersection took, but those maybe don't belong to this issue.

btw: how can I get to your feature branch? I now made another local clone of this repo for this test case, but can't I do it via my fork? Sorry, still not that experienced with hg.

[Felix Ulber]

When I switch the "With OS" option off in preferences while having a point selected, the tooloptionspropertybrowseropen does not get updated and still shows the "," separator until I select something different. A minor thing, but maybe one can trigger an update when prefs window is accepted

[Roman Telezhynskyi]

Though I had some probs with line/arc intersection took, but those maybe don't belong to this issue.

If you think it is really a bug better create new issue ticket. Just don't want lose bugs.

btw: how can I get to your feature branch? I now made another local clone of this repo for this test case, but can't I do it via my fork? Sorry, still not that experienced with hg.

Run hg update feature.

When I switch the "With OS" option off in preferences while having a point selected, the tooloptionspropertybrowseropen does not get updated and still shows the "," separator until I select something different. A >minor thing, but maybe one can trigger an update when prefs window is accepted

Maybe better close tooloptionspropertybrowser when user open options? Or maybe quick close and open again that will act like update.

This looks promising!

If you feel that we fixed bug please say me.

[Felix Ulber]

Run hg update feature.

That was what I thought but somehow your feature branch was not synced to my fork repository

If you feel that we fixed bug please say me.

hehe, yeah, my impression is its fixed, but after all I can't be completely sure. It probably needs more testing, but I would nevertheless put it in the develop branch and go on, what do you think?

[Roman Telezhynskyi]

hehe, yeah, my impression is its fixed, but after all I can't be completely sure. It probably needs more testing, but I would nevertheless put it in the develop branch and go on, what do you think?

Yes, good idea. We will close issue, but if something will show yourself we always can open again. Thanks for good work.

[Felix Ulber]

testing the feature branch, I get some strange behaviour with the arc when in "," mode. Open the attached file and play around with the arc (at A1) start and end angle values - different kinds of drawin errors occur.

[Roman Telezhynskyi]

testing the feature branch, I get some strange behaviour with the arc when in "," mode. Open the attached file and play around with the arc (at A1) start and end angle values - different kinds of drawin errors occur.

Yes, also tool don't show visualization, red arc above original and only center point. I also saw this yesterday. Working on it, thanks.

[Roman Telezhynskyi]

Close by now. All known problem was resolved.

[Felix Ulber]

Great! Thanks alot! sorry, still the tooloptionspropertybrowser/edit formula combination introduces a dot. This happens for example with the attached arc scene editing the first angle.

[Roman Telezhynskyi]

still the tooloptionspropertybrowser/edit formula combination introduces a dot. This happens for example with the attached arc scene editing the first angle.

Can't reproduce this time. Are you sure you use all last changes?

[Roman Telezhynskyi]

testing the feature branch, I get some strange behaviour with the arc when in "," mode. Open the attached file and play around with the arc (at A1) start and end angle values - different kinds of drawin errors occur.

This error still present. Maybe you want this time fix it yourself?

[Felix Ulber]

This error still present.

Really? Here this one does not occur anymore - thought that was concerning commit "Fixed ghf1 and f2 angles for arc visualization."

I pulled all the lastes changes. Let me try creating a new pattern file.

[Felix Ulber]

ok, now i reproduced both and I think they are related. Suspect the tooloptionspropertybrowser. Still have to understand more about this works. I will try to care about.

[Roman Telezhynskyi]

Too late. :)

[Felix Ulber]

ok, the drawing irregularity is fixed. What is still there is the thing in tooloptionspropertybrowser. It occurs without edirwrongformula dialog, but you have to change the value 3 or more times, then its wrong (that thing where vformula just discards the decimal separator and stores a huge dvalue internally).

This is driving me nuts Currently debugging just not yet gives me a hint.

[Roman Telezhynskyi]

It occurs without edirwrongformula dialog, but you have to change the value 3 or more times, then its wrong (that thing where vformula just discards the decimal separator and stores a huge dvalue internally).

Strange, because i still can't reproduce. I need more details.

[Felix Ulber]

Please try the following to reproduce:

1. new pattern

2. create point at distance and angle, at least one of the parameters a fractional value (could be any tool though)

3. Edit the fractional value multiple times in the tooloptions..., adding or changing values to both parts (in frond and behind the dec. sep.), hit enter after each change to apply it

4. After the 3rd or 4th edit, suddenly the number is displayed with dot and the drawing goes wrong

Ii did a fresh clone of the main repo, tested on windows and linux (no difference).

I will try do further debug, but still data flow is not completely obvious to me.

[Roman Telezhynskyi]

I still can't reproduce. Ok, this time i will try help you to debug.

suddenly the number is displayed with dot and the drawing goes wrong

This mean value gos to tool.
Please check

void VToolEndLine::SetFormulaAngle(const VFormula &value)   

when you changing angle and

void VToolLinePoint::SetFormulaLength(const VFormula &value)

when you changing length.

Class VFormula must warn tool that this expression wrong and don't need to update value. If class VFormula doesn't do that issue can be in there. But because i can't reproduce also can't check.

Also i recommend you to check class VFormulaPropertyEditor. This class use VFormula for showing and saving expression.

[Felix Ulber]

I was already targeting those getter/setter functions when debugging. What I forgot to mention is, that first time in VFormulaPropertyEditor the formula already comes in wrong, just containing the string 'Error'. After the next step the value is correct, until after several edits as desribed.

I just couldn't find out yet, from where the VFormula object containing the wrong data comes in. Still not completely understood the dataflow, more specificly between the editor and the tool.

I can only continue next week.

[Roman Telezhynskyi]

I can only continue next week.

I understand. What about create video or gif animation?

[Felix Ulber]

don't get me wrong, that doesnt mean I don't wanna care about. Simply no time until the beginning of next week. Not sure if I find the time to create a video, still to check for screencasting app

[Roman Telezhynskyi]

don't get me wrong, that doesnt mean I don't wanna care about. Simply no time until the beginning of next week.

I understand this. I will wait.

[Felix Ulber]

Hi Roman, seems I have found it - at least for me it definitely fixes that error. For you could not even reproduce it, you can't really test it... I push it into my repo, maybe you can just have a look if this seems plausible (to me it does, but you know the code better).

thanks

Felix

[Felix Ulber]

Btw: Working on this, I found some irritating variable names - eg. in VFormula there are those member variables formula and value - both QStrings. In the VProperty context same names are used for variables of type VFormula/QVariant - this was really confusin during debugging. I would do some refactoring here, but I wanted to ask you first - what do you think?

[Roman Telezhynskyi]

I would do some refactoring here, but I wanted to ask you first - what do you think?

Ok, you can do it if you want. Did you see page about Code style conventions?