- edited description
TValueHelper.FromVariant, TFmtBcdVariant return type also needs to handle Floats
A while back we made a suggestion to fix an issue with Delphi XE5 and earlier where Oracle was returning TFmtBcdVariant types, and they were losing precision because of a bug in XE5.
Please see issue #64 for the previous fix.
We have now run into an issue where the Bcd Type (which is really a decimal type) is also being used by Firedac to return floating point types and now the above fix fails as it tries to cast a floating point number to an int64
We have come up with a fix for this and wanted to present it to you, so that you might consider integrating it into the latest code.
From the Spring.pas unit see the following changed method.
class function TValueHelper.FromVariant(const value: Variant): TValue;
type
TCustomVariantTypeInfo = record
Name: string;
VType: TVarType;
end;
const
CustomVariantTypes: array[0..2] of TCustomVariantTypeInfo = (
(Name: 'SQLTimeStampVariantType'; VType: varDouble),
(Name: 'SQLTimeStampOffsetVariantType'; VType: varDouble),
(Name: 'FMTBcdVariantType'; VType: varUnknown)
);
var
typeName: string;
i: Integer;
begin
case TVarData(Value).VType of
varEmpty, varNull: Exit(Empty);
varBoolean: Result := TVarData(Value).VBoolean;
varShortInt: Result := TVarData(Value).VShortInt;
varSmallint: Result := TVarData(Value).VSmallInt;
varInteger: Result := TVarData(Value).VInteger;
varSingle: Result := TVarData(Value).VSingle;
varDouble: Result := TVarData(Value).VDouble;
varCurrency: Result := TVarData(Value).VCurrency;
varDate: Result := From<TDateTime>(TVarData(Value).VDate);
varOleStr: Result := string(TVarData(Value).VOleStr);
varDispatch: Result := From<IDispatch>(IDispatch(TVarData(Value).VDispatch));
varError: Result := From<HRESULT>(TVarData(Value).VError);
varUnknown: Result := From<IInterface>(IInterface(TVarData(Value).VUnknown));
varByte: Result := TVarData(Value).VByte;
varWord: Result := TVarData(Value).VWord;
varLongWord: Result := TVarData(Value).VLongWord;
varInt64: Result := TVarData(Value).VInt64;
varUInt64: Result := TVarData(Value).VUInt64;
{$IFNDEF NEXTGEN}
varString: Result := string(AnsiString(TVarData(Value).VString));
{$ENDIF}
varUString: Result := UnicodeString(TVarData(Value).VUString);
else
case TVarData(Value).VType and not varArray of
varSmallint: Result := TValue.From<TArray<SmallInt>>(Value);
varInteger: Result := TValue.From<TArray<Integer>>(Value);
varSingle: Result := TValue.From<TArray<Single>>(Value);
varDouble: Result := TValue.From<TArray<Double>>(Value);
varCurrency: Result := TValue.From<TArray<Currency>>(Value);
varDate: Result := TValue.From<TArray<TDateTime>>(Value);
varOleStr: Result := TValue.From<TArray<string>>(Value);
varDispatch: Result := TValue.From<TArray<IDispatch>>(Value);
varError: Result := TValue.From<TArray<HRESULT>>(Value);
varBoolean: Result := TValue.From<TArray<Boolean>>(Value);
varVariant: Result := TValue.From<TArray<Variant>>(Value);
varUnknown: Result := TValue.From<TArray<IInterface>>(Value);
varShortInt: Result := TValue.From<TArray<ShortInt>>(Value);
varByte: Result := TValue.From<TArray<Byte>>(Value);
varWord: Result := TValue.From<TArray<Word>>(Value);
varLongWord: Result := TValue.From<TArray<LongWord>>(Value);
varInt64: Result := TValue.From<TArray<Int64>>(Value);
varUInt64: Result := TValue.From<TArray<UInt64>>(Value);
varUString: Result := TValue.From<TArray<string>>(Value);
else
typeName := VarTypeAsText(TVarData(Value).VType);
for i := 0 to High(CustomVariantTypes) do
if SameText(typeName, CustomVariantTypes[i].Name) then
begin
case CustomVariantTypes[i].VType of
varDouble: Result := Double(Value);
varUnknown: {FMTBcdVariantType}
begin
if BcdScale(VarToBcd(Value)) > 0 then
Result := BcdToDouble(VarToBcd(Value))
else
begin
Result := {$IFDEF DELPHIXE6_UP}Int64(Value);
{$ELSE}StrToInt64(VarToStr(Value));{$ENDIF} // see QC#117696
end;
end
else
raise EVariantTypeCastError.CreateRes(@SInvalidVarCast);
end;
Exit;
end;
raise EVariantTypeCastError.CreateRes(@SInvalidVarCast);
end;
end;
end;
Also the dbrtl.dcp needs to be added to the Spring.base package, so that the compiler does not complain that Data.FmtBcd exists both in dbrtl.dcp and spring.base when compiling Spring.Persistence.
My apologies for not catching this earlier, and thanks for all your hard work on this project as always. Todd.
Comments (9)
-
reporter -
reporter - edited description
-
repo owner Thanks, we will look into it but I can tell you that unfortunately the presented solution can't be accepted because Spring.Base must not get a dependency to dbrtl.
-
reporter Understood. Here is a version using Frac from sysutils
case CustomVariantTypes[i].VType of varDouble: Result := Double(Value); varUnknown: begin if (Frac(Extended(Value)) > 0) then Result := Extended(Value) else Result := {$IFDEF DELPHIXE6_UP}Int64(Value); {$ELSE}StrToInt64(VarToStr(Value));{$ENDIF} // see QC#117696 end
And it returns the higher precision extended type.
-
repo owner Isn't the original workaround to convert FMTBcd to Int64 the problem and should be solved differently and shouldn't FMTBcd always convert to Double? Shouldn't you use the FireDAC Data Type Mapping instead? (see http://docs.embarcadero.com/products/rad_studio/firedac/Data_Type_Mapping.html)
-
reporter Converting a large FMTBcd (18 char or more I believe) with no Scale to a Double will cause a loss of precision as double is not large enough to hold it. That is what the VarToBcd method was doing and what was fixed by QC 117696 so converting to double would reintroduce the bug.
Oracle does not know the difference between an Integer and a floating point number and Firedac returns them all as TFMTBcd variants. Soooo if a number has no scale then it really is an integer, Since int64 can hold up 19 character numbers it works well in this scenario. If you don't want to provide the int64 conversion when there is no scale, then I would suggest that you return them as extended as it has higher precision than double. The nice thing about what my second suggested fix does is that it alleviates the need for the calling process to convert to an integer when the result really is an integer anyway.
-
repo owner - Have you tried defining some data type mappings so FD does not return a FMTBcd but a Int64 for number(19) columns which would be the correct type so far?
- Your fix is not a fix but a hack because it only works on 32bit Intel platform (see http://docwiki.embarcadero.com/RADStudio/Seattle/en/Internal_Data_Formats#The_Extended_type)
The only true solution to me is to cause FD not to return a FMTBcd but define a data type mapping for all the different column data types you are using because there is no way to find out what the Variant really holds without using the Data.FmtBcd unit. And if we have to do that it would require some callback mechanism for TValue.FromVariant because we cannot put the dependency on the dbrtl package in the Spring.Base package.
So please try the FD data type mappings and then report back.
Edit: I just tried this and I could properly let FD convert Number(19) into Int64 resulting in a TLargeIntField instead of TFMTBCDField and Number(19,4) into Double resulting in a TFloatField.
-
repo owner - changed status to wontfix
If you don't want to use the data type mapping another solution could be to use a custom adapter and implement the GetFieldValue functions of its resultset adapter accordingly.
Anyway I am closing this now.
-
reporter Sorry I didn't get back to you sooner, been slammed.
Requiring a developer to add several lines of code to each Firedac connection he creates seems like a less than optimal solution imo. The second solution I provided is only two lines of code and fixes the issue for every firedac connection created for use with Marshmallow. Additionally the use of Double instead of extended could be used for the Frac portion of the if statement but the int64 portion would definately loose precision if double were used.
Here is the code that would have to be added to each connection. And this still does not guarantee that Firedac will not still return dtBCD sometimes if one of these map rules does not cover a situation.
with FFDConnection.FormatOptions do begin OwnMapRules := True; with MapRules.Add do begin ScaleMin := 0; ScaleMax := 0; PrecMin := 0; PrecMax := 4; SourceDataType := dtBcd; TargetDataType := dtInt16; end; with MapRules.Add do begin ScaleMin := 0; ScaleMax := 0; PrecMin := 5; PrecMax := 8; SourceDataType := dtBcd; TargetDataType := dtInt32; end; with MapRules.Add do begin ScaleMin := 0; ScaleMax := 0; PrecMin := 9; PrecMax := 19; SourceDataType := dtBcd; TargetDataType := dtInt64; end; with MapRules.Add do begin ScaleMin := 1; ScaleMax := 20; PrecMin := 0; PrecMax := 15; SourceDataType := dtBcd; TargetDataType := dtDouble; end; with MapRules.Add do begin SourceDataType := dtDateTimeStamp; TargetDataType := dtDateTime; end; end;
In a perfect world I think we would get Embarcadero to fix this, but in this world we live in if I had a vote I would say this issue should be fixed in the code we have control over for now at least. Thanks for your consideration. BTW the simple fix I provided has been in our copy of the code and working great ever since I submitted this.
- Log in to comment