Unicode advanced options will be lost when we do the migration with import and export function

[Ronan Wang]

Hi,

We found an issue when we do the migration with import and export function from the client’s UAT to PROD.

We exported an app with “}APQ.UX.Dim.App.Export“ on UAT.

Then we imported this on PROD.

It succeeds in importing the app, we can see it on PROD.

But it lost the user inserted row’s name in the advanced options.

UAT

PROD

Can you help with this? Currently, we can only change it manually on PROD.
Thanks

[Scott Wiltshire]

Hi @Ronan Wang

This looks like a character encoding issue in the export process when creating the exported CSV of the app. I’m guessing it is becasue there are unicode characters in the Options JSON object. This can probably be fixed locally by

1. Change from AsciiOutput to TextOutput function
2. Using the SetOutputCharacterSet function in the export process prior to the TextOutput function

I will note this as an enhancement to use the TextOutput function in this process in order to support non-ASCII characters. I’m not sure which encoding would be the best one but my guess would be TM1CS_UTF8 [e.g. SetOutputCharacterSet( sFileOut, 'TM1CS_UTF8' ); ]

Unfortunately it’s not a straightforward simple fix as the export process is using }bedrock.cube.data.export so the change to output character set and text output needs to be made there (so this is where you would need a temporary local fix). I just raised ain issue on bedrock issue list https://github.com/cubewise-code/bedrock/issues/159 . If you want to take this up and make this change in bedrock 4 library you are most welcome.

Note you might also need to use the SetInputCharacterSet in the import process.

[Ronan Wang]

Hi @Scott Wiltshire

Thanks for your help. Already fixed it using your solution.
And I also made this change in bedrock 4 libraries https://github.com/cubewise-code/bedrock/pull/161

‌

[Scott Wiltshire]

Awesome. I will mark as resolved as soon as I migrate the updated bedrock export processes into the content store image.

[Scott Wiltshire]

fixed by integrating bedrock 4.1.1

[Scott Wiltshire]

Removing milestone: 2020_Q2 (automated comment)