Story excluded from IDML file...

[Jim Hargrave (OLD)]

I found a couple of odd things with this file.

1: In this screenshot, the circled text is not visible within the document, but it does appear inside the story XML:

Some sort of transformation appears to have been done on the story’s Text Frame to visually exclude that text.

2: When I unzipped the IDML that’s within the zip in the above comment and found the text that was not in TMS, I found that it was in the story with ID “u9cf”. When I looked for it within the document, I found something odd, which is probably the reason it wasn’t included. Inside the spread XML that contains that story, there are four different TextFrame elements that link to story ID “u9cf”. My hypothesis is that the story ID existing four separate times caused some sort of error in Okapi, so that story was skipped.

 

@557058:79a7f0de-dc04-4a62-ae38-a9fcfa160c84 , in my above comment is a zip that contains the skeleton from the TMS as well as “convertedFile”, which is the IDML created by Adobe’s InDesign server. The concern is that this following section was excluded from translation:

Do you think that the fact that the story is pointed two four different times in the IDML might make Okapi choke on that story? Or do you have any other ideas why it might not be included for translation?

[Jim Hargrave (OLD)]

@Chase Tingley Any thoughts on this one? We have permission to use the IDML file in our public repo. I have added it as 4_story_pointers.idml in our integration tests.

[Chase Tingley]

@Denis Konovalyenko What do you think?

[Denis Konovalyenko]

@Jim Hargrave (OLD) , @Chase Tingley , that was intentionally done so - every invisible pasteboard item is excluded from extraction (please refer to net.sf.okapi.filters.idml.PasteboardItem.VisibilityFilter#filterVisible for more details):

                if (!pasteboardItem.getTextualSpreadItem().isVisible()) {
                    continue;
                }

As for the actual case, there are 5 invisible pasteboard items:

invisiblePasteboardItems = {ArrayList@2491}  size = 5
 0 = {PasteboardItem@2619} 
  textualSpreadItem = {SpreadItem$Rectangle@2624} 
   textPaths = {ArrayList@2626}  size = 0
   id = "u809e"
   layerId = "ub1"
   visible = false
   transformation = "1 0 0 1 507.87119898302797 -89.90778469344266"
   properties = {ArrayList@2630}  size = 7
  anchorPoints = {ArrayList@2625}  size = 4
 1 = {PasteboardItem@2620} 
  textualSpreadItem = {SpreadItem$TextFrame@2711} 
   storyId = "u9cf"
   previousTextFrameId = "n"
   nextTextFrameId = "u900d"
   textPaths = {ArrayList@2715}  size = 0
   id = "u9e1"
   layerId = "ub1"
   visible = false
   transformation = "1 0 0 1 294.3981818181819 -39.15285714285591"
   properties = {ArrayList@2719}  size = 5
  anchorPoints = {ArrayList@2712}  size = 4
 2 = {PasteboardItem@2621} 
  textualSpreadItem = {SpreadItem$TextFrame@2720} 
   storyId = "u9cf"
   previousTextFrameId = "u900d"
   nextTextFrameId = "ufc3e"
   textPaths = {ArrayList@2725}  size = 0
   id = "ufc3a"
   layerId = "ub1"
   visible = false
   transformation = "1 0 0 1 36.5 -323.5"
   properties = {ArrayList@2729}  size = 5
  anchorPoints = {ArrayList@2721}  size = 4
 3 = {PasteboardItem@2622} 
  textualSpreadItem = {SpreadItem$TextFrame@2669} 
   storyId = "u9cf"
   previousTextFrameId = "ufc3a"
   nextTextFrameId = "n"
   textPaths = {ArrayList@2683}  size = 0
   id = "ufc3e"
   layerId = "ub1"
   visible = false
   transformation = "1 0 0 1 36.5 -170.5"
   properties = {ArrayList@2687}  size = 5
  anchorPoints = {ArrayList@2670}  size = 4
 4 = {PasteboardItem@2623} 
  textualSpreadItem = {SpreadItem$Rectangle@2662} 
   textPaths = {ArrayList@2664}  size = 0
   id = "u902c"
   layerId = "ub1"
   visible = false
   transformation = "1.0000000000000002 0 0 1.0000000000000002 453.60983790564313 176.34495711384992"
   properties = {ArrayList@2668}  size = 7
  anchorPoints = {ArrayList@2663}  size = 4

And 3 of them are text frames (u9e1, ufc3a, ufc3e), which refer to the very same story (u9cf). Here is how they look in XML:

<TextFrame Self="u9e1" ParentStory="u9cf" PreviousTextFrame="n" NextTextFrame="u900d" ContentType="TextType" ParentInterfaceChangeCount="138508 2109954642 138558 1277754891" TargetInterfaceChangeCount="" LastUpdatedInterfaceChangeCount="" OverriddenPageItemProps="" HorizontalLayoutConstraints="FlexibleDimension FixedDimension FlexibleDimension" VerticalLayoutConstraints="FlexibleDimension FixedDimension FlexibleDimension" CornerRadius="0" GradientFillStart="-150.39818181818197 -189.64714285714444" GradientFillLength="513" GradientFillAngle="0" GradientStrokeStart="-150.39818181818197 -189.64714285714444" GradientStrokeLength="513" GradientStrokeAngle="0" ItemLayer="ub1" Locked="false" GradientFillHiliteLength="0" GradientFillHiliteAngle="0" GradientStrokeHiliteLength="0" GradientStrokeHiliteAngle="0" AppliedObjectStyle="ObjectStyle/$ID/[None]" Visible="false" Name="$ID/" TopLeftCornerRadius="0" TopRightCornerRadius="0" BottomLeftCornerRadius="0" BottomRightCornerRadius="0" ItemTransform="1 0 0 1 294.3981818181819 -39.15285714285591">

<TextFrame Self="ufc3a" ParentStory="u9cf" PreviousTextFrame="u900d" NextTextFrame="ufc3e" ContentType="TextType" ParentInterfaceChangeCount="" TargetInterfaceChangeCount="" LastUpdatedInterfaceChangeCount="" OverriddenPageItemProps="" HorizontalLayoutConstraints="FlexibleDimension FixedDimension FlexibleDimension" VerticalLayoutConstraints="FlexibleDimension FixedDimension FlexibleDimension" GradientFillStart="0 0" GradientFillLength="0" GradientFillAngle="0" GradientStrokeStart="0 0" GradientStrokeLength="0" GradientStrokeAngle="0" ItemLayer="ub1" Locked="false" GradientFillHiliteLength="0" GradientFillHiliteAngle="0" GradientStrokeHiliteLength="0" GradientStrokeHiliteAngle="0" AppliedObjectStyle="ObjectStyle/$ID/[Normal Text Frame]" Visible="false" Name="$ID/" ItemTransform="1 0 0 1 36.5 -323.5">

<TextFrame Self="ufc3e" ParentStory="u9cf" PreviousTextFrame="ufc3a" NextTextFrame="n" ContentType="TextType" ParentInterfaceChangeCount="" TargetInterfaceChangeCount="" LastUpdatedInterfaceChangeCount="" OverriddenPageItemProps="" HorizontalLayoutConstraints="FlexibleDimension FixedDimension FlexibleDimension" VerticalLayoutConstraints="FlexibleDimension FixedDimension FlexibleDimension" GradientFillStart="0 0" GradientFillLength="0" GradientFillAngle="0" GradientStrokeStart="0 0" GradientStrokeLength="0" GradientStrokeAngle="0" ItemLayer="ub1" Locked="false" GradientFillHiliteLength="0" GradientFillHiliteAngle="0" GradientStrokeHiliteLength="0" GradientStrokeHiliteAngle="0" AppliedObjectStyle="ObjectStyle/$ID/[Normal Text Frame]" Visible="false" Name="$ID/" ItemTransform="1 0 0 1 36.5 -170.5">

The high-level processing logic can be observed in the scope of the net.sf.okapi.filters.idml.Document.Default#open method.

I assume we can add an option to support hidden pasteboard items extraction for this, similar to extractHiddenLayers - extractHiddenPastboardItems with false value by default. Also, I would appreciate it if you could share your thoughts on this.

[Chase Tingley]

@Jim Hargrave (OLD) It sounds from Denis’s analysis that this content is not flagged as visible in the file, so our default behavior is to exclude it. Do you think this warrants a new option, or should fold this + the hidden layers behavior together?

[Jim Hargrave (OLD)]

I think this warrants a separate option. I like having the fine grain control.

Jim Hargrave

[Denis Konovalyenko]

@Jim Hargrave (OLD) , @Chase Tingley , in my opinion, it would always be better to have well-grained capabilities in regard to what kind of hidden content is extracted. Thus, the layers are under control now. And we can add support for all hidden pasteboard items extraction and if there is a demand for even more granular handling, saying by type, it can be added as well.

For instance, in order to extract hidden text frames of a hidden layer, there would be the following options set to true: extractHiddenLayers and extractHiddenPastboardItems.

A related pull request #467 has been opened.

[Denis Konovalyenko]

I think this warrants a separate option. I like having the fine grain control.

@Jim Hargrave (OLD) , sorry, the page content was not updated while my previous message was formed. In any case, thank you for your comment! And I am looking forward to your review then. :)

[Denis Konovalyenko]

The pull request #467 was merged.