Problem with BatchTmLeveragingStep its not working with Microsoft Translator for xliff with more than 30 text units M20

Original issue 330 created by anoopsing... on 2013-04-26T06:49:31.000Z:

What steps will reproduce the problem?
1.Create a pipe line with BatchTmLeveragingStep
2.Give an XLIFF with 40 Textunits to translate
3.This will just generate the xliff for last 10 text units with a XLIFFWriter step as next step.

What is the expected output? What do you see instead?
This should give an xliff with All the the text units translated.

What version of the product are you using? On what operating system?
M20 on RHEL 5.4

Please provide any additional information below.
I tried debugging the code looks like we are returning noop even after every batch which is not getting processed properly while end batch is returning the multievent.
The issue is with following function
from line no. 82-119
***********************

@ Override
public Event handleEvent(Event event) {
switch (event.getEventType()) {
case TEXT_UNIT:
ITextUnit tu = event.getTextUnit();
batchedEvents.add(event);

                    if (!canLeverageTu(tu)) {  
                            return Event.NOOP\_EVENT;  
                    }

                    handleTextUnit(event);  
                    break;  
            case START\_BATCH\_ITEM:  
                    event = handleStartBatchItem(event);  
                    return event;  
            case END\_BATCH\_ITEM:  
                    event = handleEndBatchItem(event);  
                    return event;  
            case START\_BATCH:  
                    event = handleStartBatch(event);  
                    return event;  
            case END\_BATCH:  
                    event = handleEndBatch(event);  
                    return event;  
            case END\_DOCUMENT:  
                    event = handleEndDocument(event);  
                    return event;  
            case START\_DOCUMENT:  
                    event = handleEndDocument(event);  
                    return event;  
            default:  
                    batchedEvents.add(event);  
                    break;  
            }  
            return Event.NOOP\_EVENT;  
    }

************************
In the above function we are not returning the EventType.MULTI_EVENT that is returned by handleTextUnit() method. so the even that gets returned is Event.NOOP_EVENT
which does not let next step to process this event properly and XLIFF misses to generate the proper translation.
On the other hand if you see method handleEndDocument
*****************************************
@ Override
protected Event handleEndDocument(Event event) {
tuEventCount = 0;

            // leverage any remaining batched TextUnits for this document  
            if (!batchedEvents.isEmpty()) {  
                    batchLeverage();  
                    MultiEvent me = new MultiEvent();  
                    for (Event e : batchedEvents) {  
                            me.addEvent(e);  
                    }  
                    batchedEvents.clear();

                    // add END DOCUMENT event  
                    me.addEvent(event);  
                    return new Event(EventType.MULTI\_EVENT, me);  
            }

            return event;  
    }

*****************************************
It returns the proper event and thats why last batch gets processed properly.

I made a fix to return the multievent in my local code and the same seems to be working fine with my flow
Here is the change I made
*************************************
@ Override
public Event handleEvent(Event event) {
switch (event.getEventType()) {
case TEXT_UNIT:
ITextUnit tu = event.getTextUnit();
batchedEvents.add(event);

            if (!canLeverageTu(tu)) {  
                return Event.NOOP\_EVENT;  
            }

            event = handleTextUnit(event);  
            return event;  
        case START\_BATCH\_ITEM:  
            event = handleStartBatchItem(event);  
            return event;  
        case END\_BATCH\_ITEM:  
            event = handleEndBatchItem(event);  
            return event;  
        case START\_BATCH:  
            event = handleStartBatch(event);  
            return event;  
        case END\_BATCH:  
            event = handleEndBatch(event);  
            return event;  
        case END\_DOCUMENT:  
            event = handleEndDocument(event);  
            return event;  
        case START\_DOCUMENT:  
            event = handleEndDocument(event);  
            return event;  
        default:  
            batchedEvents.add(event);  
            break;  
    }  
    return Event.NOOP\_EVENT;  
}

*************************************

Please have a look at this and if you see this as real issue can you please let me know if the solution I am taking is good or will there be any issue with the same.
Thanks,
Anoop

Comments (2)