esbperformance / talend / README.TXT

Talend ESB SE 5.1.1

Build the OSGi artifacts which contains the proxy services by using maven

    $ cd ~/resources/esbperformance/talend/
	$ mvn install

Edit {TALEND_HOME}/container/bin/setenv and add the following lines at the top
    export JAVA_MIN_MEM=2048M
    export JAVA_MAX_MEM=2048M

Copy the configuration files from configs/etc into {TALEND_HOME}/container/etc
    This configures the thread pools and http connections.

Copy the extra endorsed libs from configs/lib/endorsed into 
    {TALEND_HOME}/container/lib/endorsed
    to work around some bugs found in the JDK libraries. (future versions of TESB
    will have this by default)


Start the Talend ESB

	$ cd {TALEND_HOME}/container
	$ ./bin/trun

You will get the karaf console

	karaf@trun>


PATCH Install:
Some testing has revealed some major issues in various parts of TESB.  These steps should help:
   1) Woodstox - 4.1.2 has a severe performance regression, particulary when used with the many
      threads of small XML (2560/512b for example).  See: https://jira.codehaus.org/browse/WSTX-277
      Downgrading to 4.1.1 helps:
        karaf@trun> uninstall woodstox-core-asl
	karaf@trun> install -s mvn:org.codehaus.woodstox/woodstox-core-asl/4.1.1
   2) CXF has an issue in the many many thread case where requests might be silently dropped.
        karaf@trun> uninstall cxf-api
	karaf@trun> install -s mvn:org.apache.cxf/cxf-api/2.6.2-SNAPSHOT
   3) CXF also has an issue where HTTP conduit settings may not get picked up from OSGi.  It also
      floods the error log with warnings if the workqueue size is exceeded.
        karaf@trun> uninstall cxf-rt-transports-http
	karaf@trun> install -s mvn:org.apache.cxf/cxf-rt-transports-http/2.6.2-SNAPSHOT
	karaf@trun> install -s mvn:org.apache.cxf/cxf-rt-transports-http-jetty/2.6.2-SNAPSHOT
   4) The DOM implementation in the JDK does a System.getProperty in the Document constructor. For
      many small XML's, this severely rate limits it.  The endorsing of xercesImpl mentioned above
      handles this problem.  
   5) There is a strange interaction bug between camel-core(xslt) and camel-cxf that is causing it
      to flip to DOM mode instead of streaming the results of the xlst.  This causes a streaming 
      performance issue, but also synchronized block issues in the DocumentFactories that camel
      is using.  To fix this:
        karaf@trun> uninstall camel
        karaf@trun> install -s mvn:org.apache.camel/camel-core/2.9.3-SNAPSHOT
        karaf@trun> install -s mvn:org.apache.camel/camel-blueprint/2.9.3-SNAPSHOT
        karaf@trun> install -s mvn:org.apache.camel/camel-spring/2.9.3-SNAPSHOT
        karaf@trun> install -s mvn:org.apache.camel/camel-cxf-transport/2.9.3-SNAPSHOT
        karaf@trun> install -s mvn:org.apache.camel/camel-cxf/2.9.3-SNAPSHOT

   After the above, you should shutdown karaf:
        karaf@trun> shutdown
   so you can restart to make sure all the patches are picked up. All the above will be part of the
   next TESB service pack due in a few weeks.



Type following to install and start each proxy services into the tesb

	karaf@trun> install -s mvn:com.talend.tesb.perf/<scenario>/<version>

you will see the bundle id of the scenario from the previous command output as follows;

	Bundle ID: [ID]

Use the display or list commands to see whether the application is successfully launched

	karaf@trun> display
	or
	karaf@trun> list | grep "Talend ESB"

Install the following features to enable the Camel XQuery processing (for the Content Based Routing tests)
        karaf@trun> install -s mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.saxon/9.4.0.1_2
        karaf@trun> install -s mvn:org.apache.camel/camel-saxon/2.9.2

Install the following applications this way, starting each bundle as described above
 	karaf@trun> install -s mvn:com.talend.tesb.perf/proxy/1.0
	karaf@trun> install -s mvn:com.talend.tesb.perf/cbr/1.0
	karaf@trun> install -s mvn:com.talend.tesb.perf/hbr-soap/1.0
	karaf@trun> install -s mvn:com.talend.tesb.perf/hbr-transport/1.0
	karaf@trun> install -s mvn:com.talend.tesb.perf/xslt-transform/1.0
	karaf@trun> install -s mvn:com.talend.tesb.perf/secure-proxy/1.0

e.g.
    karaf@trun> install -s mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.saxon/9.4.0.1_2
    Bundle ID: 166
    karaf@trun> install -s mvn:org.apache.camel/camel-saxon/2.9.2
    Bundle ID: 167
    karaf@trun> install -s mvn:com.talend.tesb.perf/proxy/1.0
    Bundle ID: 168
    karaf@trun> install -s mvn:com.talend.tesb.perf/cbr/1.0
    Bundle ID: 169
    karaf@trun> install -s mvn:com.talend.tesb.perf/hbr-soap/1.0
    Bundle ID: 170
    karaf@trun> install -s mvn:com.talend.tesb.perf/hbr-transport/1.0
    Bundle ID: 171
    karaf@trun> install -s mvn:com.talend.tesb.perf/xslt-transform/1.0
    Bundle ID: 172
    karaf@trun> install -s mvn:com.talend.tesb.perf/secure-proxy/1.0
    Bundle ID: 173
karaf@trun> list | grep "Talend ESB"
[ 168] [Active     ] [            ] [Started] [   80] Talend ESB :: Direct Proxy (1.0.0)
[ 169] [Active     ] [            ] [Started] [   80] Talend ESB :: Content Based Router (1.0.0)
[ 170] [Active     ] [            ] [Started] [   80] Talend ESB :: Header Based Router :: SOAP (1.0.0)
[ 171] [Active     ] [            ] [Started] [   80] Talend ESB :: Header Based Router :: Transport (1.0.0)
[ 172] [Active     ] [            ] [Started] [   80] Talend ESB :: XSLT Transformer (1.0.0)
[ 173] [Active     ] [            ] [Started] [   80] Talend ESB :: Secure Proxy (1.0.0)

Note:
=====
If you start the AMI EC2 image again and then the Talend ESB, sometimes the performance test may not run, as the order of starting of the bundles has an effect. If this happens, stop the bundles, and restart them in the correct order.

    karaf@trun> stop 173
    karaf@trun> stop 172
    karaf@trun> stop 171
    karaf@trun> stop 170
    karaf@trun> stop 169
    karaf@trun> stop 168
    karaf@trun> start 168
    karaf@trun> start 169
    karaf@trun> start 170
    karaf@trun> start 171
    karaf@trun> start 172
    karaf@trun> start 173

Before executing the performance test, re-generate the WS-Security related requests as Talend ESB is particular about the timestamps used
    $ cd client-scripts
    $ ./recreate-secure-requests.sh


Service URLs
============

proxy - a direct proxy service
    url             : http://localhost:9080/service/DirectProxy
    wsdl-url        : http://localhost:9080/service/DirectProxy?wsdl
    karaf-command   : install mvn:com.talend.tesb.perf/proxy/1.0

cbr - a content based router, routing the message by looking at the SOAP body content
    url             : http://localhost:9080/service/CBRProxy
    wsdl-url        : http://localhost:9080/service/CBRProxy?wsdl
    karaf-command   : install mvn:com.talend.tesb.perf/cbr/1.0

hbr-soap - a header based router, routing the messages by looking at the [{http://someuri}routing] SOAP header
    url             : http://localhost:9080/service/CBRSOAPHeaderProxy
    wsdl-url        : http://localhost:9080/service/CBRSOAPHeaderProxy?wsdl
    karaf-command   : install mvn:com.talend.tesb.perf/hbr-soap/1.0

hbr-transport - a header based router, routing the messages by looking at the [routing] HTTP transport header
    url             : http://localhost:9080/service/CBRTransportHeaderProxy
    wsdl-url        : http://localhost:9080/service/CBRTransportHeaderProxy?wsdl
    karaf-command   : install mvn:com.talend.tesb.perf/hbr-transport/1.0

xslt-transform - a transforming proxy which transforms the message to its element character inverse on the in path and
does the back transformation on the out path using XSLT
    url             : http://localhost:9080/service/XSLTProxy
    wsdl-url        : http://localhost:9080/service/XSLTProxy?wsdl
    karaf-command   : install mvn:com.talend.tesb.perf/xslt-transform/1.0

secure-proxy - a security gateway pattern proxy where the proxy created at the talend ESB enforces security with Timestamp,
Encrypted and Signed messages into and back out to the client, while the server to ESB communication is kept in plane SOAP
    url             : http://localhost:9080/service/SecureProxy
    wsdl-url        : http://localhost:9080/service/SecureProxy?wsdl
    karaf-command   : install mvn:com.talend.tesb.perf/secure-proxy/1.0


NOTE:
=====

Only if you get any issues in deploying the child configurations according to the specific README.txt files, do the following and try again

	karaf@trun> features:install camel-http

Exceptions seen on the logs during the load test execution. The Talend team confirmed this was a harmless exception thrown by the Jetty server, which does not affect performance or correct operation, and which would be fixed in a subsequent release

2012-08-04 02:57:51.485:WARN:oeji.nio:
java.lang.IllegalArgumentException: timeout value is negative
	at java.lang.Object.wait(Native Method)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:415)
	at org.eclipse.jetty.http.AbstractGenerator.blockForOutput(AbstractGenerator.java:535)
	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:182)
	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:101)
	at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:51)
	at org.apache.camel.util.IOHelper.copy(IOHelper.java:177)
	at org.apache.camel.util.IOHelper.copy(IOHelper.java:161)
	at org.apache.camel.component.cxf.interceptors.RawMessageContentRedirectInterceptor.handleMessage(RawMessageContentRedirectInterceptor.java:52)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)
	at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:262)
	at org.apache.cxf.phase.PhaseInterceptorChain.resume(PhaseInterceptorChain.java:231)
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:78)
	at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:348)
	at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:312)
	at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:943)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:879)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:188)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
	at org.eclipse.jetty.server.Server.handleAsync(Server.java:390)
	at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:446)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:48)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
	at java.lang.Thread.run(Thread.java:662)
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.