tag:blogger.com,1999:blog-80637976701782734922024-03-12T21:41:56.513-07:00Enterprise Application and Data IntegrationUnknownnoreply@blogger.comBlogger2125tag:blogger.com,1999:blog-8063797670178273492.post-70180930152269118512009-10-07T10:20:00.000-07:002009-10-07T11:03:32.577-07:00Java NIO woes : Poll Set based Selector Provider shipped with IBM AIX JVM J9<span class="Apple-style-span" style=" ;font-family:'Times New Roman';font-size:medium;"><pre style="text-align: justify;word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style="white-space: pre;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size: small;"> </span></span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size: small;">While testing our server on AIX, we ran into multiple nio related issues that serverely affected the functionality of our product. Since our server worked fine on other platforms (win 32, win 64, Solaris sparc, HP - UX), we suspected that this would be an issue with the default selector implementation that shipped with the IBM JDK. One point to note that the issues aggravated when we moved from and earlier JDK version to IBM J9 Version 1.6 SR5.</span></span></pre><pre style="text-align: justify;word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="white-space: pre;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size: small;"> </span></span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size: small;">To be specific, the issues that we encountered were frequent hangs in Selector.select() and SocketChannel.close() operations. The Stack Traces encountered were as follows </span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"> </span></span></pre><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>Thread A<br />FileDispatcher.preClose0(FileDescriptor) line: not available [native method]<br />SocketDispatcher.preClose(FileDescriptor) line: 53<br />SocketChannelImpl.implCloseSelectableChannel() line: 693<br />SocketChannelImpl(AbstractSelectableChannel).implCloseChannel() line: 212<br />SocketChannelImpl(AbstractInterruptibleChannel).close() line: 97<br /><br />Thread B : Selecting Thread (number 1 ):<br />PollsetArrayWrapper.pollsetBulkCtl(int, long, int) line: not available [native method]<br />PollsetArrayWrapper.poll(long) line: 228<br />PollsetSelectorImpl.doSelect(long) line: 62<br />PollsetSelectorImpl(SelectorImpl).lockAndDoSelect(long) line: 69<br />PollsetSelectorImpl(SelectorImpl).select(long) line: 80<br /><br />Thread C : Selecting Thread ( number 2) :<br />PollsetArrayWrapper.pollsetPoll(int, long, int, long) line: not available [native method]<br />PollsetArrayWrapper.poll(long) line: 232<br />PollsetSelectorImpl.doSelect(long) line: 62<br />PollsetSelectorImpl(SelectorImpl).lockAndDoSelect(long) line: 69<br />PollsetSelectorImpl(SelectorImpl).select(long) line: 80<br /><br /></code></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size: small;"> Other Server threads waiting to serve/complete serving requests.</span></span></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style=" ;font-size:16px;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:medium;"><span class="Apple-style-span" style="font-size: small;">What I discovered was that the default selector provider in 1.6 SR5 had been changed to PollSetSelectorImpl which provides a selector based on the pollset interface. This Poll Set selector provider is an enhancement over the poll() based selector provider that was the default selector provider on AIX J9 JDK pre 1.6 SR5. Pre 1.6 SR5, the default selector provider was PollSelectorProvider and the Selector Implementation was PollSelectorImpl. The implementation of this Selector Provider is based on the POSIX poll(...).</span></span></span></span></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span"><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="font-size: small;"> </span></span><span class="Apple-style-span" style="font-size: small;">One point to be noted is that the default SelectorProviders shipped with JDK implemented by Sun are poll() based (and epoll() based Linux Kernel Version 2.6 onwards). </span></span></span></span></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span"><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="font-size: small;"> </span></span><span class="Apple-style-span" style="font-size: small;">In IBM JDK 1.6 SR5, the default selector provider is the pollset provider (PollsetSelectorProvider). The SelectorProvider.openSelector() opens a pollset selector if it detects that the OS supports the pollset interface. This selector implementation is efficient than the default poll() based Selector Provider that comes along with pre 1.6 SR5 versions.</span></span></span></span></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="font-size: small;"> </span></span><span class="Apple-style-span" style="font-size: small;">The optimizations in the Poll Set based Selector Provider are to register only the file descriptors for whom the operations are newly registered to the kernel poll set and the poll cache mechanism implemented at the kernel level. The performance improvement of the pollset selector over the poll based selector in terms of number of requests served per second is around 13.6 % which does not seem significant. The details of the Poll Set based Selector Provider can be found </span><a href="http://www.ibm.com/developerworks/aix/library/au-pollset/index.html"><span class="Apple-style-span" style="font-size: small;">here</span></a><span class="Apple-style-span" style="font-size: small;">.</span></span></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="font-size: small;"> </span></span><span class="Apple-style-span" style="font-size: small;">In conclusion, I would be going with the poll based SelectorProvider "sun.nio.ch.PollSelectorProvider" for our server on AIX platform till we have a definitive answer for the issues that we are facing with the poll set based Selector Provider : "sun.nio.ch.PollsetSelectorProvider". To do this, we start the server JVM with the property</span> </span></span></pre><pre style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; color: #000000; background-color: #eee;font-size: 12px;border: 1px dashed #999999;line-height: 14px;padding: 5px; overflow: auto; width: 100%"><code>-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider<br /></code></pre><pre style="word-wrap: break-word; white-space: pre-wrap; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size: small;">Do check this space for updates.</span></span></pre></span>Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-8063797670178273492.post-20695195951307606512009-09-22T11:31:00.000-07:002009-09-22T20:18:54.186-07:00Leveraging the Pentaho Data Integration Carte Server for EAI<div style="text-align: justify;"><span class="Apple-style-span" style=" ;font-family:'trebuchet ms';"><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="font-family:Georgia;color:#0000EE;"> </span></span>The Carte Server that comes bundled with Pentaho Data Integration (PDI) provides a way for external applications to interface with the Penta<span class="Apple-style-span" style="font-size:small;">ho Transformations and Jobs. The operations exposed by Carte, though limited, can be leveraged to integrate Pentaho Data Integration capabilities into your Business Processes. This can be done using the HTTP endpoints provided by BPEL engines like </span><a href="http://www.intalio.org/confluence/display/PXE/Endpoint+References"><span class="Apple-style-span" style="font-size:small;">Intalio</span></a><span class="Apple-style-span" style="font-size:small;">, IBM BPEL4J and Oracle BPEL Process Manager. You may use this for Enterprise Application Integration, to implement Event Driven Workflows or to monitor your Data Integration Jobs and Tranformations with your SOA monitoring service.</span></span></div><p class="MsoNormal" style="text-align: justify;text-indent: 0.5in; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Carte consists a set of servlets that run in the lightweight servlet container </span><a href="http://www.mortbay.org/jetty/"><span class="Apple-style-span" style="font-size:small;">jetty</span></a><span class="Apple-style-span" style="font-size:small;">. A basic web based console exposes the operations provided by Carte. These can be accessed by </span><a href="http://wiki.pentaho.com/display/EAI/Carte+User+Documentation"><span class="Apple-style-span" style="font-size:small;">starting the Carte Server</span></a><span class="Apple-style-span" style="font-size:small;"> and pointing your browser to http://</span><carte host=""><span class="Apple-style-span" style="font-size:small;">:<</span></carte></span><st1:place st="on"><st1:placename st="on"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Carte</span></span></st1:placename><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span><st1:placetype st="on"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Port</span></span></st1:placetype></st1:place><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">>/kettle/status/. These are the same operations that are used by the PDI ETL Engine to execute clustered transformations.</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">The operations that are exposed by the Carte web server are listed below. This list has been compiled by inspecting / debugging the Pentaho Code and executing the requests. References to the Pentaho Java Classes have been added for those who wish to explore further.</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">[pdi code reference : org.pentaho.di.www.WebServer]</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">For the purpose of this article, we have four transformations and one job that are configured to be executed remotely of a carte server running at </span></span><a href="http://dataalp.com:8083/"><span class="Apple-style-span" style="font-family:'trebuchet ms';color:#000000;"><span class="Apple-style-span" style="text-decoration: none;"><span class="Apple-style-span" style="font-size:small;">http://dataalp.com:8083</span></span></span></a></p><div style="text-align: justify;"><br /></div><div style="text-align: justify;"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUkR8L5PG1hq4Cw5tYqmVFbvCd9LFPzENOum9fr0Cs7sxrdVFad2TgVpMQ4hg7uq8cfaOvZ95dErR0DRtc8Wj2LwdOb9FXV3LHCluCC_TlyRujmc-XoQ535VKbHVXttVifQ6wk4dMPEiAu/s1600-h/KettleScreen.jpg"><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjUkR8L5PG1hq4Cw5tYqmVFbvCd9LFPzENOum9fr0Cs7sxrdVFad2TgVpMQ4hg7uq8cfaOvZ95dErR0DRtc8Wj2LwdOb9FXV3LHCluCC_TlyRujmc-XoQ535VKbHVXttVifQ6wk4dMPEiAu/s400/KettleScreen.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5384368122323379666" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: auto; text-align: center; cursor: pointer; width: 400px; height: 123px; " /></a></div> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">The following are the requests that Carte web server handles</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Note : All the requests take a request parameter : xml. This parameter, when set to ‘Y’ returns the response in an xml format. This response format is particularly useful in EAI implementations.</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">1. Get the status of the Carte Server</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: left;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><i><span class="Apple-style-span" style="text-decoration: underline;">Request parameters :</span></i></span></span></p><p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p><p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><i><span class="Apple-style-span" style="text-decoration: underline;">Sample URL</span><span class="Apple-style-span" style="font-style: normal; "><span class="Apple-style-span" style="text-decoration: underline;"> :</span> http://dataalp.com:8083/kettle/status?xml=Y</span></i></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><i><span class="Apple-style-span" style="text-decoration: underline;">Response :</span></i></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">The status of all the transformations and jobs registered to run on this Carte server.</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">The response contains the following information for each transformation registered to run on the Carte server</span></span></p><pre name="code" class="Cpp"><br /><transstatus><br /><transname>Row generator test</transname><br /><status_desc>Running</status_desc><br /><error_desc/><br /><paused>N</paused><br /><stepstatuslist><br /></stepstatuslist><br /><logging_string><![CDATA[]]></logging_string><br /></transstatus></pre> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">[pdi code reference : GetStatusServlet]</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">2. Get the Status of a Transformation</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><i><span class="Apple-style-span" style="text-decoration: underline;">Request parameters : </span></i></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">name = </span><transformation name=""><span class="Apple-style-span" style="font-size:small;"> (mandatory)</span></transformation></span></p><p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';">xml=Y (optional)</span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Sample URL :</i></span> http://dataalp.com:8083/kettle/transStatus/?name=Process+Customer+Addition</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Response :</i></span> The Status of the transformation which includes the status of each individual steps of the transformation, number of lines output, read, written, rejected, deleted or updated, exit status, timestamp of the run, etc.</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">[pdi code reference : GetTransStatusServlet]</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">3. Prepare a Transformation for execution</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-tab-span" style="white-space:pre"> </span>This operation prepares a transformation for execution. Use this if the initialization of the transformation takes a long time. After prepare operation completes, the state of the transformation is WAITING. The transformation starts execution when the startExec operation is requested.</span></span></p> <p class="MsoNormal"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><i><span class="Apple-style-span" style="text-decoration: underline;">Request Parameters :</span></i></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">name = </span><transformation name=""><span class="Apple-style-span" style="font-size:small;"> (mandatory)</span></transformation></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Sample URL :</i></span> http://dataalp.com:8083/kettle/prepareExec?name=Process+Customer+Addition</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Response:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Status indicating if the Transformation Prepare was successful</span></span></p> <pre name="code" class="Cpp"><br /><webresult><br /><result>OK</result><br /><message/><br /></webresult></pre><p class="MsoNormal"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span><message><span class="Apple-style-span" style="font-size:small;"> </span></message></span></p> <p class="MsoNormal"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"></span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">[pdi code reference : PrepareExecutionTransServlet]</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">4. Start the execution of a transformation.</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;text-indent: 0.5in; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">This operation executes a transformation. This operation must be called after prepareExec is called else it results in an error. This operation is not exposed directly from the Carte Console. To execute a transformation, use startTrans.</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Request Parameters:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">name = </span><transformation name=""><span class="Apple-style-span" style="font-size:small;"> (mandatory)</span></transformation></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Sample URL:</i></span> http://dataalp.com:8083/kettle/startExec?name=Row+generator+test&xml=Y</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Response:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Status indicating if the transformation was successfully started</span></span></p><pre name="code" class="Cpp"><br /><webresult><br /><result>OK</result><br /><message/><br /></webresult><br /></pre><p class="MsoNormal" style="text-align: left;"><span class="Apple-style-span" style=" ;font-family:'trebuchet ms';">[pdi code reference : StartExecutionTransServlet]</span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">5. Start a Transformation</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-tab-span" style="white-space:pre"> </span>It is important to note that only those transformations that are configured to run remotely on this Carte server can be executed. If Carte cannot find the transformation name in the list of registered transformations an error is thrown. Note that this operation is a sequential execution of prepareExec and startExec.</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Request Parameters:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">name = </span><transformation name=""><span class="Apple-style-span" style="font-size:small;"> (mandatory)</span></transformation></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Sample URL</i></span>: http://dataalp.com:8083/kettle/startTrans?name=Row+generator+test</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Response:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Status indicating if the transformation was successfully started</span></span></p><pre name="code" class="Cpp"><br /><webresult><br /> <result>OK</result><br /> <message>Transformation [Process Customer Addition] was started.</message><br /></webresult></pre><p class="MsoNormal"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"></span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span style=" background-image: initial; background-repeat: initial; background-attachment: initial; -webkit-background-clip: initial; -webkit-background-origin: initial; background- background-position: initial initial; color:silver;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="color: rgb(0, 0, 0); ">[pdi code reference : StartTransServlet]</span></span></span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">6. Pause or Resume a transformation</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-tab-span" style="white-space:pre"> </span>You can pause a running transformation or resume a paused transformation with this operation. This operation toggles a running transformation between paused and running states. If the transformation is in the paused state, it starts running. If the transformation is running, it is paused</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Request Parameters:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">name = </span><transformation name=""><span class="Apple-style-span" style="font-size:small;"> (mandatory)</span></transformation></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Sample URL:</i></span> http://dataalp.com:8083/kettle/pauseTrans?name=Row+generator+test</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Response:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Status indicating if the transformation was successfully paused / resumed.</span></span></p><pre name="code" class="Cpp"><webresult><br /><result>OK</result><br /><message>Transformation [Row generator test] : pause requested.</message><br /></webresult><br /></pre><pre name="code" class="Cpp"><webresult><br /><result>OK</result><br /><message>Transformation [Row generator test] : resume requested.</message><br /></webresult></pre> <p class="MsoNormal"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"></span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">7. Stop a transformation</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-tab-span" style="white-space:pre"> </span>This operation can be used to stop a transformation</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Request Parameters:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">name = </span><transformation name=""><span class="Apple-style-span" style="font-size:small;"> (mandatory)</span></transformation></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Sample URL:</i></span> http://dataalp.com:8083/kettle/pauseTrans?name=Row+generator+test</span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Response:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">Status indicating if the transformation was successfully stopped</span></span></p><pre name="code" class="Cpp"><webresult><br /><result>OK</result><br /><message>Transformation [Row generator test] stop requested.</message><br /></webresult></pre> <p class="MsoNormal"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"></span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><b><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">8. Add a transformation</span></span><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><o:p></o:p></span></span></b></p> <p class="MsoNormal" style="text-align: justify;text-indent: 0.5in; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">This operation is used to add a transformation to be managed by and executed by the Carte Server. This is the same operation that Kettle uses when you configure a transformation to be executed remotely on a Carte Server. I would suggest you to run the transformation once on the remote Carte Server so that this transformation is registered with the Carte Server and all the operations can be invoked on this transformation. Note that you would have to repeat this process each time the transformation is changed.</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Request Parameters:</i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">xml=Y (optional)</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="text-decoration: underline;"><i>Request body : </i></span></span></span></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">The metadata of the transformation to be added (registered) in the xml format.</span></span></p> <p class="MsoNormal"><o:p><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> </span></span></o:p></p> <p class="MsoNormal" style="text-align: justify;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;">You can also have a custom transformation deployment application that you can use to deploy your transformations to the Carte Server. <span class="Apple-style-span" style="color:#3366FF;"><i>Trans.</i></span></span></span><span style=" background-image: initial; background-repeat: initial; background-attachment: initial; -webkit-background-clip: initial; -webkit-background-origin: initial; background- background-position: initial initial; "><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"><span class="Apple-style-span" style="color:#3366FF;"><i>sendXMLToSlaveServer(...)</i></span></span></span></span><span style="color:black;"><span class="Apple-style-span" style="font-family:'trebuchet ms';"><span class="Apple-style-span" style="font-size:small;"> can be referred for more details.</span></span></span></p>Unknownnoreply@blogger.com4