No response provided by Talend job

问题: I am designing a job in Talend Open Studio for ESB. My goal is to create a job for processing files with data. This job is used for web service, which is deployed on a serv...

问题:

I am designing a job in Talend Open Studio for ESB. My goal is to create a job for processing files with data. This job is used for web service, which is deployed on a server. This web service is a SOAP type and it consists with such components:

  • tESBProviderRequest,
  • tESBProviderResponse,
  • process job.

Everything works fine, until processing files with large amount of data. It takes some time and in the end it throws a runtime exception with message: "no response provided by Talend job"

And btw. everything else works fine in the job - the whole file's processing. But at the end that exception is thrown.

I can see the code generated in Java (unfortunately, cannot edit it). And I have found this code snippet:

/**
 * message exchange controller
 */
public class QueuedMessageHandlerImpl<IN, OUT> implements
        ESBProviderCallback {
    private final int MAX_QUEUE_SIZE = 1000;

    private final int WAIT_TIMEOUT_SECONDS = 120;

    private final java.util.concurrent.BlockingQueue<QueuedExchangeContextImpl<IN, OUT>> queue = new java.util.concurrent.LinkedBlockingQueue<QueuedExchangeContextImpl<IN, OUT>>(
            MAX_QUEUE_SIZE);

Which obviously says, I have only 120 seconds to process. How can I change it? Is there any workaround?


回答1:

It seems to not be done, and it cannot be done by any configuration. I use Talend ESB version 6.5.1. After migration to 7.0.1 version, this tESBProviderRequest got options in Advanced section: https://help.talend.com/reader/6pO1HSkFat02LpDF_xCmsg/XXo1~pvJC7rmkmJ0J_E0fg

So I set up new values for times and it worked.


回答2:

You may want to have a rethink of your design - using a synchronous (blocking) web service to process a file and having the client wait until processing is complete has issues when your processing time increases, as you have already seen.

Have you thought about implementing an asynchronous design? This would be where the client sends a request to a web service whose only task is to accept requests and place these requests onto a messaging queue, thus freeing up the client to do other things. At the very least, you decouple the request from the processing and avoid the issue of timeouts when the file is large. Note, that HTTP at some level also has a session timeout built into it, so it's best to avoid the issue altogether by using a different design.

One last thing, I highly recommend you try to migrate away from job usage when using Talend ESB - you should really check out Routes, which are used to generate Apache Camel code. They're much more powerful than jobs for your use case and can be coupled with ActiveMQ (which is bundled with the Studio).

Talend ESB Studio comes with some demos that you can import and check out and I'd highly recommend you start there.

  • 发表于 2019-02-14 23:14
  • 阅读 ( 236 )
  • 分类:网络文章

条评论

请先 登录 后评论
不写代码的码农
小编

篇文章

作家榜 »

  1. 小编 文章
返回顶部
部分文章转自于网络,若有侵权请联系我们删除