BLOG > Programming > Asynchronous Callback and Recursion - Solving Multiple Web Requests Problem

Asynchronous Callback and Recursion - Solving Multiple Web Requests Problem

Posted on March 15, 2012 02:22 pm | by Krissada Dechokul
Asynchronous Callback and Recursion - Solving Multiple Web Requests Problem

There are times when we have to write a program to request a lot of web-services simultaneously. Multiple of those web-service calls usually ended up “timed out”. That’s because whenever any web-service call is pending, a timer is set to that connection for some period of time before the timer expires or “timed out” due to receiving no replies from the servers. This is the default behavior of any web-browser to deal with the server crashes problem or the network connectivity issue, so that the client can take recovery action rather than infinitely waiting for the server.

To write a program to request multiple of web-services, normally, you would have to write a loop to call each web-service individually. Well, there would not be a problem if each of the web-service call is simple and does not take too much time to process. But what if the web-service call is very complex and takes some times to process the call? This would results in the problem I already mentioned at the beginning, some of the web-services might end up “timed out” because the server cannot response to all requests in a timely manner.

The Concept

Multiple rest service requests timed-out
The concept of the solution that solves this problem actually sound very simple, process each web-service call one by one. But the implementation of this solution is a bit complicated because each web-service call is processed in an “asynchronous” manner meaning that, for each call, the function will not wait until the call receives the callback’s results but will rather move on to the next call immediately. Moreover, we will never know when the server will finish processing the call. So, some techniques should have been brought to solve this “asynchronous” manner.

So you simply cannot write something like the following…

function callMultipleServices(){
     // looping through each web-service call
     for(var i=0;i<web_services.length;i++){
          // calling each web-service individually
          callWebService(web_services[ i ], onFinishCallback);
function onFinishCallback(resultset){
     // processing each result set..
     // ..

After working with the problem for some times, I have come up with a pretty neat solution to deal with this kind of problem. By using a combination of programming techniques called asynchronous callback and recursion. With the combination of the techniques, the concept to solve this problem is turned into, process each web-service call one by one, one call should return the results with some information to keep track of the progress, and after getting the result, determine the next call and move on until all the web-service calls are processed. Let’s see the implementation of the solution.

The Implementation

To illustrate the implementation of the solution, I will use the dojotoolkit, a quite power javascript framework like jQuery, to outline the structure of the solution and will make a use of the special dojotoolkit’s command called dojo.partial. This command is very useful that it enables us to attach pieces of information that will be passed back as arguments with the callback’s result set. This is exactly what we want as we have been discussing so far. With this command, we can attach the number of currently processed web-services and the total number of services to be processed with the callback’s result to keep track of the recursion progress. With the attached information, we can determine which services to be called next so that we can move on correctly.

So, instead of writing a simple loop, we will write a recursive function like the following…


// In this case, I use is_new_call variable as a flag indicating the new set of web-services
// And the current variable and the total variable to determine the next call
function recursivelyCallMultipleServices(is_new_call, current, total){
          // initializing variables for doing recursion
          is_new_call = false;
          current = 0;
          total = web_services.length;

          // calling the first web-service, specify callback function back into this function again (we are doing recursion remember?) 
          // and attaching the current variable and the total variable with the callback's results
          recursivelyCallMultipleServices(web_services[current], dojo.partial(recursivelyCallMultipleServices, is_new_call, current, total));
          // processing each result… here you might save the result to some variables…
          // ...

          // moving on 

          // keeping track of the progress
          if(current == total){ // all the services have been called?
               // end of recursion
               // calling the next web-service
               recursivelyCallMultipleServices(web_services[current], dojo.partial(recursivelyCallMultipleServices, is_new_call, current, total));

function finishRecursion(){
     // doing whatever you want with all the results…
     // …

The Conclusion

I tested this combination of techniques by making a hundred of web-service calls and managed to run my application for like 10-20 minutes without having a single web-service call “timed out” because we were calling each web-service one by one and only one afternanother. So I find this technique pretty neat and should be very useful for various kind of situations where we have to call a bunch of web-services to achieve our goals. Hope you can apply this technique to your application as well.

Related Posts

blog comments powered by Disqus