web forms processing
I have been experimenting with the web forms functionality. I find that if I have wait for return value enabled, that submitted forms go into the processing queue and have to take their turn to be processed. On a busy server, that can take minutes to be processed. Is there no priority that can be set for web form processing? Having a web form submission timeout is unacceptable behavior and makes this function only usable when you don't need any type of response back from the ThinkAuto server. Any improvements you can make here or any suggestions?
- 8 replies
- SStephen Parker @stephenparker
We will look into this. Currently the Message Processor service starts several processor tasks - and messages are given to each task on a round robin basis (meaning separate automations can have their messages processed concurrently).
We could possibly have separate tasks for messages originating from web forms - so these would get processed independently. Although there still may be a delay depending on what the web form automations are set up to do and how many forms come in.
If you have long running Automations (eg uploading files to FTP server etc) what you can do is use the 'Call' action WITHOUT the wait for response option - to send off the long running part to a different Automation. In your main automation you can then return a form response.
In my case here, I'm submitting some data and returning a dynamic response (another populated web form) based on the submitted data. You may not have designed this to work like a web server, but I think if there were tasks assigned to web forms as you've described that would work very well for us.
- In reply tostephenparker⬆:
I agree it would be good if messages originating from web forms could be processed separately. The webform feature is very helpful but it does slow down when the queue is busy.
Are you using the local http API interface or the public API? The public API should be able to handle 10-20 form submissions per second. We are not currently seeing any high volume which would require queuing on the public API. The local API would be higher (100-200) assuming its being called locally.
This would be on a simple automation. Obviously if the Automation is complex and time consuming and the web form is set to wait for the Automation response then the response time would depend on the Automation.
I am using the local API. When I have another slightly long running automation with messages in the queue, webforms (or any of my other API endpoints) do not process until after the queue (on the other long running automation) is complete.
I have tried to make sure the webform and API are setup to process Concurrent while leaving this unchecked for the other long running automation but still Webforms and APIs do not process until this queue is empty.
It would be very helpful to set priority or be able to use the webforms/API endpoints immediately.
See here: http://g.recordit.co/xpajENbzK1.gif
Since I submitted this, I've found that the performance has improved and my web form will process while other automations are still queued up. Was there any improvement made since March? Matty, you made need more processors for your machine, we've got 4 now and this seems to be handling the load ok.
Interesting, nice that it is working for you. I can't seem to make webforms or api's return data while other automations are queued up, as soon as the queue is at zero the Api/webform returns the data.
The latest release (890) now allows you to increase the number of tasks that the message processor service starts (Server Settings - Message Processor - Message Processor Tasks). It defaults to 4 but can be increased. Each Automation is assigned to one of the tasks. So if this is set to 4 then 4 separate Automations will execute messages concurrently. See: https://ta5.blob.core.windows.net/ta5help/ThinkAutomation.html#concurrent-automation-processing
Increasing the tasks may provider faster responses to webform Automations where there are multiple webforms calling different Automations. If however your Automation is long running and the 'Wait For & Include Automation Return Value In The Confirmation Message' is enabled then the webform may timeout before the Automation completes. One way around this is to disable the 'Wait For' option (webforms will then respond as soon as the message is queued). You can then send the return value to the user later via email using the %Msg_ResultsUrl% variable. See: https://ta5.blob.core.windows.net/ta5help/ThinkAutomation.html#providing-a-link-to-the-automation-results-1