Hiraash's Blog

One of the well known issues of BlazeDS is it’s limitation in scaling. To be more specific, if you are using BlazeDS for HTTP steaming using AMF, the number of concurrent streaming connections will be limited to 100s. The exact number would be dependent on multiple factors such as the capacity of the server and configuration settings on Tomcat or whatever the application server you are using. When it comes to streaming each connection would be opened and held until the client closes or the connection times out. In the case of BlazeDS this directly relates to the number of threads consumed due to how Servlets operate. Servlet consumes a thread per request and holds the thread until the request is committed. During streaming, a request is kept open throughout the streaming session which makes the servlet hold the thread through out. This particular problem cannot be solved with servlet 2.5 or before due to the limitation in Servlets itself. However Servlet 3.0 supports asynchronous requests which can be utilized efficiently to scale the same servers to support 1000s of concurrent connections.

Up to BlazeDS 4 there has not been support for endpoints that utilizes servlet 3.0. BlazeDS 4.0 turnkey ships with Tomcat 6 which uses servlet 2.5. There has been rumors that BlazeDS 4.5 will have endpoints that utilize the capability of Servlet 3.0. Still this is just a rumor and cannot be depended on. Given the current state of affairs and the growth of other frameworks and technologies I would not expect much. There have been solutions for the problem that uses various technologies and my picks are below.

• MuleRTMP – A Google code project that uses the capabilities of Red5 streaming server and streams to AMF clients over RTMP.
• A polling solution that uses Jetty 7 Continuations to provide an endpoint for BlazeDS
• The Farata Systems’ solution which uses again Jetty 7 and Servelet 3 to provide a streaming endpoint for BlazeDS.
• blazeds-servlet3-support – Another Google Code project that is a simple BlazeDS streaming endpoint that utilizes Servlet 3.

I have been personally working on the blazeds-servlet3-support project by contributing code. Right now the code looks quite messy but it works. I had to fix some of the issues and currently it is able to reliably stream messages on Tomcat 7 without causing any trouble. What this does is simply utilize the Async capability of Servlet 3 to isolate all the pushing functions into one single thread. Requests are handled and then immediately the Context of the request is handed to a the pushing thread, enabling the thread that handled the request to complete and return to the pool. This approach pretty much would allow your same server that was limited to 100s of connections to boost up to 1000s of connections. This is however not tested yet and will be done once I have completed work on the endpoint. I will run some load tests and post it up. If you wish to give this a test run here is what you need to do.

1. Install Tomcat 7 and Blaze DS 4. The turnkey install of BlazeDS 4 is not going to work since it ships with Tomcat 6.
2. Once you have Tomcat and BlazeDS working fine, change the Tomcat Connector configuration to use the NIO connector instead of the BIO conntector. The setting is in the server.xmland should be changed as follows.

   <!--<Connector port="8400" protocol="HTTP/1.1"
    connectionTimeout="20000" redirectPort="8443"/>-->
   <Connector connectionTimeout="20000" port="8400" 
    protocol="org.apache.coyote.http11.Http11NioProtocol" redirectPort="8443"/>

3. In your Tomcat BlazeDS application context change the Servlet configuration to allow asynchronous capability for the MessageBrokerServlet. This should be done in the web.xml as follows.

   <servlet>
     <servlet-name>MessageBrokerServlet</servlet-name>
     <display-name>MessageBrokerServlet</display-name>
     <servlet-class>flex.messaging.MessageBrokerServlet</servlet-class>
     <init-param>
       <param-name>services.configuration.file</param-name>
       <param-value>/WEB-INF/flex/services-config.xml</param-value>
     </init-param>
     <load-on-startup>1</load-on-startup>
     <async-supported>true</async-supported>
   </servlet>
   <filter>
     <filter-name>springSecurityFilterChain</filter-name>
     <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
     <async-supported>true</async-supported>
   </filter>

4. Copy the compiled blazeds-servlet3-support jar file to the WEB-INF/lib folder of the context.
5. Change the WEB-INF/flex/service-config endpoint as follows.

   <channel-definition id="amf-stream" class="mx.messaging.channels.StreamingAMFChannel">
     <endpoint url="http://{server.name}:{server.port}/{context.root}/messagebroker/streamingamf" 
        class="org.iobit.servlet3.endpoint.AMFServlet3Endpoint"/>
     <properties>
       <idle-timeout-minutes>3</idle-timeout-minutes>
       <max-streaming-clients>10000</max-streaming-clients>
       <server-to-client-heartbeat-millis>500</server-to-client-heartbeat-millis>
     </properties>
   </channel-definition>

6. Restart Tomcat and you should be good to go!

Since I am continuing to work on this I would very much appreciate any feedback you might have. Please let me know your thoughts in the comments.