Serving many JMS Connections

Introduction

SwiftMQ can handle many concurrent JMS connections. However, to handle a very large set of connections such as 10000 or more, some configuration changes are required.

The following example assumes to connect 5000 concurrent JMS connections to a single SwiftMQ Universal Router.

Recommended SwiftMQ Releases

SwiftMQ 7.1.0 can handle thousands of concurrent JMS connections with low cpu overhead due to an optimization fix in the Timer Swiftlet. So it is recommended to use at least 7.1.0 or later.

SwiftMQ 7.5.0 is able to limit the consumer cache size by size in kb via connection factory attribute "smqp-consumer-cache-size-kb". It further binds network output buffers via ThreadLocals to the writer threads and dramically reduces buffer space. So it is recommended to use 7.5.0 or later.

Network NIO Swiftlet of SwiftMQ 8.1.1 dramatically reduced router input buffers from 128 KB per connection to nothing.

Install the Network NIO Swiftlet

The Network Swiftlet in SwiftMQ Universal Router uses blocking I/O. That is, for each connection there is a separate reader thread, waiting on the connection's input stream. It is "blocked" which explains the term "blocking I/O". 5000 connections would spawn 5000 threads. While this is certainly possible, it is a waste of resources. It is highly recommended to install the Network NIO Swiftlet. It uses only 2 threads in an event-based nonbocking I/O manner to serve all 5000 connections.

JVM Heap Settings

Increase the heap size of the router to -Xmx1024M.

JMS Swiftlet Configuration

JMS Listener: Router-Input-/-Output-Buffer

The SwiftMQ Universal Router creates an input and output buffer for every connection. The size of the buffers is specified via attributes "router-input-buffer-size" and "router-output-buffer-size" of the corresponding JMS listener. The default size is 128KB for each so each connection will acquire 256KB of network buffers per default. While that is reasonable in terms of performance, it is too much for 5000 concurrent connections and, hence, must be reduced. We suggest to set it to the minimum value which is 4192 bytes starting with SwiftMQ 7.1.0. Prior releases have a minimum of 65535 bytes. However, the size depends on the message size, see next section.

Note: SwiftMQ 7.5.0 binds network output buffers via ThreadLocals to the threads of pool "jms.connection". So the maximum number of router output buffers are equal to the maximum number of threads in pool "jms.connection".

Example:

        <listener name="plainsocket" port="4001" router-input-buffer-size="4096" router-input-extend-size="8192"
                                                 router-output-buffer-size="4096" router-output-extend-size="8192"">
          ...
        </listener>

Connection Factory: Client-Input-/-Output-Buffer, Consumer Cache Size, Producer Reply Interval

The client-side network buffers are specified via attributes "client-input-buffer-size" and "client-output-buffer-size" of the corresponding connection factory that is used from the clients. It should be set to the same value as the "router-input-buffer-size" and "router-output-buffer-size".

JMS clients are using prefetching internally. Messages are delivered into a client-side cache and consumed from there. The cache size (number of messages) is also the maximum size of a bulk request used from the SwiftMQ Universal Router to send multiple request in one go. The consumer cache size therefore depends on the value of the "router-output-buffer-size" and the maximum size of a message.

        ConsumerCacheSize = RouterOutputBufferSize / MaxMessageSize

In our example we assume a maximum message size of 1024 bytes. Therefore, the consumer cache size is 4.

Note: Introduced in SwiftMQ 7.5.0 there is a new connection factory attribute "smqp-consumer-cache-size-kb" which specifies the size of a consumer cache in KB. This attribute has been introduced to avoid out of memory errors when transfering large messages. The default value is 2048 (2 MB). So whenever "smqp-consumer-cache-size" or "smqp-consumer-cache-size-kb" is reached, the router waits until the client has consumed the cache.

The same applies to the sending side when using non-persistent messages. A producer sends asynchronously but waits in intervals to react on flow control delays. Due to the asynchronous sending, the SwiftMQ client part is able to put multiple messages into a bulk request and sends it in one go to the router. The interval in which a producer waits for flow control is computed in a similar way but now from attribute "router-input-buffer-size" (assuming 1KB message size):

        ProducerReplyInterval = RouterInputBufferSize / MaxMessageSize

Example:

      <connection-factories>
        <connection-factory name="plainsocket@router1"
                            client-input-buffer-size="4096"
                            client-input-extend-size="8192"
                            client-output-buffer-size="4096"
                            client-output-extend-size="8192"
                            smqp-consumer-cache-size="4"
                            smqp-producer-reply-interval="4"/>
      </connection-factories>

FAQ

I have multiple consumers per connection. How do I compute the buffer sizes?

Each consumer has its own cache. Therefore, the minimum "router-output-buffer-size" and "client-input-buffer-size" is:

        MinSize = NumberConsumers * CacheSize * MaxMessageSize

I have multiple producers (non-persistent) per connection. How do I compute the buffer sizes?

A producer which sends non-persistent messages works asynchronously so the minimum "router-input-buffer-size" and "client-output-buffer-size" is:

        MinSize = NumberProducers * ProducerReplyInterval * MaxMessageSize

How do I compute the buffer sizes for persistent producers?

These producers are working synchronously:

        MinSize = NumberProducers * MaxMessageSize

How do I compute the buffer sizes for producers with transacted sessions?

It depends on the size of the transactions:

        MinSize = NumberProducers * MaxMessagesPerTransaction * MaxMessageSize