Filetransfer over JMS (since 9.4.0)

SwiftMQ 9.4.0 introduces Filetransfer over JMS and the new File Cache Swiftlet. Both enables users to transfer files of arbritary size over existing JMS connections in a very fast and reliable way. It should be used rather than to transfer large JMS messages.

Please see File Cache Swiftlet for more infos.

Transfering very large Messages

Introduction

SwiftMQ can handle the transfer of very large messages (10 MB and more) out of the box with its default configuration, however, the default configuration is not optimized for it. To achieve maximum throughput with very large messages, some configuration changes are necessary.

JMS Swiftlet Configuration

Creating a dedicated JMS Listener for the Producers

A dedicated JMS listener should be created for the producers, because the network buffer setting differ from consumers.

Example:

        <listener name="bigmsg-producer" port="7001" router-input-buffer-size="11534336">
          ...
        </listener>

This creates a JMS listener on port 7001 and creates a network input buffer on the router side of 11 MB (10 MB message body plus some overhead).

Creating a Connection Factory for the Producers

Create a connection factory below the producer JMS listener which is used from the producers:

Example:

      <connection-factories>
        <connection-factory name="bigmsg-producer@router1"
                            client-output-buffer-size="11534336"
                            smqp-producer-reply-interval="1"/>
      </connection-factories>

This creates a connection factory which sets the client network output buffer to 11 MB and the SMQP producer reply interval to 1 message. The latter ensures that the client acts on flow control on every message when using non-persistent messages. This attribute isn't required for persistent messages.

Creating a dedicated JMS Listener for the Consumers

A dedicated JMS listener should be created for the consumers, because the network buffer setting differ from producers.

Example:

        <listener name="bigmsg-consumer" port="7002" router-output-buffer-size="11534336">
          ...
        </listener>

This creates a JMS listener on port 7002 and creates a network output buffer on the router side of 11 MB (10 MB message body plus some overhead).

Creating a Connection Factory for the Consumers

Create a connection factory below the consumer JMS listener which is used from the consumers:

Example:

      <connection-factories>
        <connection-factory name="bigmsg-consumer@router1"
                            client-input-buffer-size="11534336"
                            smqp-consumer-cache-size="1"/>
      </connection-factories>

This creates a connection factory which sets the client network input buffer to 11 MB and the SMQP consumer cache size to 1 message. The latter ensures that the client receives messages one by one.

Note: Introduced in SwiftMQ 7.5.0 there is a new attribute "smqp-consumer-cache-size-kb" which specifies the size of a consumer cache in KB. This attribute has been introduced to avoid out of memory errors when transfering large messages. The default value is 2048 (2 MB). So whenever "smqp-consumer-cache-size" or "smqp-consumer-cache-size-kb" is reached, the router waits until the client has consumed the cache. However, the router delivers at least one message in case the message size is greater than the "smqp-consumer-cache-size-kb".

Complete JMS Swiftlet Configuration

The complete configuration of the JMS Swiftlet is therefore:

Example:

      <swiftlet name="sys$jms">
        <listeners>
          <listener name="bigmsg-producer" port="7001" router-input-buffer-size="11534336">
            <connection-factories>
              <connection-factory name="bigmsg-producer@router1"
                                  client-output-buffer-size="11534336"
                                  smqp-producer-reply-interval="1"/>
            </connection-factories>
            <host-access-list/>
          </listener>
          <listener name="bigmsg-consumer" port="7002" router-output-buffer-size="11534336">
            <connection-factories>
              <connection-factory name="bigmsg-consumer@router1"
                                  client-input-buffer-size="11534336"
                                  smqp-consumer-cache-size="1"/>
            </connection-factories>
            <host-access-list/>
          </listener>
        </listeners>
      </swiftlet>

Producers perform a JNDI lookup on "bigmsg-producer@router1" and consumer on "bigmsg-consumer@router1", respectively.

Queue Manager Configuration

When tranfering large messages, the flow control is an important instrument to avoid overloading the router with large messages. Dependend on which message model is used, point-to-point or pub/sub, different attributes have to be adjusted. Further, the queue cache size has to be adjusted to limit the number of messages held in memory.

Point-to-Point

Adjusting the flow control behavior is done via the attribute "flowcontrol-start-queuesize". The cache size is defined via the attribute "cache-size". Both attributes are defined per queue:

Example:

      <queue name="testqueue" cache-size="5" flowcontrol-start-queuesize="4"/>

Sets the cache size of queue "testqueue" to 5 messages and generates flow control delays on that queue if the queue size is at least 4.

Pub/Sub and Routing

When using pub/sub or routing, the default attributes for "flowcontrol-start-queuesize" and "cache-size" are used. These are defined in the "swiftlet" element itself:

Example:

      <swiftlet name="sys$queuemanager" cache-size="5" flowcontrol-start-queuesize="4">
        ...
      </swiftlet>

These values are used for all pub/sub, temporary and system (routing) queues.

Topic Manager Configuration

When using pub/sub, the pub/sub flow control has to be enabled. Because this is the default setting, you don't have to change anything.

Store Swiftlet Configuration

When using persistent messages, the Store Swiftlet's cache and checkpoint sizes have to be adjusted.

Cache Size

The cache size determines how many pages are held in memory. Say, you use a single queue with a queue cache size of 5 messages and each message is 11 MB, you'd need 55 MB of cache. Divide it by 2 KB per page and you get 28160 pages. Plus some overhead for index pages, you'd need to have a cache of 30000 pages:

Example:

      <cache min-size="30000" max-size="30000"/>

Transaction Log Checkpoint Size

The checkpoint size of the transaction log determines the interval in which checkpoints are made. A checkpoint synchronizes the transaction log with the database (page.db) and flushes all dirty cache pages. This takes time with a large cache so you should set the checkpoint size as high as possible. The following example sets it to 500 MB.

Example:

      <transaction-log checkpoint-size="524288000" path="../../store/router1/log"/>

Threadpool Swiftlet Configuration

Extend the number of threads for the "jms.connection" and "jms.session" pool to ensure that every client has always a free thread. The best way is to set the "max-threads" attribute to the number of clients you have:

Example:

      <pool name="jms.connection" kernel-pool="true" max-threads="5" min-threads="1">
        <threads>
          <thread name="sys$jms.connection.service"/>
        </threads>
      </pool>
      <pool name="jms.session" kernel-pool="true" max-threads="5" min-threads="1">
        <threads>
          <thread name="sys$jms.session.service"/>
        </threads>
      </pool>

Routing Swiftlet Configuration

When transfering large messages over a routing connection, the attributes "inbound-transaction-size" and "inbound-window-size" of the resp. routing listener/connector needs to be set to 1 to ensure that only a single message is put into an XA transaction and that the previous XA transaction has been fully committed before the next XA transaction will be send.

Sender Router

A sender router sends large messages to a receiver router, thus it requires an appropriate network output buffer. Following 2 examples of a routing listener and a routing connector of a sender router. One of them needs to be defined, dependent on which router connects and which router listens.

A routing listener:

Example:

        <listener name="plainsocket"
                  port="5100"
                  router-output-buffer-size="11534336">
          <host-access-list/>
        </listener>

A routing connector:

Example:

        <connector name="router1"
                   hostname="localhost"
                   port="5100"
                   router-output-buffer-size="11534336"/>

Receiver Router

A receiver router receives large messages from a sender router, thus it requires an appropriate network input buffer plus a value of 1 for the attributes "inbound-transaction-size" and "inbound-window-size".

A routing listener:

Example:

        <listener name="plainsocket" port="5100"
                  router-input-buffer-size="11534336"
                  inbound-transaction-size="1"
                  inbound-window-size="1">
          <host-access-list/>
        </listener>

A routing connector:

Example:

        <connector name="router1"
                   hostname="localhost"
                   port="5100"
                   inbound-transaction-size="1"
                   inbound-window-size="1">
                   router-input-buffer-size="11534336"/>

Sender/Receiver Router

A sender/receiver router sends and receives large messages on both directions, thus it requires an appropriate network input and output buffer plus a value of 1 for the attributes "inbound-transaction-size" and "inbound-window-size".

A routing listener:

Example:

        <listener name="plainsocket" port="5100"
                  router-input-buffer-size="11534336"
                  router-output-buffer-size="11534336"
                  inbound-transaction-size="1"
                  inbound-window-size="1">
          <host-access-list/>
        </listener>

A routing connector:

Example:

        <connector name="router1"
                   hostname="localhost"
                   port="5100"
                   inbound-transaction-size="1"
                   inbound-window-size="1">
                   router-input-buffer-size="11534336"
                   router-output-buffer-size="11534336"/>

Flow Control

During routing, messages are produced to queues during inbound processing. On each XA commit, a flow control delay is returned. An attribute "inbound-flow-control-enabled" can be set to true or false. Setting to true, the flow control is respected and further XA commits are delayed, depending on the flow control delays. These delays are cascaded through the whole router network back to the producer client if each hop on the route has "inbound-flow-control-enabled" set to true. If it is set to false (default), flow control delays are not respected and XA commits are not delayed. So in the worst case, a router is flooded with messages.

For the transfer of large messages you should enable flow control at the receiver router:

Example:

        <swiftlet name="sys$routing" inbound-flow-control-enabled="true">
          ...
        </swiftlet>