Message Queues

Asynchronous vs. Synchronous Messaging

Asynchronous messaging refers to a communication pattern where a sender and a receiver do not need to be active or available at the same time. In contrast, synchronous messaging requires both the sender and receiver to be available at the same time for a communication to take place.

The use of message queues in asynchronous messaging can provide several benefits over synchronous communication:

  1. Scalability: Asynchronous messaging allows for communication between loosely coupled systems, which can be scaled independently, leading to better overall system scalability.
  2. Resilience: By decoupling systems, message queues provide a buffer between the sender and receiver, allowing the receiver to process messages at its own pace. This can improve system resilience, as it reduces the risk of cascading failures.
  3. Improved performance: Asynchronous messaging allows for non-blocking communication, enabling the sender to continue processing requests while waiting for a response. This can lead to improved system performance and responsiveness.
  4. Increased flexibility: Message queues allow for the creation of flexible and dynamic communication patterns, where messages can be routed to multiple recipients, transformed, and processed in parallel.

Overall, the use of asynchronous messaging and message queues can provide greater flexibility, scalability, and resilience compared to synchronous messaging.

GLU Message Queue Connectors

GLU.Ware supports integration to both ActiveMQ (Produce and Consume) and Rabbit Message Queues (Produce). Often useful for Asynchronous offloading of a task in a flow, MQ’s can help to speed up transaction round-trip times.


The Client DevOps team will be responsible for the MQ setup and configuration as for any other End-Point that the GLU.Engine Connects to as such configuration of the MQ’s is not covered here.


Based on the setup of the MQ Broker, the GLU Connector can be configured to support the appropriate Protocol e.g. TCP or AMQP.


The GLU.Engine will contain all the libraries to facilitate the connections. So as an example for RabbitMQ all the client components are in the GLU.Engine no additional software is required.

Publish/Subscribe Integration Pattern

ActiveMQ and RabbitMQ are two popular message brokers that support the Microservice Publish/Subscribe integration pattern using topics for consumption by subscribers. GLU plans to expand its support for Pub/Sub pattern topics by adding compatibility with additional messaging systems such as Amazon SQS, IBM MQ, RocketMQ, Google Pub/Sub, Apache Pulsar, and Apache Kafka. Please log any requirements for GLU.Ware to priorise support for any of these via the GLU.Support desk.


The Publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorise published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are.

This pattern is commonly used in messaging systems as part of larger message-oriented middleware systems. Many messaging systems, including Java Message Service (JMS), support both the Pub/Sub and message queue models.


In the Publish-subscribe model, subscribers typically receive only a subset of the total messages published. The process of selecting messages for reception and processing is called filtering. In a topic-based system, messages are published to “topics” or named logical channels. Subscribers in a topic-based system will receive all messages published to the topics to which they subscribe. The publisher is responsible for defining the topics to which subscribers can subscribe.

RabbitMQ

RabbitMQ can be configured to either produce a RabbitMQ transaction on a Queue or consume from a RabbitMQ Queue. If a consumer is to be used then this will need to have a connector set up which has OUTBOUND/INBOUND set to BOTH.

Both consumer and producers can be configured so that the settings for the rabbitMQ can be adjusted from the application.yml file.

Connector Setup

Set Protocol to RABBIT_QUEUE. In the Connector Environment Settings panel, set the Username and Password for access to the RabbitMQ:

Producer

When RabbitMQ is set up in a Connector, it will send transactions of the RabbitMQ as a producer.

Using application.yml file

To have the configuration setup so it is possible to configure the RabbitMQ from the application.yml file, you will need to tick the “Use Queue Holder” checkbox:

When this is ticked the default queue holder value needs to be setup. This value will be the initial value which is held in the application.yml file and used for the GLU.Engine interactions with RabbitMQ.


The parameters in the default queue string should use “&amp ;” separator values.

When not using application.yml

To have the configuration setup so it is not possible to configure the RabbitMQ from the application.yml file, you will need to untick the “Use Queue Holder” button.


When this is unticked the Queue field should contain the rabbitMQ queue which the transaction will go to. The Options field should contain the string of characters which define the settings for the transaction going to RabbitMQ.


The parameters in the default queue string should use “&amp ;” separator values.

Consumer

When RabbitMQ is set up as a connector it will receive the incoming transactions.


When the GLU.Engine is run then then this API will listen on the Queue from RabbitMQ and if there is transactions in the Queue it will pull them in and start the orchestration process.

Using the application.yml file

To have the configuration setup so it is possible to configure the RabbitMQ from the application.yml file, you will need to tick the “Use Queue Holder” button.


When this is ticked the default queue holder value needs to be setup. This value will be the initial value which is held in the application.yml file and used for the integrations interactions with RabbitMQ.


The parameters in the default queue string should use “&amp ;” separator values.

When not using application.yml

To have the configuration setup so it is not possible to configure the RabbitMQ from the application.yml file, you will need to untick the “Use Queue Holder” button.


When this is unticked the Queue field should contain the rabbitMQ queue which the transaction will go to . The options field should contain the string of characters which define the settings for the transaction going to RabbitMQ.


The parameters in the default queue string should use “&amp ;” separator values:

Build Manager Errors

The Build Manager provides an additional layer or ‘error’ assistance. At build time the Build Manager will present a pop-up with the list of ‘Warnings’. See screenshot below.

The ‘Go to Configuration’ button will take the User directly to the relevant Integration to enable the User to address the Warnings prior to re-attempting the ‘Build’ process.

There are times when some warnings may not be of material concern to the User. For example if the warnings relate to a transaction that is not the primary focus at that point in time or it may still be under configuration … whereas other transactions in the Build are not throwing any warnings. In this case the option to ‘Proceed’ can be used to create the GLU.Engine Build not-withstanding the ‘Warnings’. In this way the User can proceed with testing the ‘working’ transactions within the Build but those associated with the ‘accepted’ “Warnings’ will likely fail.

It must also be noted that some warnings relating to Environments, Connectors, Auth Collections or Static / Reusable Parameter Collections may impact all transactions within a GLU.Engine. Some of these warnings will prevent a build from occurring. As such proceeding with building a GLU.Engine that is flagging ‘warnings’ is generally not advised as there is no certainty that any transactions will actually work.

In other words – if there are ‘Warnings’ – ‘Proceed’ with your Build with caution.

GLU.Console Configuration Errors

Integration Builder Validations

Throughout the GLU.Console – validation logic has been added to enable configuration errors to be ‘flagged’ to the user. These ‘flags’ are reflected as ‘Warnings’, a summary of all ‘Warnings’ can be seen by using the ‘Warnings’ button within the tool options at the top of the Configuration screens. See example in the screenshot below:


Warnings are generated in near real-time i.e. as configurations are added, warnings will pop-up to guide the user. In the example below, the Parameter Name is flagged as ‘mandatory’ due to the * … the warning shows up to prompt the user to add the mandatory field:

Build Manager Validations

Where errors that have not been addressed in the Configuration will be highlighted and listed prior to allowing a ‘Build’ to proceed. Additionally, any errors during the actual ‘Build’ process will be displayed after a build fails (these errors will be visible in the ‘Build History’ – see sample below – for later reference).

GLU.Console Sessions and Timeouts


Session timeouts occur when a user remains inactive on the GLU.Console for a specified interval. The timeout duration varies depending on the tool being used: 30 minutes for Integration Builder and USSD Menu Builder tools, and 2 minutes for all other tools. When a session timeout occurs, the user’s session status is changed to ‘invalid’ on the server side, and the web server is instructed to terminate it, resulting in the deletion of all unsaved data.

To alert users before their session expires, a pop-up notification with a countdown timer (20 seconds) appears as the inactivity time approaches the timeout interval. Users have the option to stay connected, refreshing their session, or to log out. If there is no response from the user, the session is terminated, and they are automatically logged out.

Note: Any configuration changes not saved will be lost. It is important to regularly save configuration changes that you want to keep to avoid potentially losing configuration work that you may be busy with.

If you encounter an error on the login page stating “User is currently logged in” when trying to log back in after a session timeout, you can resolve this by forcing your user to be logged out using the following URL: https://www.console.glu.global/glu-console/logout.


Orchestration Response Handlers

Orchestration Response Handlers enable business rules / logic to be applied to the flow and based on those rules the transaction flow can take different paths or actions can be triggered in the flows to ultimately conclude with the ‘Response’ to the Initiating System.

How do response handlers work?

Handlers can be added to any Inbound API Request or Response or any downstream system Response.

  • A Handler on an Inbound API Request could for example trigger the routing of the request to another Inbound API on the same GLU.Engine.
  • A Handler on an Inbound API Response could trigger the flow to be re-routed
  • A Handler on an API response could based on a parameter matching a certain value overwrite the parameter with another value.

A Handler on the Response to an outbound Connector call is the most widely used mechanism for orchestrating your flows.

Within the Orchestration Manager, Handlers can be configured based on response Content, HTTP Status Codes or response conditions. When the handler conditions specified in the handler are met, the defined Handler Action will be triggered. The appropriate configuration box is displayed depending on the action specified.

The Response Handler can be used to end a transaction or result in a different outcome or Response to the Initiating System.

Handler Conditions

Handlers can be configured to run based on:

  • Parameter values
  • HTTP Status Codes
  • Expressions – used to trigger the processing of a Expression in a flow
  • Or ‘none’ – if you just want to trigger a routing event at a point in the flow use the ‘None’ category for you Handler.

Important notes on handlers: 

  • Handlers are always executed in order first to last, so ensure your logic handlers are ordered correctly.
  • The value in the condition cannot contain a function or formula. If required, use a Derived Parameter to apply a function or formula, then reference the Derived Parameter in the Handler Value.
  • Note that if for example you have a set of 3 status codes all of which should result in the same handler condition being applied, you can list all three codes in the ‘Value’ field e.g. 1000, 1001, 1002

Parameter Handler

If the Parameter specified via the ‘Name’ dropdown box passes the comparison check (or other boolean operation) specified by the ‘Condition’ dropdown box, it’ll trigger the ‘Action’.

HTTP Status Codes

The GLU Engine needs to specifically be told to handle unsuccessful HTTP status codes (e.g. 400, 401, 500 etc). This is done by adding a status codes handler. Without this handler, the GLU Engine is unable to unmarshall the response payload. Ensure that you create the required HTTP status code handlers in order to orchestrate or unmarshall.

None Handler

A ‘none’ handler means that no triggering event, code or parameter value needs to be present but that the handler will ALWAYS be executed.

Handler Action Types

Route To

Routes the request to a specific flow code.

Send to Many Routes


Routes all requests to the specified flow codes.

  • Parallel Processing: If enabled then sending messages to the recipients occurs concurrently. Note the caller thread will still wait until all messages have been fully processed before it continues. It’s only the sending and processing the replies from the recipients which happens concurrently.
  • Stop On Exception: If checked Requests will fail if there’s an exception on any of the flow codes. In all situations, the recipient list will stop further processing. This is the same behaviour as in the pipeline, which is used by the routing engine. The default behaviour is to not stop but continue processing till the end.
  • Timeout: Sets a total timeout specified in ,milliseconds, when using parallel processing. If the Recipient List hasn’t been able to send and process all replies within the given timeframe, then the timeout triggers and the Recipient List breaks out and continues.

Stop Process

This forces the process to stop and the message gets processed back through the Response Manager. Either the Stop Process can trigger a ‘successful’ response back, alternatively, it can trigger an error template. The error response is comprised on an ‘Error Code’ and an optional ‘Error Message’ describing the error in detail. The error code and error response values will be mapped into the parameter values are that are specified as the response parameters as below.

Overwrite Parameter

‘Overwrite Parameter’ Action will always take place at the step in the flow at which the Handler resides BEFORE any other Handler rules are applied such as a ‘ROUTE TO’ redirection handler. This means that even if the sequence of Handlers has ROUTE TO handlers pre-ceding ‘OVERWRITE’ handlers, the ‘OVERWRITE’ handlers will always be run and only thereafter will the ‘ROUTE TO’ handlers be run – in the order in which they are placed in the list of Handlers (top-down).

The parameter can be overwritten with a value or with other parameter, below some examples:

  • Overwrite parameter with a value, e.g. text: HelloWorld or Int: 123 ..etc
  • Overwrite parameter with GLU static parameter, e.g. ${GLUstaticParam}
  • Overwrite parameter with other parameter, e.g. ${otherParam}
  • Overwrite parameter with derived parameter,e.g. ${derivedParam}

Save in the Cache

If the tick box is selected, then the data value of the Parameter will be stored in memory for the duration of the time the GLU.Engine is running. It’s important to note, if the GLU.Engine is restarted then the data will be lost. This can be used in conjunction with the “Run on startup” feature to populate data to be used for all API/Transactions in the GLU.Engine.


Data which is saved in the cache will be available to all APIs in the GLU.Engine.

To reference cached values, use the following GLU Reserved Words in a Derived Parameter:

Connector Retry function

There are different ways to trigger a retry. A retry can be triggered on a timeout as well as on a certain response received from a third party.

Retry on Timeout

Step 1: Set the timeout value

On the outbound request as shown below, the timeout value needs to be set in milliseconds.


Step 2: Set the Handler rules

On the Outbound Response a Handler can be added to determine the retry rules. When the timeout is triggered, this Handler will facilitate the retry:

  • The name of the retry Handler.
  • Select the ‘none’ option in the Response Handler Control Panel.
  • Select the ‘Retry on timeout’ Action in the dropdown.
  • Enter the Retry Count: The number of times the retry will attempt to send the transaction
  • Enter the Retry Delay: Each retry will be sent with a delay of x milliseconds.

Step 3: Response after retry attempts

During the retry process, the GLU.Engine will either receive a response from the third party, or it will complete the retry attempts and then send back a timeout response. The timeout response that is sent back will be as per the configuration in the response control panel. The Timeout Response Code and Timeout Response Message are configured as below.



Retry on a specific response parameter value

Step 1: Set the handler rules

An outbound response handler can be used to determine the retry rules for a transaction. When a particular condition is met, this handler will facilitate the retry.

Once the retry handler is processed, the rest of the handlers in the orchestration will be processed (i.e. Retry handlers are first to be processed.)

Note: the condition’s value can be one value or multi values separated by a comma e.g. ValueOne,ValueTwo,ValueThree….. etc



Route to a new flow code on Timeout

One is able to have two orchestrations setup for timeout such that you first ‘Retry’ upon TIMEOUT and thereafter you can ‘Route to’ upon TIMEOUT. To route to a different Flow code upon a timeout, configure the Handler as outlined below.

Step 1: Set the timeout value

On the Outbound Request as below, the timeout value needs to be set in milliseconds.


Step 2: Set the handler rules

  • The name of the retry handler
  • Select the ‘none’ option in the response handler control panel
  • Select the “Route to on timeout” Action in the dropdown.
  • Flow Code: Select the flow code of the transaction that you want to route to when a timeout is triggered from within that transaction.

Retry delivery counter variable

The GLU_REDELIVERY_COUNTER is a variable that increments with each retry attempt sent by GLU. It provides a count of the number of iterations that the retry process completes. This needs to be set up in the derived parameter, see example below:

Handling the Receiving System Response

Handling the response from the receiving system involves two main steps:

  1. Setting up the response parameters needed
  2. Then processing any necessary logic on the response

To streamline the process of setting up response parameters, especially when dealing with different connector types, you can utilise the Generate Parameters from Template tool. This tool allows you to input a response payload sample, which it then uses to auto-generate the entire response payload along with any associated object or collection path mappings. This means that the tool creates the full parameter set and unmarshalling paths automatically, unless you specify otherwise.


Once the response parameters are set up, you can create response handlers to perform specific logic on the received response from the downstream system. These response handlers enable you to customise how the response is processed and what actions are taken based on its content.

Orchestration Manager Overview


In the Integration Builder, navigate to the Orchestration Manager section highlighted in green within the Transaction pattern. Here, you define the downstream steps for each transaction being configured, allowing you to interact with multiple receiving systems as needed.

Each step in the flow to a downstream system is defined by the Request that the GLU.Engine will send and the Response it will receive via the pre-configured Receiving System Connector.


Each step has several possible configuration fields:

Flow Code

  • Used by Handlers to redirect the transaction flow to specific Flow Codes.
  • This allows for non-sequential orchestration flow, independent of the sequence in which transaction steps with receiving systems are defined in the Console.
  • Example: ‘db1’ is assigned as the Flow Code.

Exclude from Build

  • This enables exclusion of the connection defined by this Flow Code from the build. It’s important to note that if this option is selected you won’t be able to route to Flow Code via this build.
  • It’s useful when building a GLU.Engine and the connector is not needed in the current version.
  • If ticked, the connector will not be included in the build or GLU billing.

Async

  • If selected, the route will be executed asynchronously alongside other routes.
  • By default, all routes are performed synchronously.
  • Note: The flow between routes can be altered using Handlers, but routes marked as ‘Async’ will be invoked regardless of Handlers.

Process

  • Marking an orchestration as ‘Process’, indicates that it will solely handle logic and routing via Handlers.
  • There is no outbound request and response available on a Process orchestration.

Is StreamCall

This process is in Beta development with GLU.Ware team. Further descriptions will follow in later releases of GLU.Ware, however if you are interested in using streaming connectors please speak to GLU support.

Connector

  • Refers to possible pre-defined Connectors for the Receiving System.

These configuration fields provide flexibility and control over the orchestration flow, allowing you to tailor the behaviour of each transaction steps to meet your specific integration requirements.

Content Types

Content Type refers to the format and structure of the data being sent or received as part of a message, while connector types and protocols determine how the message is transmitted from one system to another.

The relationship between content types and connector types/protocols comes into play when sending or receiving messages. For example, if you want to send a message containing JSON data over HTTP, you would need to use an HTTP connector and set the content type of the message to application/json so that the receiving system knows how to interpret the data.



GLU connectors support various content types for messages, which can be specified using the Content-Type selector. The following are the content types that GLU supports:

  • text/plain
  • application/xml
  • application/json
  • application/octet-stream
  • multipart/form-data
  • application/x-www-form-urlencoded
  • application/soap+xml
  • application/atom+xml
  • application/rss+xml
  • application/pdf
  • text/xml; charset=UTF-8
  • application/xml; charset=UTF-8
  • application/jose

text/plain

This is used for plain text messages like emails or SMS.

application/xml

This is used for XML documents used in data exchange. It can be specified as application/xml or text/xml. In GLU, you can use either of these to specify the content type of XML documents being sent as part of a message to an external system or API. The application/xml type is more commonly used, and it indicates that the content of the message is an XML document. The text/xml type is also valid, but it is less commonly used and has the same semantics as application/xml.

application/json

This is used for JSON documents, commonly used for data exchange.

application/octet-stream

This is used for binary data like images or audio files.

multipart/form-data

This is used for submitting data including binary data, like file uploads.

application/x-www-form-urlencoded

This is used for submitting form data encoded as key-value pairs, similar to how HTML form submissions are handled. It’s important to note that parameters included in this content type cannot contain special characters. In order to encode these, you can apply the following function to the parameters before sending them on an outbound request. For more information see, URL_ENCODER.

application/soap+xml

This is for SOAP messages, commonly used in web services communication.

application/atom+xml

This for Atom feeds used to syndicate web content.

application/rss+xml

This is for RSS feeds used to syndicate web content.

application/pdf

This is used for PDF documents that is a popular file format used for document exchange, widely supported by various software applications and platforms. In GLU, you can use the application/pdf content type to send PDF documents as part of a message to an external system or API.

text/xml; charset=UTF-8

This means that the message contains XML documents encoded in the UTF-8 character set. When we use the text/xml media type, it’s a way of saying that the XML document is treated as plain text. We specify the character encoding of the document using the charset parameter. However, it’s more common to use the application/xml content type to indicate an XML document, and you can still specify the character encoding using the charset parameter. So, application/xml; charset=UTF-8 is another way of indicating that the message contains an XML document encoded in UTF-8.

application/xml; charset=UTF-8

Is a valid content type that indicates that the content of the message is an XML document encoded in the UTF-8 character set. The application/xml media type is used to indicate that the content of the message is an XML document, and the charset parameter is used to specify the character encoding of the document.

application/jose

application/jose is a recognized content type designed for handling encrypted and/or signed JSON data. JOSE (JSON Object Signing and Encryption) comprises a set of standards defining methods to secure data using JSON format. It facilitates digital signing, encryption, and authentication of JSON data, enhancing security for inter-system and inter-service communications. In GLU, application/jose serves as the content type for transmitting and receiving JOSE-formatted messages. GLU effectively manages JSON Web Tokens (JWTs) and JSON Web Encryption (JWE) messages. It supports tasks like JWT creation, verification, and decoding, as well as encryption and decryption of JWE messages. When dealing with signed messages, the “alg” parameter specifies the signing algorithm, while for encrypted messages, the “enc” parameter indicates the encryption algorithm in use.

It’s important to note that there are several other media types associated with JOSE, including application/jwt for JSON Web Tokens, application/jwe for JSON Web Encryption, and application/jws for JSON Web Signatures. These media types are utilised to denote specific types of JOSE messages and may include additional parameters for specifying algorithms and other metadata.






Fill the form and we’ll contact you shortly

    I agree with

    cookies
    We uses cookies to make your experience on this website better. Learn more
    Accept cookies