The Health Checks allow you to configure and manage health checks for various services connected to your GLU.Engine instance. This guide will walk you through the setup process to ensure your services are correctly monitored and health checks are properly configured.

Setting up is complete in 2 parts. Initially for a GLU.Engine the Health Check Manager is used to configure and switch on the type of health check needed. Once this is done the connectors will need to be setup up with the details of the Endpoints which are used to check complete the health checks on.

See below details on these 2 parts

1. Health Check – Manager

Navigate to Health Check Manager

Navigate to Integration Builder

Choose a GLU.Engine and click on the Health Check Manager icon (see heart shaped icon below).

Switching on Health Check

Health checks can be switched on and configured for various states of your service i.e. Live, Ready, Full, and Startup. It also provides the ability to define a new health check state. See below which shows the Health Check Manager with endpoints for keycloak, slack and GLU Inbound and tabs for Live , Ready, Full and Startup.

To configure a ‘Live’ state:

a. Select the State:

• Click on the Live tab to configure the health check for the Live state of your service.

b. Setting Up Health Check Details:

• Name: Assign a meaningful name for your health check. This value will be displayed on the tab. (e.g., Live)

• Context: Define the endpoint context for the health check. This is the path which will be used to call the API (e.g., /health/live)

• Collection Name: Specify a collection name to group related health checks. (e.g., Entries )

• Active: Ensure the checkbox is selected to activate the health check. If the tick box is switched off then the Method will not be available as a method for the GLU.Engine API.

c. Adding Connectors:

• Available Connectors: Select the services you want to include in the health check. In this example, you can see connectors like keycloak, slack and GLU Inbound.

• Enable Connector: Click the checkbox next to the desired connector to include it in the health check. For instance, slack is enabled in this example. A “Error 404 Not Found” will be returned if an API is called for a Connector which is not enabled.

d. Configuring Connector Details:

• Method: Choose the method for the health check. Options are HTTP or Ping. For slack, HTTP is selected. Please not the method will need to be configured in the connector to support the check see Health Check – Configuring a Connector below.

• Group: Specify a group for organizational purposes. (e.g., Slack-Group)

• Tag: Assign a tag to help identify and filter this health check. (e.g., Slack-Tag)

e. Saving the Configuration:

• Click Save to store your health check configuration.

f. Adding Additional Endpoints:

• It is possible to also add more endpoints, click on the Add Endpoint button and add in details as above.

Troubleshooting

• Ensure all required fields are filled out.

• Verify that the connectors are correctly selected and configured.

• Check that the endpoint contexts are unique and properly formatted.

• Make sure to click Save after making changes to preserve your configurations.

2. Configuring a Connector

Navigate to the Connectors screen

Choose a connector and edit it

Choose the appropriate environment and click on “Step 2: Connector Environment Settings.” Then, navigate to the “Health Check” tab.

In this screen, you can configure the endpoints for HTTP requests used to check the health of a service. Additionally, you can set up PING settings if the endpoint supports pinging to verify its health status.

Connector Environment Settings

1. Select the Environment:

• Choose the environment you wish to configure from the “ENVIRONMENTS” dropdown. In this case, select docker-hub-DeskTOP.

2. HTTP Setting Configuration:

• URL: Enter the webhook URL to which the HTTP POST request will be sent. For Slack, this will be in the format https://hooks.slack.com/services/....
  • Example: https://hooks.slack.com/services/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.
• Method: Select POST from the method dropdown as we are sending data to the server.

3. Request Body:

• Enter the JSON payload to be sent with the HTTP request.
  • Example:
    json { "text": "Health check" }

4. Response Body to Validate:

• Enter the expected response body to validate the health check.
  • Example: ok.

5. Ping Setting:

• Optionally, specify a server to ping for checking network connectivity. This is not configured in the provided example.

6. Submit Configuration:

• Review all settings.
• Click on the Submit button to save the configuration.

3. API structure for health check

This explains how to access the Health Checks once the GLU.Engine has been built and deployed.

When GLU.Engine is up and running there will be an API exposed that provides details on endpoints. The path available will depend on the Health Check manager configuration. As an example if the FULL path is Active then the following method will be available.

API Endpoint

• URL: http://<gateway-host>:<connector-port>/health/full
• Method: GET

Response Format

The API returns a JSON object containing detailed information about the status and duration of each endpoint. Below is an example of the response:

{
    "status": "ALIVE",
    "totalDuration": "00:00:00.001482",
    "entries": {
        "keycloak": {
            "status": "ALIVE",
            "data": {},
            "tags": [
                "Hkey"
            ],
            "duration": "00:00:00.001074"
        },
        "slack": {
            "status": "ALIVE",
            "data": {},
            "tags": [
                "Slack-tag"
            ],
            "duration": "00:00:00.001482"
        }
    }
}

Response Fields

• status: Indicates the overall status of the API.
• Example: "ALIVE"
• totalDuration: Shows the total time taken to query all endpoints.
• Example: "00:00:00.001482"
• entries: Contains individual entries for each endpoint.
• keycloak: Represents the Keycloak endpoint.
  • status: Status of the Keycloak endpoint.
  • Example: "ALIVE"
  • data: Additional data related to the endpoint (currently empty).
  • Example: {}
  • tags: Tags associated with the Keycloak endpoint.
  • Example: ["Hkey"]
  • duration: Time taken to query the Keycloak endpoint.
  • Example: "00:00:00.001074"
• slack: Represents the Slack endpoint.
  • status: Status of the Slack endpoint.
  • Example: "ALIVE"
  • data: Additional data related to the endpoint (currently empty).
  • Example: {}
  • tags: Tags associated with the Slack endpoint.
  • Example: ["Slack-tag"]
  • duration: Time taken to query the Slack endpoint.
  • Example: "00:00:00.001482"

4. Tips and Troubleshooting

Network Issues: If the health check fails, check network connectivity and ensure there are no firewall rules blocking the request.

Response Validation: Make sure the response body you expect (ok in this case) is accurate and matches the actual response from the server.

Linking the swagger to your connector

For any REST connector, the green button is available for you to link a swagger.

The first time you use the Swagger Load tool the pop-up below will prompt you to either upload a .json or .yaml file or to point to a URL for the Swagger file you wish to use.

Once this is done and the Swagger document has been loaded against the connector, when you view the connector in the Connector screen you will see the BOTH button will be pink and an extra button will appear in the ACTION column.

Using the swagger to generate configuration

You can access the Swagger Load tool within the Orchestration Manager. It will appear if the Connector you have selected is REST. If not, it will not appear.

If you have previously loaded and used a Swagger file, when you click on ‘Generate Endpoint from Swagger’ it will bring up the most recently loaded Swagger (example below). If you want to use a different Swagger, click on ‘Load New Version’.

The Connector Swagger Manager popup (above) will show all API Transactions available within the Swagger file. You can use the radio buttons to select the API Transaction you want to generate you config for.

Below, you can see the ‘API Transaction’ selected is ‘Find Pet by ID’ and you are then given the option to define the Request and Response content types depending on what the API Transaction chosen supports.

You then click ‘Generate’ to create the configuration for this leg of your Orchestration. You then simply need to clear the validation warnings by setting the Parameter Names to use for each Parameter on the Request and Response.

The “Default Schema Name” defined in the connector will be stored in a GLU variable named GLU_schemaName. This variable can be utilised within an integration.

Please note the camel case for the word GLU_schemaName.

The screen below shows where the Default Schema Name can be defined in the Console Connector screen:

The screen below shows how the variable is used in the call to the stored procedure for a Database endpoint connector.

${GLU_schemaName}

The folder “directoryName” is the only path parameter, and there are 92 available query parameters that allow you to execute various actions, such as renaming, removing, avoiding changes, sorting, searching within sub-directories, and more.

Refer to this Camel Component page for details on all the actions this connector type offers: https://camel.apache.org/components/latest/file-component.html

File Connectors enable a GLU.Engine to provides a set of endpoints for reading and writing files. This component supports various file-related operations, such as consuming files from a directory, splitting and aggregating large files, and transforming data as it is being processed.

The connection requires 3 path parameters:

1. Host: The host parameter specifies the location or address of the file system you want to connect to. This could be a hostname, IP address, or a file path depending on the configuration of your file system.
2. Port: The port parameter is typically used for network file systems that require a port number for communication. If you’re connecting to a local file system or a network file system that doesn’t require a port number, you may leave this parameter empty or specify a default port number.
3. Directory Name: This parameter specifies the name or path of the directory within the file system that you want to interact with. This could be the root directory or a specific sub-directory where your files are located.

It also includes Query Parameters that enable you to perform all sorts of actions such as renaming/removing / not touching / sorting / sub-directory searching / etc.

Refer to this Camel Component page for details on all the actions this connector type offers: FTS Apache Camel and contact GLU support for more information on how to utilise File connectors.

FTP & SFTP Connectors

The File Transfer Protocol (FTP) is an application protocol in the Internet protocol suite. It allows file transfers among unlike hosts in diverse inter-networking environments. Using FTP, you can move a file from one computer to another, even if each computer has a different operating system and file storage format. Files can contain data, programs, text, or anything that can be stored online.

Two versions of Client FTP are supported by GLU.Ware – Client FTP2 and Client FTP.

GLU provides robust support for FTP (File Transfer Protocol) processing. The FTP component allows for the transfer of files between a local file system and a remote FTP server. The component supports both the FTP and SFTP (SSH File Transfer Protocol) protocols.

With GLU, you can perform various FTP-related operations, such as uploading and downloading files, listing files in a directory, and deleting files. You can also use GLU’s built-in data format support to convert data between different formats as it is being transferred to and from the FTP server.

GLU provides a flexible and extensible FTP endpoint that can be used in various types of use cases, such as data integration and ETL (Extract, Transform, Load) pipelines. The endpoint can be configured using a variety of options, such as username, password, hostname, port number, and the path to the FTP directory.

Additionally, GLU’s built-in support for error handling, retries, and exception handling make it easy to ensure reliable and efficient FTP processing, even in cases where network connectivity or other issues may cause temporary failures.

Connector Configurations

By default there are 6 Connector Properties that are pre-populated being: 

1. noop: If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, GLU will set idempotent=true as well, which avoids consuming the same files over and over
2. stepwise: Whether or not stepwise traversing directories should be used or not. Stepwise means that it will CD one directory at a time. You can disable this in case you can’t use this approach.
3. binary: Specifies the file transfer mode, BINARY or ASCII. Default is ASCII (false).
4. username: Specifies the username to use to log in to the remote file system.
5. passiveMode: Specifies whether to use passive mode connections. Default is active mode (false).
6. useList: Whether to allow using LIST command when downloading a file. Default is true. In some use cases you may want to download a specific file and are not allowed to use the LIST command, and therefore you can set this option to false. Notice when using this option then the specific file to download does not include meta-data information such as file size, timestamp, permissions etc, because this information is only possible to retrieve when LIST command is in use.
   
   

You can use the ‘Add Property’ tool to add as many more as your FTP configuration requirements.

Refer to all property options here Apache Camel FTP2 and FTP Apache Camel

Companies store usernames, passwords, email addresses, printer connections, and other static data within directories. LDAP is an open, vendor-neutral application protocol for accessing and maintaining data. LDAP can also tackle authentication, so users can sign on just once and access many different files on the server. LDAP is a protocol, so it doesn’t specify how directory programs.

As some examples, LDAP can be used to validate usernames and passwords with Docker, Jenkins, Kubernetes, Open VPN, and Linux Samba servers. LDAP single sign-on can also be used by system administrators to control access to an LDAP database.

To configure an LDAP connector, follow these steps:

1. Provide a Connector Name: Choose a descriptive name for your LDAP connector. This name will help you identify the connector in your configuration.
2. Choose OUTBOUND for the Connection Direction: Since LDAP operates based on a client-server model, you typically establish outbound connections from your system to the LDAP server.
3. Choose LDAP for the Protocol: Select LDAP as the protocol for your connector. This protocol is specifically designed for accessing and maintaining data stored in LDAP directories.
4. Optionally Provide a description: You can provide a brief description of the connector to help other users understand its purpose or functionality.
5. Use the tick box to choose the Applications to which this connector applies.
   

Note: Host / Service Name (for docker) must always start with ldap://

With special adjustments in the GLU console, you allow Redis to be used instead of the existing cacheing mechanism.

If Redis is used for cacheing, it provides a persistent data store for the cache values (i.e. if the GLU.Engine is switched off the cached data is still stored in the Redis database, making it available for when the GLU.Engine comes alive)

For more details on Redis please refer to https://redis.io/

To use Redis a connector needs to be set up to point to a Redis database.

1. Choose the Redis Connector

2. Choose INBOUND | OUTBOUND Connectors

3. Set the host and port for the Redis Database

If the redis database setup requires password, these can be added under properties.

4. On The transaction level of the integration choose Redis and the Redis connector

The redis option and connection dropdown will only be accessible if there is a redis connector setup and assigned to the GLU.Engine. Redis is available for the cache in the integration as well as for session management on GLU.USSD.

When this is selected, this will mean all the cache commands and cache variables will be stored in Redis.

For more information on the cache commands to access these values refer to GLU Functions

Redis Database Indexing

GLU seamlessly integrates with Redis, utilising the database index to specify the connector’s targeted database. Redis, by default, supports 16 databases, each identified by a number at the end of a Redis URL, such as redis://localhost:6379/0. The default is database 0, but you can assign any number from 0 to 15. If necessary, Redis can be configured to support additional databases through redis.conf. Each database offers a distinct keyspace, providing independence from one another. To configure it, kindly insert the index number in the connector’s property, as demonstrated in the example below.

Asynchronous messaging refers to a communication pattern where a sender and a receiver do not need to be active or available at the same time. In contrast, synchronous messaging requires both the sender and receiver to be available at the same time for a communication to take place.

The use of message queues in asynchronous messaging can provide several benefits over synchronous communication:

1. Scalability: Asynchronous messaging allows for communication between loosely coupled systems, which can be scaled independently, leading to better overall system scalability.
2. Resilience: By decoupling systems, message queues provide a buffer between the sender and receiver, allowing the receiver to process messages at its own pace. This can improve system resilience, as it reduces the risk of cascading failures.
3. Improved performance: Asynchronous messaging allows for non-blocking communication, enabling the sender to continue processing requests while waiting for a response. This can lead to improved system performance and responsiveness.
4. Increased flexibility: Message queues allow for the creation of flexible and dynamic communication patterns, where messages can be routed to multiple recipients, transformed, and processed in parallel.

Overall, the use of asynchronous messaging and message queues can provide greater flexibility, scalability, and resilience compared to synchronous messaging.

GLU Message Queue Connectors

GLU.Ware supports integration to both ActiveMQ (Produce and Consume) and Rabbit Message Queues (Produce). Often useful for Asynchronous offloading of a task in a flow, MQ’s can help to speed up transaction round-trip times.

The Client DevOps team will be responsible for the MQ setup and configuration as for any other End-Point that the GLU.Engine Connects to as such configuration of the MQ’s is not covered here.

Based on the setup of the MQ Broker, the GLU Connector can be configured to support the appropriate Protocol e.g. TCP or AMQP.

The GLU.Engine will contain all the libraries to facilitate the connections. So as an example for RabbitMQ all the client components are in the GLU.Engine no additional software is required.

Publish/Subscribe Integration Pattern

ActiveMQ and RabbitMQ are two popular message brokers that support the Microservice Publish/Subscribe integration pattern using topics for consumption by subscribers. GLU plans to expand its support for Pub/Sub pattern topics by adding compatibility with additional messaging systems such as Amazon SQS, IBM MQ, RocketMQ, Google Pub/Sub, Apache Pulsar, and Apache Kafka. Please log any requirements for GLU.Ware to priorise support for any of these via the GLU.Support desk.

The Publish–subscribe is a messaging pattern where senders of messages, called publishers, do not program the messages to be sent directly to specific receivers, called subscribers, but instead categorise published messages into classes without knowledge of which subscribers, if any, there may be. Similarly, subscribers express interest in one or more classes and only receive messages that are of interest, without knowledge of which publishers, if any, there are.

This pattern is commonly used in messaging systems as part of larger message-oriented middleware systems. Many messaging systems, including Java Message Service (JMS), support both the Pub/Sub and message queue models.

In the Publish-subscribe model, subscribers typically receive only a subset of the total messages published. The process of selecting messages for reception and processing is called filtering. In a topic-based system, messages are published to “topics” or named logical channels. Subscribers in a topic-based system will receive all messages published to the topics to which they subscribe. The publisher is responsible for defining the topics to which subscribers can subscribe.

RabbitMQ

RabbitMQ can be configured to either produce a RabbitMQ transaction on a Queue or consume from a RabbitMQ Queue. If a consumer is to be used then this will need to have a connector set up which has OUTBOUND/INBOUND set to BOTH.

Both consumer and producers can be configured so that the settings for the rabbitMQ can be adjusted from the application.yml file.

Connector Setup

Set Protocol to RABBIT_QUEUE. In the Connector Environment Settings panel, set the Username and Password for access to the RabbitMQ:

Producer

When RabbitMQ is set up in a Connector, it will send transactions of the RabbitMQ as a producer.

Using application.yml file

To have the configuration setup so it is possible to configure the RabbitMQ from the application.yml file, you will need to tick the “Use Queue Holder” checkbox:

When this is ticked the default queue holder value needs to be setup. This value will be the initial value which is held in the application.yml file and used for the GLU.Engine interactions with RabbitMQ.

The parameters in the default queue string should use “&amp ;” separator values.

When not using application.yml

To have the configuration setup so it is not possible to configure the RabbitMQ from the application.yml file, you will need to untick the “Use Queue Holder” button.

When this is unticked the Queue field should contain the rabbitMQ queue which the transaction will go to. The Options field should contain the string of characters which define the settings for the transaction going to RabbitMQ.

The parameters in the default queue string should use “&amp ;” separator values.

Consumer

When RabbitMQ is set up as a connector it will receive the incoming transactions.

When the GLU.Engine is run then then this API will listen on the Queue from RabbitMQ and if there is transactions in the Queue it will pull them in and start the orchestration process.

Using the application.yml file

To have the configuration setup so it is possible to configure the RabbitMQ from the application.yml file, you will need to tick the “Use Queue Holder” button.

When this is ticked the default queue holder value needs to be setup. This value will be the initial value which is held in the application.yml file and used for the integrations interactions with RabbitMQ.

The parameters in the default queue string should use “&amp ;” separator values.

When not using application.yml

To have the configuration setup so it is not possible to configure the RabbitMQ from the application.yml file, you will need to untick the “Use Queue Holder” button.

When this is unticked the Queue field should contain the rabbitMQ queue which the transaction will go to . The options field should contain the string of characters which define the settings for the transaction going to RabbitMQ.

The parameters in the default queue string should use “&amp ;” separator values:

The GLU.Ware ISO8583 Connectors make use of the opensource JPOS ISO8583 framework. More information can be found on JPOS here. GLU makes use of JPOS which is distributed under the GNU Affero General Public License V3.

JPOS is the de-facto OpenSource implementation of the international ISO8583 standard. JPOS is used in highly demanding production environments in over 80 countries processing literally thousands of transactions every single second, 24x7x365. JPOS provides an open-source software bridge between ISO 8583-based messages exchanged between processing networks and systems. ISO8583 is well suited to real-time, low-latency, and continuous communication scenarios. As such, it serves as the messaging foundation for systems that exchange electronic transactions made for example by cardholders using payment cards.

Connections, Session and Persistence

The GLU ISO8583 connector uses a persistent session-based protocol, it relies on establishing persistent connections meaning that the connection between the client and the server is established once and maintained throughout the processing of multiple transactions. Once a connection is established, the client and server can exchange multiple messages within the same session. The session remains open until either the client or the server terminate it explicitly or if a predefined session timeout occurs. This approach enables a faster and more efficient flow of data and reduces the need for frequent connection setup and teardown.

Multiplexing the ISO Channel with a MUX

Imagine an acquirer implementation that receives several requests at a time from several POS terminals and that has to route those requests to an issuer institution by means of an ISO Channel. While you can establish one socket connection per transaction, it is common practice to setup just one socket connection (handled by an ISO Channel instance) and multiplex it. A ‘MUX’ is basically a channel multiplexer.

The default multiplexing interface used by GLU.Ware is the QMUX component. QMUX is an asynchronous, powerful, Q2 service that implements the MUX interface to Multiplexing an ISO Channel.

In order to match requests to responses, fields that have unique values that are echoed back are used. For QMUX to return the response matching the request, it needs to know which fields are always identical between the request and response. QMUX by default uses DE11 and DE41 (default field11 + field41 is used) as the matching keys.

When a message arrives on a MUX’s underlying ISO Channel, the MUX implementation checks to see if that message’s key is registered as a pending request. Should that key match a pending request, the response is handed to the waiting thread. If the key was registered as a request, or the response comes in too late then that response is (depending on the configuration) ignored, forwarded to an ISORequestListener or to a well-defined Space queue. Under many situations, the same channel that a client application may use to send requests and wait for responses may also receive requests coming from the remote server.

Here’s how the QMUX works:

• Device Connections: The QMUX manages the communication channels (connections) between the GLU.Engine Client application and the Server-side system. It abstracts the low-level communication details and provides a higher-level API for the application to interact with the devices.
• Queue Management: Each device connected to the QMUX has its own input and output queues. These queues ensure that communication between the application and the devices is managed in an orderly and efficient manner, preventing data loss and ensuring proper synchronization.
• Asynchronous Processing: JPOS follows an event-driven architecture. When a device generates an event (e.g., a barcode is scanned, a paper roll is low), the corresponding event handler in the application is notified asynchronously. The QMUX manages these events and ensures they are properly dispatched to the application’s event handlers.
• Concurrency: The QMUX handles concurrent access to the device queues and ensures that requests from multiple threads are processed correctly and efficiently.
• Device Independent: The QMUX provides a unified API for interacting with various types of devices. This abstraction allows developers to write applications that are not tightly coupled to specific device implementations.

MUXPools

As an alternative to the QMUX component, GLU Connectors support MUXPools. MUXPools provide failover as well as load balancing options at the MUX level. The mechanism desired is referred to as a Strategy.

Client side MUXPools establish connections to the Server. If you have set MUXPools to be = 4, then the Client side will create and use up to 4 connections in parallel. These can all connect to the same IP/Port on the Server side. Each MUXPool connection is like a pipe. The GLU.Engine will use those pipes to process transactions within, the pipes once connected stay open to process transactions within until they are disconnected.

Disconnects can be triggered by the GE (client side) or by the Server side.

In the Q2 log if you see:

• <io-timeout/> … that means the GLU.Engine is initiating a connection closing event.
• <peer-disconnect/> … that means the GLU.Engine has received a connection closing event from the far side.

The GLU.Engine could trigger a connection to be closed if for example the timeout threshold is reached.

When configuring your GLU.Engine with an ISO outbound connector, you can set the timeout for those connections. That timeout setting will be applied to all 4 connections. If there is no traffic over a specific connection for longer than the timeout setting, the GE will disconnect that MUXPool connection. It will re-establish that connection within 5 seconds.

To ensure there is always a Client-Sever connection active, in particular if you have low traffic volumes, you can use an application level Echo message (0800/0810’s) and / or a network level Keep Alive to maintain the Client – Server connection. The frequency of your Echo messages should be configured to ensure that connection timeouts don’t occur.

If using an Echo message along with say x4 MUXPools, the Echo message will be sent on any of the x4 available MUXPool connections. A successful response, with no other traffic being processed will result in that MUXPool connection remaining active until the expiry of the GLU Connector Message Timeout setting (defaulted to 45s). Thereafter that MUXPool connection will timeout.

In the meantime, if your Echo message delay is say 55s (it is recommended to make this repeat frequency lower than the connector timeout) so that the next Echo message will be sent on any of the available active MUXPool connections. When a MUXPool connection times out, the system will re-establish the connection within 5 seconds. The Echo messages will ensure that there is always at least one active MUXPool connection.

MUXPool Strategies

PRIMARY SECONDARY (this is the default strategy)

• This strategy divides the MUX channels into two groups: primary and secondary. The primary channels are used for processing normal transaction requests, while the secondary channels serve as a backup for handling fallback scenarios. When a transaction is initiated, the MUXPool will attempt to use a primary channel first. If a primary channel is available and working, the transaction is processed using that channel. However, if a primary channel is unavailable or fails to respond within a specified timeout, the MUXPool will switch to using a secondary channel to process the transaction.
• Typically, the primary and secondary channels are connected to different backend systems or network paths. The idea is to provide redundancy and ensure that transactions can be processed even if there is a failure in one of the primary channels.

ROUND-ROBIN

• This strategy is a load-balancing strategy that distributes transactions evenly across the available MUX channels in a round-robin fashion. When a transaction is initiated, the MUXPool selects the next available channel in a cyclic order to process that transaction. Each new transaction is assigned to the next MUX channel in the sequence, creating a balanced distribution of transactions across all channels.
• This strategy that distributes transactions evenly across the available MUX channels in a round-robin fashion. A MUX Pool is a collection or pool of multiple MUX Channels. The MUX Pool is responsible for managing these channels, making them available for transaction processing, and ensuring that the transactions are evenly distributed or processed based on the defined strategy. The number of MUX Channels in a MUX Pool is determined by how you initialise and configure the MUXPool class in your application. When creating a MUX Pool instance, you have the flexibility to specify the number of MUX Channels you want the pool to manage.

ROUND ROBIN WITH OVERRIDE (BETA – Contact GLU.Support prior to using this Strategy)

• To be used if you need to force completions and reversals to go out the same route as their corresponding originals.

SPLIT BY DIVISOR (BETA – Contact GLU.Support prior to using this Strategy)

• By using a splitField, ISO Messages can be distributed to different channels within the Pool.

If making Strategy selection by configuration from the GLU.Console the entry format is handled, however if making changes directly in the GLU.Engine application.yml then you need to set the Strategy option as one of the below, note these are case and format is sensitive. The Default strategy must be in CAPS, others all lower-case i.e., options are as follows:

• PRIMARY-SECONDARY: Use this option for primary-secondary strategy.
• round-robin: Use this option for round-robin strategy.
• round-robin-with-override: Use this option for round-robin strategy with override.
• split-by-divisor: Use this option for split-by-divisor strategy.

For further context on MUXPool see: http://jpos.org/doc/javadoc/org/jpos/q2/iso/MUXPool.html

Channels, Connections, Keep Alive’s and Echo’s

Given the session based nature of ISO8583 protocol, ISO Server and Client systems employ two mechanisms to maintain session connections that might otherwise expire / end when there is no other traffic being processed.

Channel:

A channel in jPOS represents a logical connection to a remote endpoint, typically a payment terminal or another financial system. It abstracts the communication protocol being used (e.g., ISO 8583) and provides an interface for sending and receiving messages. Channels handle the encoding and decoding of messages and manage the flow of messages between jPOS and the remote endpoint.

TCP Connection:

A TCP (Transmission Control Protocol) connection is a lower-level network connection that provides a reliable, ordered, and stream-oriented communication between two devices over an IP network. In jPOS, a TCP connection refers to the underlying network connection established between the jPOS instance and the remote endpoint (such as a payment terminal or another financial system) to exchange ISO 8583 messages.

A single Channel can manage multiple TCP Connections simultaneously.

Keep Alive:

Primarily a network-level mechanism to prevent connection closures due to inactivity by periodically exchanging messages at the network level. The addition of the Keep Alive is configured in Step 2 when configuring your ISO8583 Connector.

Echo:

An application-level mechanism to validate the operational status of a remote component by sending a message and expecting an identical response over a Channel.

Both mechanisms can be beneficial in different scenarios and can even be used together to ensure both network-level stability and application-level operability in a financial transaction system.

Connector Settings

Packagers

An ISO8583 packager is responsible for encoding the data into the correct message format before sending it over the network, and for decoding the received message into usable data. It also performs tasks such as error checking, data validation, and data formatting to ensure that the message is transmitted accurately and efficiently.

The diagram below illustrates the GLU ISO8583 Connector components and Packager functions.

ISO8583 Packagers handle the following:

• Message Structure Definition: A packager defines the structure of an ISO 8583 message, specifying the layout and order of fields within the message. ISO 8583 messages consist of a fixed set of data elements, each representing different aspects of the transaction, such as the transaction type, amount, cardholder data, and more.
• Data Encoding and Decoding: When you configure an ISO 8583 message in your GLU.Engine application, the packager is responsible for encoding the data from the application’s fields into the binary format required by the ISO 8583 standard. Conversely, when an incoming ISO 8583 message is received, the packager decodes the binary data into a structured format that the application can understand.
• Field Formatting and Validation: The packager handles data formatting and validation for each field in the ISO 8583 message. It ensures that data is correctly formatted according to the standard’s specifications and performs basic checks to ensure data integrity and consistency.
• Reusable Templates: The packager allows you to define reusable templates for various message types. This means you can define the structure and format of different ISO 8583 message types once and reuse them across your application, reducing redundancy and improving maintainability.
• Customisation: While ISO 8583 defines a set of standard fields, financial institutions and payment processors often have specific requirements that go beyond the standard. Packagers can thus be customised to accommodate these specific needs by adding or modifying fields as necessary.

It is important to understand what packager the system you are connecting to uses, GLU supports a wide range of packagers that can be selected from the Console. Support for bespoke / custom packagers is also provided via support request.

Channel Type

Refers to the type of communication channel used for transmitting ISO8583 messages between systems.

Send Message Length

Adds the length of the message to the request.

ISO8583 Messages

Data elements in an ISO8583 payload are identified by their position within the message string. The position of each data element within the message is specified by its field number and its position within that field.

Sample Raw ISO8583 Message Payload

This is an incoming MTI 1200 ISO8583 message with binary values on the left and the message string on the right. When unpacked, this MTI 1200 looks like this:

Those values can be seen in the message string. GLU.Ware needs to be configured to unmarshall those values so the payload can be unpacked.

Sample ISO8583 MTI 0800 Message Payload:

0000  30 38 30 30 82 38 00 00  00 00 00 00 04 00 00 00  0800.8……….

0010  00 00 00 00 30 33 32 32  31 31 30 35 34 30 32 37  ….032211054027

0020  30 35 31 35 31 33 30 35  34 30 30 33 32 32 33 30  0515130540032230

0030  31                                                1

An ISO8583 message is made of the following parts:

• Message type indicator (MTI)
• One or more bitmaps, indicating which data elements are present. It consists of primary bitmap and secondary bitmap. This isn’t human readable.
• Data elements, the actual information fields of the message- can be readable.

The placements of fields in different versions of the standard varies; for example, the currency elements of the 1987 and 1993 versions of the standard are no longer used in the 2003 version, which holds currency as a sub-element of any financial amount element.

https://www.ibm.com/docs/en/integration-bus/10.0?topic=formats-iso8583-messaging-standard

Unmarshalling of ISO parameters from the above data element section of an ISO payload relies on the specified position of the parameter in the data payload. The above data payload prints out with this value:

• 03221105402705151305400322301

This is unmarshalled in GLU by specifying the position and length of each parameter. Field lengths are specified in the packager:

<isofield id=”7” length=”10” name=”Date and time, transmission” class=”org.jpos.iso.IFA_NUMERIC”/>

<isofield id=”11” length=”6” name=”Systems trace audit number” class=”org.jpos.iso.IFA_NUMERIC”/>

<isofield id=”13” length=”4” name=”Date, Effective” class=”org.jpos.iso.IFA_NUMERIC”/>

Example above unmarshalls as follows:

• Field[(7) Value (0322110540)] to Param :[transmissionDateAndTime]
• Field[(11) Value (270515)] to Param :[systemTraceAuditNumber]
• Field[(12) Value (130540)] to Param :[timeLocalTransaction]
• Field[(13) Value (0322)] to Param :[dateLocalTransaction]
• Field[(70) Value (301)] to Param :[NetMngInfoCode]

Unmarshalling Example in GLU.Console

Field 11 has a Field Number and some Field Data Settings:

Note: Subfields are referenced as field.subField: 63.2

This tells the GE that the transmissionDateAndTime parameter comes from Field #7 and is 10char.
This gives us the value above:  0322110540

That leaves the remainder of the payload to be broken down into the remaining parameters.

The parameters can then be used in requests to other systems in different protocols/content types as needed. Standard handler options are available for these values if required.

Unmarshalling from Content of ISO8583 values

Further unmarshalling of the contents of an ISO8583 value is also possible. For example, XML sent inside ISO8583 can be unmarshalled by the path of the required value.

Example:

<isomsg direction="incoming">
	<!-- org.jpos.iso.packager.GenericPackager[packagerISO8583.xml] -->
	<field id="0" value="1210"/>
	<field id="3" value="310000"/>
	<field id="4" value="000000002000"/>
	<field id="7" value="0308134849"/>
	<isomsg id="62">
		<!-- org.jpos.iso.packager.GenericSubFieldPackager -->
		<field id="3"><![CDATA[18IFSFData3584<IFSFData xmlns:ns0="http://www.nrf-arts.org/IXRetail/namespace">
	<ProductSets>
		<MessageText>~0011*      Terms and Conditions Apply      **    For queries contact Us    **     Customer Care 0860 30 40 50      *\</MessageText>
		<BalanceOfPoints>000000020</BalanceOfPoints>
		<PointsForTransaction>00000020</PointsForTransaction>
		<BasePointsEarned/>
		<AvailablePointsBalance>0000002725</AvailablePointsBalance>
		<NewPointsBalance>0000002725</NewPointsBalance>
		<PointsToMoneyRatio>000100</PointsToMoneyRatio>
		<RedemptionRefNumber>167827</RedemptionRefNumber>
	</ProductSets>
</IFSFData>]]></field>
	</isomsg>
	<field id="63" value="1CLRG4112033663407SLEV"/>
</isomsg>

The path for RedemptionRefNumber is IFSFData.ProductSets.RedemptionRefNumber- this would return the specific value for that XML parameter: 167827

Marshalling Example in GLU.Console

To respond to an ISO8583 request, the Marshalling is similar for simple payloads:

Specify the Field #:

Specify the parameter to be sent with parameter length:

For parameters with variable length, the value can be left-padded with leading characters (usually) zero OR the length can be calculated and added to the header of that parameter.

Create a value from other parameters:

To build up a single ISO parameter from multiple GLU parameters you add them in sequence and specify the position each value has for the ISO parameter:

This creates an ISO amount format with additional ISO info: accountType, currency etc:

• Field[54] : 1002800C000002079748

Broken down from the above:

• 10 = accountType
• 02 = amountType
• 800 =currencyISOUGS (ISO currencyCode)
• C = amountSign (Credit or Debit)
• 000002079748 = amountTransaction – this is an amount of 2079748, left padded with zeros to fill the required char count for the field. This value was previously rounded in the Derived Parameter.

Length Calculation

NOTE: There’s a function for this- see here: https://glu.global/glu-guide/glu-functions/#get-iso-message-with-lengths-function

1. Get the Length:

2. Get length of the length (how many char for fieldLength):

3. Marshall these values into the required parameter:

Template:

18IFSFData${lengthField12722Length}${Field12722Length}${Field12722AllData}

Value:
Field[127.22] :
18IFSFData3588<IFSFData xmlns:ns0=”http://www.nrf-arts.org/IXRetail/namespace”><ProductSets><MessageText>*      Terms and Conditions Apply      **    For queries contact Pick n Pay    **     Customer Care 0800 11 22 88      *</MessageText><BalanceOfPoints>46047</BalanceOfPoints><PointsForTransaction>0000215</PointsForTransaction><BasePointsEarned>0</BasePointsEarned><AvailablePointsBalance>0000046262</AvailablePointsBalance><NewPointsBalance>0000046262</NewPointsBalance><PointsToMoneyRatio>000100</PointsToMoneyRatio><RedemptionRefNumber>BP0000049184</RedemptionRefNumber></ProductSets></IFSFData>

• The Blue Text is the Value (AllData)
• The 588 is the length of AllData ((number of characters for the AllData value)
• The 3 is the length (number of characters for the 588 length value)

NOTE: There’s a function for this- see here: https://glu.global/glu-guide/glu-functions/#get-iso-message-with-lengths-function

ISO Exception Handling

The GLU Connector settings (Step 2 when configuring your ISO8583 Connector) provide a configuration option that will turn a property – ignoreISOException – on / off. If ‘on’ the GLU.Engine will ignore ISO Exceptions such that those ISO Exceptions do not cause the GE to drop its connection to the Server.

ISO8583 Log Examples

In addition to the ‘normal’ GLU.Engine logs, since GLU.Ware uses Q2 (the QSP version 2 container within jPOS), Q2 logs generated by GLU.Engines that have ISO8583 connectors. The Q2 logs should be used to trace jPOS Channel activity. These can be used in conjunction with the gluware.log files.

Q2 logs (saved a q2.log files) are generated in an ISO8583 GLU.Engine. This happens before ISO8583 payloads are ingested into the GLU.Engine and prior to those events being printed in the gluware.log.

Any investigation into an ISO8583 issue should start with the q2.log. The q2.log uses the specified packager to validate payloads.

_{<isomsg direction=”incoming”>
<!– org.jpos.iso.packager.GenericPackager[packagerISO8583.xml] –>
<field id=”0″ value=”1200″/>
<field id=”3″ value=”310000″/>
<field id=”4″ value=”000000002000″/>
<field id=”7″ value=”0308131930″/>
<field id=”11″ value=”167827″/>
<field id=”12″ value=”230308011930″/>
<field id=”22″ value=”123456789112″/>
<field id=”49″ value=”200″/>
<field id=”59″ value=”292″/>
<field id=”63″ value=”1CLRG4112033663407SLEV”/>
</isomsg>}

When something goes wrong you might see an error packing/unpacking field X:

<session-error>
    <iso-exception>
      org.jpos.iso.IFA_LLCHAR: Problem unpacking field 12 (org.jpos.iso.ISOException: Field length 98 too long. Max: 25) unpacking field=127

This is saying Field 127.12 is too long. This can be misleading – if any of the fields sent have different length compared to what the packager is expecting, the first field to fail will print in the log. In this case, an extra Field was being sent- this skewed the unmarshalling/packager validation of subsequent Fields.

The best troubleshooting approach is to get the request message that was sent to compare it to what the packager supports/requires and what the config in the GLU.Engine is looking for. If the 3rd party is anable to change their request, the packager can be updated to cater for their payload.

If the extra parameter is required for further use in the GLU.Engine, it can be unmarshalled. If it’s not required, the GLU.Engine can pass the value through without unmarshalling.

If there is no error in the q2.log, troubleshooting can continue with the gluware.log:
Standard Unmarshalling, Handler, Derived values and protocol translation issues or errors will print in the gluware.log.

TCP/IP is an abbreviation for Transmission Control Protocol / Internet Protocol. It is a set of protocols that define how two or more computers can communicate with each other. The protocol is effectively a set of rules that describe how the data is passed between the computers. It is an open standard so can be implemented on any computer with the appropriate physical attributes. 

Properties Tab

If Properties need to set for the TCP connector, for example for a TCP/IP connector to an HSM the key = textline must be set to the value = true as shown in the example below.

As another example, by default, TCP/IP Connectors are asynchronous. If you require the Connector to be synchronous, the key = synchronous must be set to the value = true.

Decoder / Encoder Configuration

As a final, slightly more complex example, a messages sent over TCP/IP include a variable byte length header known as the Variable Length Indicator (VLI), proper configuration of the decoder and encoder is important. Here’s how to handle such requirements:

Variable Length Indicator (VLI):

• Typically consisting of 2 bytes, the VLI precedes every message sent to or from the TCP/IP endpoint.
• Bytes 1-2 indicate the number of bytes in the message, excluding the first 2 bytes.
• These 2 bytes represent a 16-bit unsigned integer in network byte order.
• Note: If the message is compressed before transmission, it must first be compressed to determine its length. As an example, suppose that you were to look at just the text (excluding the 2 byte header) making up an XML message from a particular example and that you then counted the characters and they happened to add up to 299 characters. If you took a scientific calculator (the one in Windows for example) and you typed in 299 with “Dec” (for decimal) selected, then you select “Hex”, the value that would be shown is “12B”, which means the same as 01 2B, which is exactly what one would expect to see for the first two bytes of the message if the message had been dumped to a file and then opened with a hex editor.

Configuration Properties:

• Configure the decoder type as ‘StringDecoder’ and the encoder as ‘LengthFieldPrepender’.
• Set the length to ‘2’ to handle the 2-byte VLI.
• Additionally, set the property ‘useByteBuf’ to ‘true’. This instructs GLU to convert the message body into ByteBuf before transmission, allowing for efficient byte-level manipulation.

In the screenshot below the above described Decoder / Encoder and Field Length settings are shown. Additionally you’ll see the TCI/IP property key = usebyteBuf is set to value = true … with this setting GLU will turn the message body into ByteBuf before sending it out. Just like an ordinary primitive byte array, ByteBuf uses zero-based indexing. It means the index of the first byte is always 0 and the index of the last byte is always capacity – 1.