GLU.Engine Configuration Control: Multi-Environment Support

Introduction
This feature is designed to mitigate potential risks associated with testing and deploying the engine in production environments. Our goal is to provide a solution that allows for seamless adjustments to the GLU.Engine’s core settings without the necessity of a complete rebuild. This approach reduces the likelihood of introducing untested modifications during production deployments.

The configuration files are exposed within the GLU.Engine system, which can be readily accessed and modified by system administrators and developers. These configuration files can be located in the “../engine/config/” directory and include the following:

  • application.yml
  • camel-context.xml
  • flows.json
  • iso8583
  • loggingSetting.xml
  • menu.json

Benefits
By exposing these configuration files, users gain direct control over the engine’s core settings. This empowers them to make necessary adjustments as required, without the need for a time-consuming rebuild. This approach significantly mitigates the risk associated with introducing untested changes during production.

Important Considerations
It is imperative to note that any modifications made to these configuration files should undergo thorough testing and review before deployment in a production environment.

Guidance on Access and Download


To access and download the configuration files, please follow these steps:

  1. While creating a new configuration or making changes to an existing one, save your modifications
  2. Navigate to “Build Manager” in the system.
  3. From the “Action” dropdown menu, select “Build/Build…”
  4. In the resulting pop-up window, click on “Download Configuration.”

GLU SchemeName

The “Default Schema Name” defined in the connector will be stored in a GLU variable named GLU_schemaName. This variable can be utilised within an integration.

Please note the camel case for the word GLU_schemaName.

The screen below shows where the Default Schema Name can be defined in the Console Connector screen:


The screen below shows how the variable is used in the call to the stored procedure for a Database endpoint connector.

${GLU_schemaName}

Setting REST Connection Limit per Route

Max Connections

For REST API outbound connections you can control the number of concurrent connections that can be made by the GLU.Engine to each outbound connector. The default configuration for this setting is 200 Connections.

In your GLU.Engine, if the number of incoming requests exceeds the concurrent connection count configured for outbound connections, the excess requests will be queued until a connection becomes available. This is known as connection throttling, which limits the number of concurrent connections that can be made to a specific endpoint.

When the maximum number of connections is reached, new requests will be queued until a connection becomes available again. The queuing mechanism is implemented by GLU’s connection pool, which manages a pool of connections that can be shared across multiple threads. This ensures that the outbound connector does not become overwhelmed and that requests are processed in a controlled and predictable manner.

The number of inflight transactions which are spawned is controlled by; Throttle Type 2: Concurrent Requests using Thread Pools.

For each service, there is an output connection spawned in the Connection Pool to establish the connection to the outbound REST API. The number of outbound connections is controlled by the setting below. This needs to be concatenated to the outbound URI for the outbound connector. See the example below.

Where 200 is the number of connections. Note, if you have query parameters in you Outbound Connector you must use the “&” characters to separate the Query parameters, for example:

/PinlessRecharge?connectionsPerRoute=20&api_key=ugz2hd7e4ejfqcq8hkgfqqjs


File Connectors

File Connectors enable your GLU.Engine to connect to a Folder location to perform various actions with Files.

The folder “directoryName” is the only path parameter, and there are 92 available query parameters that allow you to execute various actions, such as renaming, removing, avoiding changes, sorting, searching within sub-directories, and more.

Refer to this Camel Component page for details on all the actions this connector type offers: https://camel.apache.org/components/latest/file-component.html

FILE / FTP / SFTP Connectors

File Connectors

File Connectors enable a GLU.Engine to provides a set of endpoints for reading and writing files. This component supports various file-related operations, such as consuming files from a directory, splitting and aggregating large files, and transforming data as it is being processed.

The connection requires 3 path parameters:

  1. Host: The host parameter specifies the location or address of the file system you want to connect to. This could be a hostname, IP address, or a file path depending on the configuration of your file system.
  2. Port: The port parameter is typically used for network file systems that require a port number for communication. If you’re connecting to a local file system or a network file system that doesn’t require a port number, you may leave this parameter empty or specify a default port number.
  3. Directory Name: This parameter specifies the name or path of the directory within the file system that you want to interact with. This could be the root directory or a specific sub-directory where your files are located.

It also includes Query Parameters that enable you to perform all sorts of actions such as renaming/removing / not touching / sorting / sub-directory searching / etc.

Refer to this Camel Component page for details on all the actions this connector type offers: FTS Apache Camel and contact GLU support for more information on how to utilise File connectors.

FTP & SFTP Connectors



The File Transfer Protocol (FTP) is an application protocol in the Internet protocol suite. It allows file transfers among unlike hosts in diverse inter-networking environments. Using FTP, you can move a file from one computer to another, even if each computer has a different operating system and file storage format. Files can contain data, programs, text, or anything that can be stored online.

Two versions of Client FTP are supported by GLU.Ware – Client FTP2 and Client FTP.

GLU provides robust support for FTP (File Transfer Protocol) processing. The FTP component allows for the transfer of files between a local file system and a remote FTP server. The component supports both the FTP and SFTP (SSH File Transfer Protocol) protocols.

With GLU, you can perform various FTP-related operations, such as uploading and downloading files, listing files in a directory, and deleting files. You can also use GLU’s built-in data format support to convert data between different formats as it is being transferred to and from the FTP server.

GLU provides a flexible and extensible FTP endpoint that can be used in various types of use cases, such as data integration and ETL (Extract, Transform, Load) pipelines. The endpoint can be configured using a variety of options, such as username, password, hostname, port number, and the path to the FTP directory.

Additionally, GLU’s built-in support for error handling, retries, and exception handling make it easy to ensure reliable and efficient FTP processing, even in cases where network connectivity or other issues may cause temporary failures.

Connector Configurations

By default there are 6 Connector Properties that are pre-populated being: 

  1. noop: If true, the file is not moved or deleted in any way. This option is good for readonly data, or for ETL type requirements. If noop=true, GLU will set idempotent=true as well, which avoids consuming the same files over and over
  2. stepwise: Whether or not stepwise traversing directories should be used or not. Stepwise means that it will CD one directory at a time. You can disable this in case you can’t use this approach.
  3. binary: Specifies the file transfer mode, BINARY or ASCII. Default is ASCII (false).
  4. username: Specifies the username to use to log in to the remote file system.
  5. passiveMode: Specifies whether to use passive mode connections. Default is active mode (false).
  6. useList: Whether to allow using LIST command when downloading a file. Default is true. In some use cases you may want to download a specific file and are not allowed to use the LIST command, and therefore you can set this option to false. Notice when using this option then the specific file to download does not include meta-data information such as file size, timestamp, permissions etc, because this information is only possible to retrieve when LIST command is in use.

You can use the ‘Add Property’ tool to add as many more as your FTP configuration requirements.


Refer to all property options here Apache Camel FTP2 and FTP Apache Camel

LDAP Connectors

The Lightweight Directory Access Protocol (LDAP) is a directory service protocol that runs on a layer above the TCP/IP stack. It provides a mechanism used to connect to, search, and modify Internet directories. The LDAP directory service is based on a client-server model.

Companies store usernames, passwords, email addresses, printer connections, and other static data within directories. LDAP is an open, vendor-neutral application protocol for accessing and maintaining data. LDAP can also tackle authentication, so users can sign on just once and access many different files on the server. LDAP is a protocol, so it doesn’t specify how directory programs.

As some examples, LDAP can be used to validate usernames and passwords with Docker, Jenkins, Kubernetes, Open VPN, and Linux Samba servers. LDAP single sign-on can also be used by system administrators to control access to an LDAP database.

To configure an LDAP connector, follow these steps:

  1. Provide a Connector Name: Choose a descriptive name for your LDAP connector. This name will help you identify the connector in your configuration.
  2. Choose OUTBOUND for the Connection Direction: Since LDAP operates based on a client-server model, you typically establish outbound connections from your system to the LDAP server.
  3. Choose LDAP for the Protocol: Select LDAP as the protocol for your connector. This protocol is specifically designed for accessing and maintaining data stored in LDAP directories.
  4. Optionally Provide a description: You can provide a brief description of the connector to help other users understand its purpose or functionality.
  5. Use the tick box to choose the Applications to which this connector applies.

Note: Host / Service Name (for docker) must always start with ldap://

Redis Connectors

GLU supports connecting to Redis open source (BSD licensed), in-memory data structure store to be used as a database, cache, and message broker.

With special adjustments in the GLU console, you allow Redis to be used instead of the existing cacheing mechanism.

If Redis is used for cacheing, it provides a persistent data store for the cache values (i.e. if the GLU.Engine is switched off the cached data is still stored in the Redis database, making it available for when the GLU.Engine comes alive)

For more details on Redis please refer to https://redis.io/


To use Redis a connector needs to be set up to point to a Redis database.

1. Choose the Redis Connector


2. Choose INBOUND | OUTBOUND Connectors

3. Set the host and port for the Redis Database

If the redis database setup requires password, these can be added under properties.

4. On The transaction level of the integration choose Redis and the Redis connector

The redis option and connection dropdown will only be accessible if there is a redis connector setup and assigned to the GLU.Engine. Redis is available for the cache in the integration as well as for session management on GLU.USSD.

When this is selected, this will mean all the cache commands and cache variables will be stored in Redis.

For more information on the cache commands to access these values refer to GLU Functions

Redis Database Indexing

GLU seamlessly integrates with Redis, utilising the database index to specify the connector’s targeted database. Redis, by default, supports 16 databases, each identified by a number at the end of a Redis URL, such as redis://localhost:6379/0. The default is database 0, but you can assign any number from 0 to 15. If necessary, Redis can be configured to support additional databases through redis.conf. Each database offers a distinct keyspace, providing independence from one another. To configure it, kindly insert the index number in the connector’s property, as demonstrated in the example below.

Configuring Stubs and Mock Services

With GLU.Ware you can configure your own ‘mocks’ and ‘stubs’ – a fundamental to delivering fully simulated and tested APIs. Mocking and/or stubbing is essential in scenarios where you are faced with external dependencies which may hinder the completion of your test/delivery cycle. The purpose of both is to eliminate testing all the dependencies of a class or function so your tests are more focused and simpler in what they are trying to prove.

A stub is an interface that simulates the actual object and returns pre-defined and expected Responses irrespective of input.

A mock is an interface with a bit more ‘intelligence’ in that you can configure you mock services to return results based on what you send.

Benefits

  • Speedy delivery
  • Focused and reliable testing
  • The ability o test components accurately in particular contexts
  • Obtain feedback quickly from a manual or automated test

When to use a stubs & mocks?

If we are developing the back end of a minor application with a third-party to interact with an API, we can implement an interface that can disconnect us from the third-party library. Eventually, that interface will act as a stub and yield hard-coded values. Conclusively, we can use values in unit tests.

Definition of Done

As a Release Manager, it is possible to define the Definition of Done (DoD) for the Analyst and the Quality Assurance / Testers for a particular build.

By using the applicable ‘Manage Analyst DoD’ the Release Manager can define any number of specific criteria that the Analyst will be required to confirm as having been ‘done’ for that particular build. A different DoD list can be defined for the QA team.

The sample below shows the DoD for the Analyst.

The Analyst / Tester is prompted to ‘tick’ each DoD task prior to progressing the workflow to the next stage. These DoD results form part of the ‘Build History’ and Audit trail for every GLU.Engine ever built. This serves to establish sound discipline within the team using the GLU.Console.

Build Manager Tool Overview

This section will help you understand the build versions, integration specifications, and menu specifications associated with each GLU.Engine. It also explains how to interpret the status of the build cycle and the build tagging system used in different environments.

GLU.Engine List

Upon accessing the application, you will see a list of GLU.Engines available to you. These will be displayed in the ‘GLU.Engines’ column. Each GLU.Engine may have multiple build versions associated with it.

Build Versions and Specifications

Every GLU.Engine is linked to one or more build versions. Each build version is connected to an ‘Integration Version’ and, optionally, a ‘Menu Version’. The Versions column displays the available build versions for each GLU.Engine.

Status Column

The ‘Status’ column indicates the current status of the build cycle for each version. This helps you keep track of the progress of different builds.

Build Tagging System

It is crucial to understand the build tagging system used in GLU.Engines. Build versions for environments preceding the ‘Production’ Environment are tagged with the suffix ‘SNAPSHOT’. On the other hand, a final build ready for release to Production will be tagged with the suffix ‘RELEASED’. These tags help identify the state of a build version.

Spec Versioning

Please note that the build version is independent of the Integration and Menu Spec versions. Spec versioning allows you to maintain a history of configurations as your specs evolve over time. As specs may evolve independently of actual builds being promoted through your Software Development Life Cycle (SDLC), it is essential to pay attention to the build version and its associated specs when creating a build.

Importance for Docker Container Managers

For builds destined for Docker container managers, the ‘tag’ and name will apply. Ensure to keep track of the correct build version and its associated spec to avoid any confusion while deploying Docker containers. Case is important.

i.e. autotest:1.0-SNAPSHOT

To create a new Build, as a User associated with a ‘Release Manager’ profile, click on the ‘Edit / Increase Version’ button, select the Application with which you want the build to be associated (note if your User profile is only associated with a single Application, only that Application option will be presented). Then the screen below will appear. Define the Build version increment applicable by adjusting the Major and Minor version numbers, and select the associated Menu and Integration Specifications. If you are building for a container manager add the docker service name for the docker instance (see GLU.Engine Settings for setting the docker credentials).

Follow the steps below to create a new Build as a User associated with a ‘Release Manager’ profile:

  • Step 1: Access the Build Creation Page by logging in to your account and navigating to the Build Manager screen. Locate the ‘Add Build’ button at the top of the screen and click on it to show the Build Manager dialogue. (Note. if the Add Build button does not exist, this means there all the builds accessible to the user are displayed on the screen already.)
  • Step 2: Select the GLU.Engine from the list of Applications associated with your profile which will be displayed. If your User profile is linked to a single GLU.Engine, that specific GLU.Engine option will be presented automatically. Choose the relevant GLU.Engine with which you want to associate the new Build.
  • Step 3: Define Build Version Increment Upon selecting the GLU.Engine, the Build creation screen will appear. Here, you can define the Build version increment by adjusting the MAJOR and MINOR version numbers. Ensure you choose the appropriate version increments for your new Build.
  • Step 4: Select MENU VERSIONS and INTEGRATION VERSION Specifications Next, select the associated Menu and Integration Specifications relevant to your Build. These specifications will determine the functionalities and integrations that your Build will include.
  • Step 5: For Container Manager Builds If you are building for a container manager, you need to add the DOCKER SERVICE NAME for the docker instance. Please refer to the ‘GLU.Engine Settings’ documentation for instructions on setting the docker credentials correctly.
  • Step 6: Confirm and Create Build Review all the selected options and configurations to ensure they are accurate. Once you are satisfied, click the ‘Submit’ button to initiate the Build creation process. The system will begin generating the new Build, and you will receive a confirmation message upon successful creation.

The newly created Build will reflect on the Build Manager home screen with a status of ‘Analysis Phase’. Users associated with the ‘Analyst’ profile will receive an email notification to indicate that the Release Manager has created a new build requiring Analysis.

Actions

These are the ‘Actions’ in the drop-down menu.

The ‘Edit/Increase Version’ presents the same screen as for the ‘Add New Build’ but in this case the screen is an ‘Edit Build’ screen.


The ‘Delete’ option will present a confirmation screen to confirm that the selected Build Version should be deleted.


The ‘Download’ option will present a pop-up with the history of builds that have been created. Note that typically only the QA team can create a build, as such normally the Analyst would not have the permissions to create a build or to download a build.


The ‘Send DoD’ option at this point in the workflow (Analysis Phase) will present the Analyst DoD with a set of check-boxes for the Analyst to indicate which items on the DoD list have been done.


Upon clicking ‘Send’ an email will be triggered to the Release Manager to highlight the Analyst’s job completion and the associated DoD record. Thereafter the status will change to ‘Awaiting Approval’.

One-Click Builds

This button to the left of the ‘Action’ button can be used to create a ‘one-click build’. It will create the build for the Environment specified in the ‘Environments’ column. The environment in the environment column is the last environment build attempted and represents the environment that was built for, which can be seen at the top of the GLU.Engine Download Manager screen. If the build failed the environment will still be shown. However, if the build failed due to a problem with the configuration setup of the environment, then a warning will be present instead in the column.

History

The ‘History’ option will show the life of a GLU.Engine with all build manager actions (including DOD results) which have been carried out against the GLU.Engine.

Workflow: ON

If Workflow is ‘ON’ for a GLU.Engine, the Release Manager Action options are slightly different to those for the Analyst of QA Roles. For example, ‘Approve Testing’ will promote the build to the next step in the SDLC and the QA Team will be notified that a build is ready to be created and tested. ‘Reject DoD’ will return the build to ‘Analysis Phase’ for the Analyst to complete whatever critical steps are pending based on the DoD.


The ‘Build’ action will trigger the build of the GLU.Engine. This build will embed within it the Environment parameters applicable to the relevant stage in the SDLC that this build is at e.g. Testing / Pre-Prod etc. Alternatively, the QA User can create a build for a specific environment in which case a pop-up will present the available pre-defined environments as below. You have the option to add a Release Note to each Build, this is recommended to help keep track of changes being made.

Download Configuration

The Download Configuration button in the Build Manager pop-up (which appears after clicking the ‘Build …’ action option under the Actions menu) provides a the configuration files associated with the GLU.Engine. These files can be swapped for the files included in a build which will allow for direct changes to the GLU.Engine without requiring a complete rebuild, reducing the possibility of introducing untested modifications.

The download will generate a zip file which will include the following configuration Files:

  1. application.yml
  2. camel-context.xml
  3. flows.json
  4. pom.xml
  5. loggingSetting.xml
  6. swagger.json
  7. OPTIONAL menu.json (depending on whether USSD menu spec is included.)
  8. OPTIONAL ISO8584( depending on whether building for an ISO connection.)


This capability improves the flexibility and efficiency of managing the GLU ecosystem and SDLC. Users can now make targeted modifications to the GLU.Engine’s configuration files as needed, ensuring a streamlined development and deployment process.


Please note that any modifications made to these configuration files should be carefully tested and reviewed before deploying them in a production environment.


GLU.Engine Download Manager screen

Once a build is created, the ‘Download’ action will present a list of previous builds that have been created. The ‘Download’ link will result in the GLU.Engine package being downloaded to the QA Users local downloads folder from where it can be deployed to the GLU.Engine server. If a build fails for some reason, the Build Error will show as per the example in the screenshot below.


Once the Workflow has reached the Production point, you will need to complete a RELEASE to get the final build of the GLU.Engine, which is classed as the RELEASE and should be the GLU.Engine which is taken to production.


If however for some reason you are not ready to go to RELEASE, it is possible to cancel the release. With the ‘Cancel Release menu option’.

A Column exists ‘Java’ which describes the Version of Java which the GLU.Engine was built for.

Fill the form and we’ll contact you shortly

    I agree with

    cookies
    We uses cookies to make your experience on this website better. Learn more
    Accept cookies