Splitting an array and parallel processing are commonly employed in to enhance efficiency and optimise resource utilisation when dealing with large datasets or complex tasks.

Splitting an Array

It is possible to take a large payload and use the Splitter function to break it up into multiple elements to be orchestrated.

In the example below we will take a payload and store each element in a table of a database.

Body Payload

See below this simple array:

{ “report”: [ { “reportLine”: { “glcode”: “AG82652-3635”, “name”: “Sticky Buns”, “balance”: “500” } }, { “reportLine”: { “glcode”: “AG82634-3635”, “name”: “Iced Donuts”, “balance”: “2800” } } ] }

This is an array called “report” containing elements made up of a class called “reportLine”.

Target Table

The intended table structure is as follows:

Each item from the array will occupy a distinct row within the table. Furthermore, the attributes of each class instance will be appropriately assigned to the corresponding column names.

glCode	Name	Balance
AG82652-3635	Sticky Buns	500
AG82634-3635	Iced Donuts	2800

Console Configuration

Request Parameters setup

To configure the marshaling set up the Request Parameters as per the example below. The Object / Collection path stipulates the path within which to find and store the class element being unmarshalled called ‘glcode’.

Properties

Unmarshaling

Handler setup

To orchestrate the splitting of each element in the incoming payload, you need to set up a Handler specifically for splitting the payload. Below are the steps outlined based on the numbered diagram and accompanying table:

Handler Parameters

Parameter	Console Label	Example Value	Description
1	Parameter, None, Expression	None	To show the splitter options this needs to be set to ‘None’.
2	ACTION	ROUTE TO	Indicates how the splitter will be used.
3	Not from list	Not ticked	BETA description to follow.
4	FLOW CODE: Choose the flow code of the orchestration which will be triggered for each element of the array	splitterRoute944482	Flowcode of the orchestration which will be executed
5	ENABLE THREAD POOL: FALSE	FALSE	See Throttle Type 2: Concurrent Requests using Thread Pools
6	Splitter		Select this to call the flow code for each element in the array
7	Parallel Processing	Not ticked	If this is ticked then each orchestration will be run in parallel. To be used to speed up the processing.
8	Streaming	Not ticked	BETA description to follow.
9	Stop on Exception	Not ticked	BETA description to follow.
10	Skip First	Not ticked	BETA description to follow.
11	TIMEOUT	0	BETA description to follow.
12	DELIMITER	Blank	To be used for SFTP, FTP, and FILE functions see FTP / SFTP Connectors & FTP/S
13	Concatenate response	UN TICKED	Tick if the response from each of the Orchestration of the elements needs to be returned back as a Response to the API
14	unlabelled	report	Provide the name of the array (as defined in the Request Parameters
15	COMBINED ROUTE	BLANK	BETA description to follow.

ENDPOINT MANAGER PANEL

Each element within the array will be stored in the database, with each element occupying a single row. Below outlines the configuration steps for the ENDPOINT MANAGER PANEL:

REQUEST MANAGER PANEL

The below shows the setup of the REQUEST MANAGER PANEL, specifically configured with a mySQL DB connector to insert rows into a table:

insert into report(glCode,Name,Balance) value (${glcode},${name},${balance})

RESPONSE MANAGER PANEL

Ensure to include the appropriate Response Manager properties to avoid failure in case the Response Manager panel is incomplete.

Please refer to Response Manager for further details on how to complete the response manager panel.

Parallel Processing

When parallel processing is enabled, the Splitter uses a thread pool to process the messages concurrently. The thread pool is, by default, configured to use 10 threads, making it up to 10 times faster.

The Request Manager is used to define the details of an incoming request. The contents of this block will vary depending on the type of API it is connected to (see the Transaction manager for API configuration details).

Within the control panel, you can specify the structure of the request:

The Transaction defined in the request manager can be triggered in several ways. The most common is for it to be triggered from an API call inbound to the GLU.Engine. However, it is also to trigger the transaction through setting up a schedule to trigger itself OR on startup of the GLU.Engine, at which the transaction can be set up to only run once on startup.

Scheduled Tick Box

The Schedule and the “run on startup” are mutually exclusive.

Currently, Scheduler, Quartz, Cron & Timer are features that are available to use. 

Scheduler

The scheduler feature can be set up for all the REST methods, i.e. PATCH, POST, etc. 

The Scheduler feature allows repeatable transactions to be triggered by a scheduler. Each time the API is triggered, it is like a new API request triggering the transaction. This means Derived Parameters and Handlers in the API Request are all actioned. It is also possible to set default values in the inbound parameters and these will take effect. 

The Scheduler feature enables the automation of repeatable transactions via scheduling. Each time the scheduler runs the API call triggered is treated as a new request and all associated Derived Parameters and Handlers are executed. Request parameters associated with the API will be passed as null unless default values are set at which point these will be taken for the transaction.

Delay:

• This is the value for the delay between each transaction being repeated.
• This value cannot be 0.
• example: 10
• (If left empty then the default value will apply of 500)

Initial Delay:

• This is the delay before the first transaction is issued.
• This value can be 0.
• example: 5
• (If left empty then the default value will apply of 1000)

Repeat Count:

• This is the number of times which the transactions will be iterated.
• If this value is set to 0 then it will repeat for ever.
• example: 6
• (If left empty then the default value will apply of 0)

Time Unit:

• for INITIAL DEALY and DELAY.
• POSSIBLE Values: NANOSECONDS, MICROSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS, DAYS
• example: SECONDS

Trigger API:

• Is used to start an API repetitively at a fixed time based on the configuration.
• It works in the same way as a cron job is scheduled in Linux.

Quartz

Quartz allows you to schedule repeated running of an API based on a frequency from the point the GLU.Engine starts up. The counter settings all happen from the point the GLU.Engine has started.

Repeat Count:

With this feature, you can define how many times a job should be repeated. By setting the repeat count to a specific number, such as 3, Quartz will automatically execute the job the specified number of times.

Repeat Interval:

This feature determines the time gap between consecutive job executions. By setting a repeat interval, such as 10 seconds (10,000 milliseconds), Quartz will schedule the job to run at regular intervals, ensuring efficient and timely execution.

Trigger Start Delay:

This feature allows you to introduce an initial delay before a job starts executing. By specifying a delay in milliseconds, such as 5 seconds (50,000 milliseconds), you can ensure that the job begins its execution after a specific interval from the scheduler’s start.

Time Zone:

To ensure consistent scheduling across different time zones, Quartz provides a parameter to specify the time zone in which the scheduler operates. You can set the desired time zone, such as “GMT” or “America/New_York,” ensuring that the scheduled jobs align with the intended time zone, regardless of the system’s default setting.

Cron

The schedule expression can be made of 5 to 7 parts that represent the timing of the job. When expressions are composed of 6 parts, the first items is the “seconds” part (and year is considered missing). Each field represents a specific unit of time and can have a range of values:

• Field 1 (Seconds):
  • Represents seconds (0-59 or *).
  • Optional.
• Field 2 (Minutes):
  • Represents minutes (0-59 or *).
• Field 3 (Hours):
  • Represents hours (0-23 or *).
• Field 4 (Day of the Month):
  • Represents the day of the month (1-31 or *).
• Field 5 (Month):
  • Represents the month (1-12 or *).
• Field 6 (Day of the Week):
  • Represents the day of the week (0-7; both 0 and 7 represent Sunday or ? or MON-FRI).
• Field 7 (Year):
  • Represents the year.
  • Optional.

Examples:

1. 0/3 10 * * * ?:
   • Triggers an API every three seconds only in the tenth minute of each hour.
2. 0/2 * * * ?:
   • 5 parts, an API call every two minutes.
3. 0 0/2 * * * MON-FRI 2030:
   • 7 parts, an event every two minutes for weekdays only in the year 2030.

Timer

An Inbound Connector needs to be configured, typically as a simple REST Inbound Connector. It’s against this Connector that the scheduler configuration is defined. Additionally, it is also possible that even though the transaction is scheduled to run as ‘self-initiated’, this Inbound Connecter API can also be called from an external system.

The API Context Name is not applicable in this case however the Method, Content Type, Save Request As, and Add to INFO Logs are all applicable.

PERIOD:

• Similar to DELAY in Scheduler.
• Represents the value for the delay between each transaction being repeated, measured in microseconds.
• This value cannot be 0.
• Example: 1000 (for 1000 microseconds, 1 second).
• If left empty, the default value of 500 microseconds will apply.

DELAY:

• Similar to INITIAL DELAY in Scheduler.
• Represents the delay before the first transaction is issued, measured in microseconds.
• This value can be 0.
• Example: 2000 (for 2000 microseconds, 2 seconds).
• If left empty, the default value of 1000 microseconds will apply.

REPEAT COUNT:

• This parameter defines the number of times the transactions will be iterated.
• If set to 0, the process will repeat indefinitely.
• Example: 6
• If left empty, the default value of 0 will be applied.

Run once on startup Tick Box

If this is ticked then the Transaction will trigger itself only on the startup of the GLU.Engine. This will work with all inbound protocols.

Context Name

Within the Request Manager Control Panel, follow these steps to assign a ‘Context Name’ for a particular API call:

1. Common Path:
   • Define the ‘Common Path’ in the Connectors section of the Request Manager Control Panel. This is the shared part of the API URL that is common to multiple calls.
   Example: https://devportal.com/api
2. Context Name:
   • The ‘Context Name’ is the part of the URL that comes after the ‘Common Path’. It is essentially the unique identifier for a specific transaction type.
   Example (for the provided URL): /clients/8a85877a5e4ef2b767/loans
3. Combining Common Path and Context Name:
   • The full path or URL for a particular API call is created by combining the ‘Common Path’ and the ‘Context Name’.
   Example (full URL): https://devportal.com/api/clients/8a85877a5e4ef2b767/loans
4. Forward Slash ‘/’ Recommendation:
   • It’s recommended for consistency to add a ‘forward slash’ ‘/’ to the front of your ‘Context Name’. However, this isn’t mandatory, as GLU will automatically add it when building the URLs if it’s not provided.
   Example (with recommended forward slash): /clients/8a85877a5e4ef2b767/loansExample (without forward slash, added automatically by GLU): clients/8a85877a5e4ef2b767/loans

By assigning a unique ‘Context Name’, you can differentiate between different Transaction Types of transactions within the Request Manager Control Panel.

Context Name with Parameters

GLU allows for the use of a parameter within the definition of the context name. The parameter can be defined inside the context name when using the syntax: {parameterA} See below for example:

To successfully unmarshal this parameter, the following three items must be included in the configuration:

1. A request parameter must be created
2. The unmarshalling parameter attribute Name must be aligned to the context name
3. The source ‘From’ must be selected as the header.

Context Name & Route Versioning

Route Versioning is where you include a version indicator such as /v1/ or /v2/ into the URI via the Context Name field, is generally considered to be a clear and concise mechanism for the API user to ensure they are consuming the correct API version and to reference the correct documentation.

In this way, you can expose multiple versions of an API on the same GLU.Engine until such time as ‘legacy’ versions are ready to be deprecated at which time you can simply remove that version from the config. In such cases, it is advisable to include a Handler to return a 30x HTTP Response Code with an appropriate Error Message to advise any system that attempts to connect to a deprecated API version that that version is no longer in use.

As an alternative to Route Versioning is to include the versioning in the Header information. This article provides some further insight into the topic of API Version Controls. See article on versioning

Method

Based on the Initiating System Protocol the ‘Method’ drop-down options will be presented.

The available request types vary depending on the overarching API specification.

Selecting either GET or DELETE (available for HTTP/REST calls) will remove the ability to define a Content Type (see below).

Save Request As

The “Save Request As” field is used to record the datetimestamp of when the transaction was initiated in a variable which could then be used later in the Application.

This is used if for instance an audit of when the APIs need to be recorded in a Database. The parameter as part of orchestration could record the value in a database.

Add To INFO Logs

This check box should be ticked if the value from the Save Request As (i.e. the datetimestamp API is triggered) should be put in the logs.

This is put in the Logs if the logging is set to INFO.

Throttle Panel

Controls the number of transactions to put through for a period of time, i.e. 5000 transactions in 400ms

see Managing Load with Throttles for details.

Time Period (milliseconds)

The Time period the number of transactions will be controlled to. i.e. 400ms

Requests Per Time Period

The number of transactions to control i.e. 5000