How to start integration projects
Jan 21

How to start integration projects

Are you a company that acts as a service provider or has a lot to do with service providers or somehow acts as a service integrator? Having interfaces for (IT) services of all kinds? Yes, in this case you should read on here.

The key to optimizing processes with your partners is to automate and integrate the tools and its interfaces and let the tools communicate with each other.

Service Integration
Step 1: Workflow Analysis

Entry and exit points

Start by analyzing the workflow of your tool and adjust it if necessary. At which workflow points are there entry and exit points to partner systems? Also define what data you can expect at entry points (e.g. solution text, assignment group, …)

Status Change

These are mostly status changes such as a “ticket open”, “ticket assign to partner”, “ticket solve”, inquiries, “on hold” for various reasons etc.
Which status changes are only used for internal ticket processing, which status changes are also interesting (relevant) for the partner system? Which status changes have to come from the partner system?

Comments / Worklogs / Notes / Remarks

Almost every tool has objects for notes. These are called differently depending on the tool (Jira – Comments, ServiceNow – Worklogs, …). Define or analyze whether you differentiate between internal and external comments and which you want to transfer to a partner tool.

Required attributes

Define your mandatory attributes and your “nice to have attributes” (optional). Which fields can e.g. only received at a “ticket open”, which will be set with a later update (e.g. resolution)?
Can priority / severity / urgency / impact only be set when opening – or changed later by the partner tool? Can these fields be set freely or are they calculated or automatically set with service categories?

Free text or referenced attributes?

Can data attributes such as personal data or device data be passed as free text or database identifiers such as Active Directory names for people or CMDB IDs for devices? When passing IDs, make sure their data sources are synchronized with those of their partner tools. (regular synchronization-processes or -tasks)

Fallback mechanism or REJECT

If the partner system creates a ticket with wrong data (e.g. an unknown person), define whether the creation should really fail or whether the creation should be opened for a default person so that at least the ticket can be accepted (some minimum information).

Step 2: API analysis

Regardless of whether you want to use an HTTP rest API or an email interface with XML attachments.

After you have completed the workflow analysis and know exactly where partners are communicating with your workflow and what data attributes are required. You need to define the API exactly and, if necessary, analyze what your API can do by default and what may need to be adjusted.

Authentication

What does the authentication look like? oauth, oauth2, HTTP Basic Authentication, SAML, ….

Inbound Transaction

How can you make ticket updates in your tool?

What is the API payload for each step? What does a ticket look like? How do you add a comment? How do you do status updates? Prepare examples for each of these cases.

Outbound Transaction

How do your partners recognize ticket updates from outside?

How can a connected tool recognize ticket updates? How can it e.g. distinguish rejection from solution? Are these updates pushed out from your system (webhook) or does the connected tool have to actively pull changes?

What happens, if such a webhook cannot be delivered because e.g. the connected system is inaccessible for some reason (network, maintenance, error …)?

Errors

The error cases in particular are not (exactly) defined in many API documents and often have to be found out exploratively.

Define the error cases. What happens in the API if, for example, a wrong priority is set?

Is there a case that a ticket is “blocked”/”locked”? For example, an agent works on a ticket, this means this ticket is currently open in the application and therefore the ticket is blocked for all other users. Is it also blocked for the API? What happens in the API in such a case? Does it respond with an understandable and, above all, electronically evaluable / parseable error message?

Step 3: Test your API with an API test tool of your choice

Depending on the API (REST, soap, email, file via SFTP, …) you may need different test tools. For example, SoapUI or of course the LOMNIDO SIAM simulator.
Define test cases. A test case contains an API call with defined data and contains the expected result. Also define test cases for errors.

Step 4: Implementation of the remote API - API mapping

After analyzing your own API, it’s time to analyze the partner API. This must be done for each connection.

API mapping

How can you access the partner’s API with your own API? Can your tool do anything the other party demands? Can the partner’s tool do anything your API requires?

Example: authentication – there are many different methods:

  • oauth (2) with all XX minutes token refresh
  • HTTP Basic Authentication
  • SAML
  • HTTPS Client Certificate Authentication
  • ….

Example: status change – there are various requirements for different systems:

    • e.g. three HTTP requests for a status change:
      • Ticket lock
      • Create a Worklog
      • Assign ticket
      • and then there are systems in which these 3 transactions may not be sent “too quickly” in a row.
  • One REST transaction
  • Transaction details via URL parameters, attributes via multipart / form data in the body

Further there are just as many different methods for transferring attachments:

Some systems only allow attachments to be created directly when creating user data and not when there is a change of ownership – they have to be pending for so long
Some systems require uploads in HTTP multipart format
Some systems require uploading a binary octet stream

  • There are systems that require a three-stage upload to an Amazon AWS S3 bucket
  • Some systems only allow attachments to be created directly when creating payload and not when there
    is a change of ownership – they have to be queued for so long
  • Some systems require uploads in HTTP multipart format
  • Some systems require uploads a binary octet stream

Detect errors correctly – synchronous or asynchronous?

Recognize the errors in the partner system correctly. Temporary errors such as Connection Refusal, Read Timeout, Connection Timeout, temporary server errors and workflow-related errors such as the differentiation of incorrect field values in order to be able to react differently. While temporary errors have to
be queued, errors such as unknown values lead to further process steps so that the tickets do not diverge.

Some systems only deliver the error asynchronously. For example, your tool sends an HTTP transaction that is always acknowledged with HTTP 200. Then they receive an acknowledgment with OK or error back on their server via a webhook and have to react to it.

Queue for messages in the event of an error

The order of transactions per ticket is extremely important when linking tickets. It is clear that a ticket update must of course not go out until a ticket has even been created. Sounds easy, but is often a problem if, for
example, the ticket creation fails due to a temporary error and the ticket update has to be queued until the creation is complete.

If you use a single queue, all transactions for other tickets are behind it (and wait). Proper implementation should include one queue per ticket so that pending transactions do not paralyze the entire system.

Or e.g. there are systems in which a ticket status change consists of three HTTP transactions, e.g. B.: Status- Update to “In progress”, then create a worklog entry and then define the assignment group. It can happen
that the ticket is locked and the status-update fails. Here all 3 messages have to be queued until the ticket is “unlocked” again.

Message queuing for asynchronous communication

If a ticket is only acknowledged with an asynchronous confirmation with a ticket number (the ticket number of the partner system), all messages (of the first system) must be held back as long as there is no ticket ID (from the partner-system / -tool) is available, on which the ticket could be updated.

Your system is no longer an independent system

As soon as you let your system interact with others, your system is no longer an independent system (a stand-alone system). It is now much more a “microservice” of another higher-level system. You need to take this into account when making any changes to your own tool. The more interfaces they have, the higher the test effort for changes, of course.

Please include all your interface-partners in your integration-tests and system-integration

Lomnido SIAM Broker

You are the service specialist and know your process best. All APIs and technical things were mastered by the Lomnido SIAM broker from the start. Before you implement all of this on your tool, you should rely on a professional!

Click Me