Integration Broker Terminology

This section will introduce some terminology to the reader. You can reference back to this section as needed.

PeopleTools

A PeopleSoft application is built on standard PeopleTools technologies. You can think of PeopleTools as similar to the Microsoft .NET framework or Java. It is a development platform that all PeopleSoft applications like HR, Finance, and Campus are built on top of. As part of any PeopleSoft application built with PeopleTools, you get the integration broker built-in. There are several supported integration technologies to communicate with the application. The Integration Broker being the one of focus in this book.

Integration Broker (IB)

The PeopleTools Integration Broker (IB) is a complex module that enables web services in PeopleSoft. It facilitates providing web services to outside systems and acting as a client to outside services. The “Integration Broker” is a delivered as part of every PeopleTools installation. There are software components on the web servers, application servers and setup tables that work together. All these pieces together make up the “Integration Broker” (IB).

The module has many parts that a PeopleSoft technical administrator must configure and secure to allow a third party to receive or post data to the application. As far as the third party is concerned, they post XML to a URL and a result is returned and some work is done. The integration broker supports asynchronous messaging and synchronous request/reply patterns. The IB is also responsible for sending communicating with external web services from PeopleSoft. If a PeopleSoft database needs to communicate to an outside system over web services or other protocol, the IB handles this interaction.

We will discuss each pattern in this book. Like any software, The IB has its pros and cons. It is not perfect.

Here is a high-level schematic of the IB pieces.

Integration Gateway (PSIGW)

The Integration Gateway (PSIGW) is a Java servlet installed on the web-server. It is the external facing component of the Integration Broker (IB) that the third party interacts with.

• For inbound messages, the client passes parameters and data to Listening Connectors to handle the inbound message and pass it to the back end application.
• For outbound messages, Target Connectors are use to communicate to the external systems in different protocols like: PeopleSoft internal (MIME), HTTP, JMS, SMTP, SOAP, etc.

The actual application logic that handles inbound messages or invokes outbound messages sits on the application server which the gateway can connect to.

• For inbound messages, Handlers are responsible for returning data to the client.
• For outbound messages, some code inside the application is initiating the call and parsing the result and using it inside some business logic.

Every PeopleSoft installation is configured differently based on the companies infrastructure.

• The gateway can be setup in a manner that one common gateway (URL) is handling and forwarding integration messages to many different back end PeopleSoft databases. In this shared gateway setup, the third party must specify the “destination node” that it is sending messages to.

• Alternatively, each PeopleSoft application could have it’s own gateway. This is highly dependent on how the system administrators have chose to configure the application.

• The gateway is typically in the DMZ or a firewall product proxies requests for the gateway so systems can make connections.
• If you wanted to post XML over HTTPS to the integration gateway you would use a URL in this form: https://[yourhost]/PSIGW/HttpListeningConnector
  • The PSIGW in the URL stands for “PeopleSoft Integration Gateway”. The “HttpListeningConnector” is a connector which we will discuss later.

For inbound messages, the gateway receives the request, performs some basic validation and routes the request to the correct PeopleSoft application server which runs all the code to parse the message and update the database.

Target and Listening Connectors

The gateway has “listening” and “target” connectors. These are just pieces of code that can speak different protocols and transform that into something that PeopleCode can read and process in a protocol agnostic manner. For Examples

• Client –> HTTP –> MIME –> PeopleCode Encoding
• Client –> JMS –> MIME –> PeopleCode Encoding

The HTTP connectors are pretty well documented and I have personally used them in many different integrations.

Connector PeopleBooks Documentation

What does this mean to the 3rd party?

• For inbound into PeopleSoft, the target connector is in the URL you submit to https://[yourhost]/PSIGW/HttpListeningConnector. The HttpListeningConnector expects xml data.

• For outbound from PeopleSoft, PeopleSoft will be using the target connector and will need to post to an http web-server.

• Listening Connectors

• Target Connectors

Node

A node is a object in the IB framework that represents an integration endpoint. These only apply to non-REST services. For REST, the node object does NOT come into play. Every remote PeopleSoft application has a unique node in the system. For example, if you own HR and Finance from Oracle each of those instance will have a node that represents each other. Additionally, every third party application interacting with PeopleSoft generally has a node defined. This allows security to be setup so that only certain third-party applications can call a subset of the hundreds of APIs that exists in a PeopleSoft database. Again with REST, nodes do not.

Message

The PeopleTools Message object is really just a representation of the message structure for a request, response, or fault. Messages have a schema tied to them and they are associated with a Service Operation. This is the lowest level object to build an integration into or out of PeopleSoft. It typically defines the message format. There are different message types:

• Rowset-based: This is a format that easily maps to PeopleSoft in-memory data structures. If you use this format it makes it easy to publish messages by copying in-memory page data objects into the message. You don’t have to write code to format the data. You just call a copy method from an in memory pointer to the message object.
• non-rowset based: These message formats are typically xml formats. Code has to be written to generate or parse the XML.
• Document: A message based on a Document object.
• Container: A message that is composed of different “part” messages.
• Rowset Part: This is a “building block” type that allows you to create a “part” that composes a larger message object. For example, you may have a “container” message with many rowset parts.

Documents

The Documents “technology” was introduced in an 8.5x release. A portion of the document technology has to be used in REST operations. I would avoid using the document technology as much as possible as it is very hard to work with and is poorly documented. 👎 It is also riddled with bugs. 🐛 🐛 It looks good if you just read the documentation. However, the actual implementation is poor. With REST services you have to use them to parse the REST URL parameters. You don’t need them to parse the payload of the message.

You have been warned!

Service

A service is a grouping of service operations. It really provides zero functionality other than grouping them.

Service Operation

A Service Operation (SO) is a PeopleTools name for consuming and providing web services. A Service Operation brings together different types of objects and layers of the application to enable web services. There are different types of operation types that can be used based on the interaction model that is required. Here are the different types of Service Operations in the order of importance and frequency of use.

• Synchronous
  • This is the standard request/response model. For inbound service operation, the handler PeopleCode is executed and the calling application waits for a response to the request. The PeopleSoft application will invoke some code while the applications hangs on the wire and the PeopleSoft system is updated or an error is returned. The PeopleCode could do SQL updates or invoke a CI.
• Synchronous REST Service Operations
  • This is PeopleTools version of request/response using the REST paradigm.
• Asynchronous one-way
  • This is PeopleTool’s version of a queueing integration. It is a “fire and forget” service where the publisher posts a message and the only response required is that the receiving system received and logged it. They are not actually guaranteed to run any code in PeopleSoft (for inbound) or they may never be published to external systems (for outbound). They could be canceled by an administrator for whatever reason. You use these types of messages when the other system does not need real-time information. Asynchronous integrations are better for decoupled systems as the target systems can be down and the messages will queue up and be retried when the connection is re-established.
• Asynchronous to synchronous (not commonly used)
• Asynchronous request / response (not commonly used)

We will discuss the first 3 in-depth in this book.

Service Operations have their own place in the PeopleSoft object hierarchy. Here is a high level perspective on the objects.

• A service operation “belongs to” a service
• A service operation has at least one routing
• A service operation has messages associated for the request and response
• A service operation can have many different types of handlers
• A service operation is invoked by submitting a request to a listening connector on the integration gateway.
• A service operation has one default message version
• A service operation has 0 to many non-default message versions

See the Examples Section for some start to finish examples.

Security Options

On inbound integrations from a PeopleSoft node, the PeopleSoft system looks for a user ID to associate with the permission list set for a service operation in the following order.

1. Authentication token.
2. Default User ID.

On inbound integrations not from a PeopleSoft node (External nodes and third-party systems), the PeopleSoft system looks for a user ID to associate with the permission list set for a service operation in the following order.

1. External Name/External Password.
2. External User ID/External Password.
3. Default User ID.

Routings

Service Operation routings are defined between two nodes. You define a publisher and a receiver. For example, if we wanted to publish messages from HR to Finances, we would define a routing from the HR node to the Finance node. Without this routing, the integration engine would not publish it anywhere. A service operation can have many routings to different nodes.

In the routings, you can also specify a transformation program. This is a specialized application engine that will transform the message from one version to another. It uses XSLT

Handlers

A “handler” defines a piece of code or action to be run during certain publishing or subscription events. A handler is typically defined for inbound service operation but there are cases where you want to run some code prior to a publication contract getting created.

Queue

A queue is “lane” of service operations that are grouped so that they process in a specific order. For example, the PERSON_DATA queue has many service operations. PERSON_ID_CHANGE, PERSON_BASIC_SYNC, PERSON_DISABILITY_SYNC, PERSON_VISA_CITIZEN_SYNC.

The PERSON_DATA queue is all about a person at the EMPLID level. You group these together so that the subscribing system processes them in a logical order.

For example, when an employee is hired into the system, several service operations can trigger during one save. When you create a person generally, the PERSON_BASIC_SYNC is triggered and the PERSON_DISABILITY_SYNC is also triggered.

Queues can be partitioned by a key field. This allows an error for one EMPLID to partition off and stop messages for the same EMPLID to get queue up. However, messages for different EMPLID are not held up. If HCM errors on subscribing to a “create Person” operation for a new student, other student’s data will NOT be held up for processing.

Queues can be paused to have the messages build up in the queue. This is generally something that is only done during a known system outage on one of the nodes.

Asynchronous Publishing and Subscribing Steps and Terminology

When the integration broker receives a service operation from either a third party or from a local business event, the asynchronous processing goes through several steps depending on several factors. Asynchronous service operations are not guaranteed to actually process, publish or update any data as we will see.

Relevant Blog Post: High Level Overview of Integration Broker Publishing Steps

Operation Instance

When the integration broker receives a new service operation to process, first an operation instance is created. The operation gets a GUID transaction id assigned. The data is just stored in some integration broker tables and nothing has occurred. The integration broker will send an “I got it” acknowledgment. The operation instance is just a placeholder of an XML request to be processed at some point in the future. A certain server process called the “Broker Dispatcher” running on the application server will look for new Operation Instances and determine what should happen and what other server processes need to know about the message. The server process will look at the setups for the service operation and determine what “subscription contract” and “publication contracts” need to be created (if any). The Broker dispatcher look at the publishing node and determine who to publish the message to by creating “publication contracts” which are actually published later and subscription contracts which are also published later.

The Broker Dispatcher process could be inactivated by an administrator and business events and external systems can continue to post to the integration broker. Those service operations will just queue up and no external system will be impacted. There are times when the dispatcher may have crashed or have been brought down by an administrator.

If the QUEUE for a service operation is “paused” then the service operations in that queue will remain in the “new” status for the operation instance. Operation Instances can be canceled and the publication and subscription contracts will never be created.

Typically, if all the server processes are running, the operation instances process in under a second and the publication contracts and subscription contracts are created.

Publication Contracts

Publication Contracts are entries into publication contract integration broker tables. These get created by the “broker dispatcher” process from the operation instance based on active routings in the service operation setup. If no active routings existed when the broker dispatcher processed the operation instance, then no publication contract will be created. When publication contracts get created they all get their own GUID transaction ID. If there is more than one routing active, there will more than one publication contract created (one for each node). Those publication contracts are process independently of one another. Therefore, if you are publishing to 3 systems, one of those systems can be down and the other two continue to get the messages. The publication contracts bound for the down node will go into error after a number of retries.

Another server process called the Publication Dispatcher actually handles the publication. It wakes up periodically and looks for new publication contracts. This process can crash or be brought down intentionally by an administrator.

If the publication errors anyone with the proper security could actually cancel the contract and the receiving system will never get the message. Sometimes this can be OK if the business event can be re-triggered. Other times this can be bad if there is no way to trigger the business event again.

Subscription Contracts

Subscription contracts are entries very similar to publication contracts but they are for code to run in the local system. These get created by the “broker dispatcher” process from the operation instance based on active handlers in the service operation setup. If no active handlers we found when the operation instance was processed, then no subscription contracts will be created. Subscription contracts get their own GUID transaction id and process independently of each other in their own DB transaction. A server process called the subscription dispatcher actually processes the contract.

Typically, a handler is a piece of PeopleCode that needs to run to look at the message and perform some update or maybe even decide to not update anything. It is up to the handler code. The subscription contract is just an entry in the integration broker system that a piece of code need to be triggered with the data in the service operation. The subscription dispatch is responsible for invoking the PeopleCode handler. The subscription contract code could error out and the subscription contract will go into error status. The error could be correct by massaging the XML in that contract, fixing some setup data or even canceling the contract.

If the subscription contract is in error anyone with proper security could cancel the contract and the system will not be updated or it will have to be updated manually.