Why Use Web Services?

I have been working with PeopleSoft since 2000 AD 👴. I work with varying levels of technical and functional staff. I often see teams reach for “flat file” integrations and SFTP servers to transmit data. I don’t think flat-file integrations will ever go away. However, I always try to avoid that approach if both sides of the integration can handle it. I often have clients fight me on using integration broker and web services. The reason is that their staff may not know what web services are or the advantages of using them. Or they tried to use PeopleSoft web services and ran into one of the many rough edges and gave up because the documentation is so horrible. Hopefully, this book will help with this last road block. 😀

Let’s discuss some reasons why web services are better than flat files and FTP servers. Let’s look at an example where a file is being sent into PeopleSoft for processing. Here is a rough overview of the FTP flat-file components.

What could go wrong and what pieces are there to maintain?

• Flat file integrations have more pieces and servers involved. The more pieces you have in a solution, the more chances for support and security issues. At large organizations, this also means more departments have to get involved like security, server admins, and change control teams.
• With this approach you have to deal with SFTP servers and credentials.
  • How do you handle production versus non-production instances? Is this a different directory or server or credentials? Not every vendor can handle every method.
• What happens when a scheduled process fails to read the file or a user fails to run the process?
• With this approach the vendor has zero visibility or feedback if the file was processed and what data issues there were.
• How are bad data elements in the file communicated back to the vendor? With flat-file integration there has to be some back channel that is often manual.
• How do you safe guard against double-file processing?

Let’s look at the components when a web service is used.

• The vendor knows if the data submitted was processed. A “signal” is sent back in real-time if the file was processed.
• Production versus non-production is often just a different URL.
• Bad data is communicated back in real-time.
• No reliance on scheduled processes and the data is processed as soon as the vendor pushes it.
• There are fewer pieces involved.

Design Shift - Multiple Transaction to a Single Transaction

With Flat File integrations, you often see one file with many transactions that are processed. You could have a “clean” file and all the transactions are processed correctly. However, you often run into situations where a few rows in a file cannot be processed correctly. As mentioned above, this requires some design and a contract between the client and server on how those issues are communicated back.

When you migrate to a web services approach, it is often beneficial to design the web services around a single transaction. This is not a requirement, but you will find it beneficial in the design. It often simplifies the “contract” between the client and the server.

For example, Let’s say that you have an inbound file that that triggers the creation of user accounts in your PeopleSoft system. If we imagine that you have a PeopleSoft Finance system and some other HR system. The HR system tells the Finance system that new accounts need to be created when someone is hired.

• In a Flat file integration, the HR system may send a file nightly with the 20 new people that got hired and need an account. The file will be picked up based on a scheduled interval. When the finance system processes the file, all 20 accounts are created.
• In a web services design, you could have the HR system send all those new users in one POST to the service. However, it can be problematic if there is some issue with one of the employees in the file. From a design perspective, it is often better to have the service geared around one transaction. So that would be one new user account. The return would signal success or failure of the account creation. So the HR system would have to invoke the web service once for each new account. The HR system would get a “signal” back on each web service call if it was successful or not. If there is some error the HR system may queue that up for a re-attempt or it could add it to some administrator workflow or notification.

The web service could easily be designed to handle multiple transactions. The issues arise around partial failures. If you design a good message structure where you can signal back partial failures it will work just fine. However, I have found it is often the most simple approach to gear your services around one transaction. So if this is a web service that is geared around a person like updating an address or adding some data the service would take one EMPLID at a time.