Industry trials are conducted on behalf of the entire industry, ensuring that all Communications Providers (CPs) benefit from the progress and results.
Here are all the lessons learned throughout industry trials, this page will be updated regularly.
Here are all the lessons learned shared by all trial participants.
We regularly update this page.
1. Responding to the Hub
When the Hub pushes a message to the intended recipient, the recipient should return a 202 in standard http format to the Hub to confirm receipt. If the recipient returns a 400, 404, 501, 502 or 511, the Hub will stop attempting to deliver the message and will return a message delivery failure to the sender of the original message. If the recipient returns any other response to the Hub (or no response), it will continue to retry in line with the published Message Delivery Policies until the message either times out or one of the previously mentioned codes is returned.
2. Validating messages prior to responding to the Hub
The Hub validates the message envelope before accepting a message from the sender and pushing it to the recipient (please see details in the API specification). The Hub checks that the message is a valid JSON file, but does not validate the content of the message body in any way. The process and Hub design was based on the principle that users would perform only a lightweight validation on the message before accepting. However, we have noticed through integration testing that a lot of users are validating message bodies before accepting them and have then rejected them with a 400 rather than accepting them and responding with an asynchronous response message. This rejection causes the Hub to return a message delivery failure to the original sender of the message, but it does not inform them why the message was rejected. The Industry Process Group have published a number of guidance bulletins around accepting messages without over validating which are all available here.
3. Rejecting messages with a response
While it is best practice to accept messages and respond asynchronously whenever possible, there may be circumstances where users will need to reject a message by returning a response that causes the Hub to stop delivering a message – e.g. a 400. When this is returned, it is recommended that users provide text for the reason that the message was not accepted along with the http 400 response. This will then be recorded in the Hub’s logs and the technical ops team will have access to that response which will help them work with the sender and intended recipient of the message in order to diagnose what the issue was.
4. Accepting messages within the Hub’s delivery attempt
The Hub will attempt to deliver messages in line with the Message Delivery Policies. It has been observed that many users are not accepting messages within the Hub’s initial delivery attempt which causes the Hub to retry the message until the message is accepted, rejected or the delivery period times out. The Industry Process Group have published Bulletin 67 on this topic which suggests a target time of 200ms to accept a message from the Hub.
5. Accepting message delivery failures
When the Hub is unable to send a message to the intended recipient, either because the recipient has rejected the message or because the message delivery policy has expired, the Hub will return a Message Delivery Failure to the sender of the message. Please note that these messages will be sent from a source identity of “TOTSCO” and they will not contain a source correlation ID, unlike the OTS messages. These messages cannot be replied to and are just notification from the Hub. Details are contained in the API Specification section 2.1.8.
6. Time Zones
While the TOTSCo Hub uses UTC for all of its internal timestamps and logs, this is entirely separate from the timings that the Hub users will operate within when they conduct their business. We expect users to operate within the applicable local time, be that BST or GMT, regardless of the Hub’s logs being in UTC. On this basis, if one user sent a message at 00:30 BST on a Monday morning, the recipient should treat that message as being received at 00:30 BST on a Monday morning rather than 23:30 UTC on a Sunday evening – even though the Hubs internal timestamps and logs will display the time in UTC.
7. Correlation IDs
Every request type message will contain a source correlation ID and every response (confirmation or failure) type message will contain a source correlation ID and a destination correlation ID – the destination correlation ID should mirror the source correlation ID that was sent in the request message. The Industry Process Group’s Bulletin 67 advises that it would be best practice for these to be unique, where possible, to decrease the possibility of any confusion when correlating responses to messages. However, it also states that users should not perform duplication detection on the basis of correlation ID and RCPID. If this is implemented, messages may be rejected or go unactioned by the recipient if the sender were to reuse a correlation ID. Reuse of the correlation ID is permitted as per 5.10 of the Industry Process.
A correlation ID has a limit of 256 characters. If this is exceeded, the Hub will reject the message with a 400 bad request synchronous response (currently being resolved as a defect).
8. Empty strings in optional message fields
There are a number of fields in the OTS message specification that are optional. It has been observed in testing that most users are simply omitting the field from the message if no value is provided, although some users have been including the field in the message with an empty string as the value – for instance in a residentialSwitchMatchRequest message including “account”: “”. While no formal statement has been provided to state that it is best practice to omit any optional fields without values, the Industry Process Group have published a Message Examples document in which this is evident throughout.
In the event that a message is received with an empty string in an optional message field, it is recommended that the recipient does not reject the message, but accepts it and tries to process it as if the field had been omitted. In the example of an empty account in a match request, it is recommended that the recipient neither rejects the message with a 400 nor returns a match failure on the basis of an invalid account, but instead proceeds to attempt the match on the basis that no account details were provided.
9. Assist URLs
The TOTSCo account management portal gives users the opportunity to add two optional URLs that will appear in their record in the directory – a customer assist URL and a sales assist URL. While this is an optional field, it has been noted that, so far, only a few users have chosen to utilise it (though we understand others are working on their solutions). For more details on this field and how best to utilise it, please refer to section 4 of Matching Best Practice Guide available here.
10. Ensuring credentials are valid before interacting with the Hub
We have noted that several users do not seem to be tracking the expiry of their credentials and are sending messages to the Hub after the credentials have expired. This causes the Hub to reject the message and return a synchronous 401/900901 error message and the recipient is then prompted to renew their credentials. It is recommended that users record the expiry of their credentials and renew them before attempting to interact with the Hub. Please refer to section 3.2.1.1 of the API Specification for more details.
