#DataHub #DataRequest

The Data Request (also known as Scope of Interest) is a new key functionality offered in the last major releases of Neoxam Datahub. It is actually a broad set of deep functionalities, covering multiple use cases. I was really enthusiastic to analyze it and to share high level views and feedbacks here.

What are the use cases covered ?

1. Data exports

This use case is the easiest one. The "subscription" datarequest allows to define a scope of objects (agents, securities...) with classical scope definition or with SQL, it leverages a linked feedproc to define the export mapping. The datarequest scope definition overrules the feedproc scope. It is based on an orchestration scheduled task to be launched. This datarequest will generate a "brother" client request (with the objects of the scope listed one by one for every launch – 2344 in the picture below) and will ultimately generate the desired export files or data flows.

Let's be honest, this use case is already addressed with a feedproc, its feedmap and its scope definition, and a launching task. The use of datarequest does not really bring specific added value. No surprise because it leverages only a small part of what datarequest can offer. But it helps to understand some principles and capabilities.

2. Second Use Case for DataRequest : agent/securities creation with providers (static data)

This use case is far more rich :

• you may request different fields depending on the security you want to create
• you may request issuers
• you may request the underlying security in case of a derivative

All these points are covered by Datahub capabilities for years, but you need to create, set up and coordinate multiple components. I can say the Datahub product team intention was to facilitate that coordination, with datarequests. Quite a challenge, indeed.

Here I have to introduce the vendor strategies :

• it defines the feedproc for each security type
• it allows multiple steps, the most interesting situation is : the first provider request to query the security type, the second one specific to the security type returned by the provider

Other sophistications can of course happen, such as querying multiple data providers or dealing with underlying.

In the case of derivative, the principle is to instantiate a new datarequest to deal with the underlying creation. Of course the derivative has to be linked with its underlying. The security is created in 2 or 3 steps, and after the first steps the new security has to be “identified” and associated to the datarequest. Unfortunately this is managedby Datahub compiled code, meaning you can’t see nor modify it. During all the creation process, datarequest security creation status has to be updated. I am always pushing for robust state machines, and here I have some caveats about status based on indirect assumptions.. No doubt these 2 points can (will?) be improved.

In the case of derivative, the datarequest derivative status should take into account the underlying creation status, to avoid orphan derivative creation without any alert.

In this second use case, a single datarequest can cover one or multiple securities -if they are needed at the same time-. But the way of use is to instantiate a new datarequest each time you want to query one or multiple securities. This is a difference with the first use case (one datarequest executed multiple times).

3. Third Use Case for DataRequest : Collect and distribute market data (dynamic data)

Marketdata can be quotes and quote linked data (volumes, greeks...), yield curve, volatilities, ... In this use case, you may consolidate multiple marketdata needs from different internal clients:

• you typically create a "client" datarequest for each client
• all datarequests are linked to the same scheduled task
• all datarequests are linked to a vendor strategy (the same), that defines the feedproc to collect and acquire provider data (i.e the field to get from the provider and the mapping of these fields). Again, the vendor strategy allows to define multiple feedproc depending on security types, to query and acquire different fields depending on security types.

When the task is launched :

• all "client" datarequests scopes are aggregated into a unique vendor request
• the vendor request is transmitted to the provider (based on the vendor strategy)
• depending of the jobs in the scheduledtasks, marketdata are sent to each client, according to each client scope

The last 2 use cases (acquisition and diffusion of static and dynamic data) are crucial for master data management. There not doubt the Neoxam DataHub "data request" offer makes a lot of sense.

That said, the variety of use cases, the number of concepts to articulate (security types, multiple feedproc, multiple scopes definition, files names with providers, files names/data flow for internal clients) make the understanding and the use of datarequests really complex :

• client data request/vendor request/subscription request : a client request can instantiate a linked vendor request or/and a subscription request
• in some use cases, the same data request is executed multiple time, in some other cases, you create a new datarequest for each data need
• the fields (and tables) of data request object reflect this complexity : on the fly table for agent/securities creation, . A UML purist would not approve.
• the datarequest create a new level of coordination above scheduled tasks : parameters are passed from datarequest to feedproc and it's abstract, scheduled tasks are launching other tasks, again very abstract and makes job report quite unconfortable to understand (I wish for a "datarequest level" execution report)

From my experience of supporting a number of NXDH clients, it would be a challenge for them.

Then, is it worth to implement or to move to datarequest ?

For use case 1, no.

For clients with existing process to collect/distribute static & dynamic data (the use cases 2 & 3), they should have already created processes based on feedproc, and I see no cost-benefit advantage to move to datarequest.

For new clients, datarequests have to be considered. I recommend my Neoxam friends to make this component more easy to use and to understand.

?