Anypoint Datagraph: an even faster solution to leverage your Application Network and everything you need to know before using it

Everybody knows that Mulesoft with its several connectors, assets, templates, tools and so on, in combination with a good API modeling, basically following API-led principles, results in an impressive high performance when it is necessary to create a new API. It aims to reduce the delivery gap that exists from IT sectors because of the fast technology evolution.

Despite that, when there is a new requirement where the current APIs in you Application Network does not meet, it is necessary some additional work in order to meet this.

Rest APIs are consumed one API at a time, so depending on the requirement, it is necessary to write multiple requests in order to retrieve the necessary information that is, your ProcessAPI will orchestrate some requests on SystemAPIs or even other ProcessAPIs in order to return the expected result to the consumers.

Another "problem" is that for each request, you will get all the fields described on the API contract (raml, swagger..) and the consumer sometimes doesn′t need all of them, in this case, some additional transformations need to be created in order to remove unnecessary fields. We call the excess of information "over-fetching". This "additional work" can bring complexity and take time to be executed.

We need to move from "some additional work" to "NO additional work" phase. That′s where Anypoint Datagraph comes to play.

But before we talk about Anypoint Datagraph, let's understand the concepts and technologies behind of it.

GraphQL

"GraphQL is a query language for your API, and a server-side runtime for executing queries using a type system you define for your data. GraphQL isn't tied to any specific database or storage engine and is instead backed by your existing code and data." Definition by https://graphql.org/

GraphQL is not a programming language, it is a specification of a language used to query application servers that implements this.

Clients use the GraphQL query language to make requests to a GraphQL service. The service will receive the Query and will orchestrate the calls among the mapped data sources (APIs, database, files and so on) that will return the queried information.

A GraphQL service is created by defining types and fields on those types, then providing functions for each field on each type.?

The following types are defined by GraphQL:

Query and Mutation Types

Every GraphQL server has a query and may or may not have a mutation type.

These are very similar to other object types, but they have a special function defining an entry point of every GraphQL query.

Query type as the name suggest, is used to fetch information from the mapped services and the Mutation type is used to create or modify data.

Example of query

query {
     user {
        id
        name
        address {
           street
           number
        }
     }
}        

We interpret this as a query to retrieve the id, name, street and number from an User.

The result would be something like this:

"data" : {
    "user": {
        "ID": "6872364826843263"
        "name": "Josue Nogueira",
        "address": {
           "street": "My street address",
           "number": 1234
    }          
}        

It is possible to pass parameters to the queries, so if you need to retrieve the above information from a specific user, you could send a query like this:

query {
     user(id: 12345) {
        name
        address {
           street
           number
        }
     }
}        

The schema to define this query, would be something like:

type Query {
    user(id: ID!): User
}        

The schema to define the User type, would be something like:

type User {
   id: ID
   name: String
   address: Address
}        

And the Address:

type Address {
    id: ID
    street: String
    number: int
}        

If you are familiar with Typescript, you will get used to do this quickly.

IMPORTANT: these definitions were made using Javascript and the structure will vary according to the programming language you are using. Check the supported programming languages here .

Scalar Types

It comes with a set of default scalar types: Int, Float, String, Boolean and ID. ID is a special type that represents a unique identifier used to refetch an object as the key for a cache.

There are another types like Enums, Interfaces, Unions and more. For more details, check the documentation .

Resolvers

We have seen so far that GraphQL has a schema where types are declared to be used on the queries or mutation. Scalar types that are similar to primitive types on traditional programming language and special types like Query and Mutation that defines the entry point of the server.

You may asking yourself: "ok, but how exactly the server knows where to retrieve information from, in order to create the result of the query?". That exactly what resolvers do.

Resolvers are functions used to, as the name suggest, resolve the types declared. Let's see how it works creating a resolver for the types we have seen so far.

Query : {
    user(parent, args, context, info) {
       // your code to fetch user information. JSON User object type
    }
}        

So, on the implementation of the method, it is up to you to define where to fetch the data from. You basically need to return the structure according to the type declaration and the GraphQL server will return only the information contained on the query. If there is a need to remove any field returned from the APIs, the server will do it automatically.

You don′t need to create a resolver to return an object full of its properties. When there is a relationship of types, sometimes you need to call another API in order to complete the object, however, in this example, if the client application doesn′t include address information on the query, you will do an unnecessary call to AddressAPI.

You can create a resolver only for the Address type declared on User:

Query : {
    user(parent, args, context, info) {
       // your code to fetch user information. JSON User object type
    }
    

},
User: {
    address(parent, args, context, info){
       //your code to fetch user′s address
    }
}        

Declaring the types this way, the API to return the address information will be called only if the Query contains information related to the address object. Images below to represent this.

In this first example, the client application is requesting only for ID and name, from User object. Both informations are found on UserAPI

This second one, the client application is asking for Address information as well so, the GraphQL knows that these information are resolved by calling another API.

So, no over-fetching and unnecessary calls here.

Advantages of GraphQL:

• Consumers application can retrieve information from a single endpoint in a single request
• No over/under-fetching
• Evolution of the solution without API Versioning

Now that you know the very basics of GraphQL, let's talk about the main reason this article was created for.

Anypoint Datagraph

Anypoint Datagraph leverages the power of GraphQL on Anypoint Platform to enable consumers to query APIs on the Application Network dynamically, from a single endpoint.

Every time you add an API to your Application Network, you can add this API to DataGraph which will store a graph of metadata in a single schema.

How it works?

Stay calm. You don't need to do all the stuffs that I've shown during this article in order to create a schema to the GraphQL Server of Anypoint Platform. No need to write types, neither resolvers. Guess what? Anypoint platform does it automatically.

Let's consider the previous APIs mentioned here: UserAPI and AddressAPI. Considering API-Led Connectivity approach, these APIs are Systems API.

The RAML files of these APIs would be something like:

UserAPI:

In our example, UserAPI stores an ID of the address, not a whole JSON Object.

AddressAPI:

Once we have our APIs published, let's add to Anypoint Datagraph. On the left-side menu, select DataGraph.

Once it opens, you can select one or more APIs you want to include on the Unified Schema, allowing consumers to query information from this API via GraphQL.

After selecting, the next step(click on "Next: Configure URL"button) is to confirm the API Version and the endpoint of its implementation. In our case, we are mocking the APIs, so I informed the mock URL.

Now, you can click save and proceed to the next step to configure the security mechanisms of the API you are adding. There are few policies. Select the one that is applied to the API you created and go to the next step. In our case, No Auth is required.

Now it is time to have a look on the schema generated from this API.

Something familiar here? Anypoint generated automatically the Query type, our entry point! Even more... it created the attributes of the query type that are being called methods. Right below the Query type on the left side, you can see that it also created a User type. Everything based on the RAML file!

We can represent this Query type in a javascript code like:

type Query {
   users: [User]
   usersById(id: String!): User
}        

The explanation indicates that the attribute is required to be passed.

Remember the endpoints of UserAPI? Yes, it converted these endpoints into methods of the Query type and generated the name accordingly.

• https://<base_url>/users generated a users() method
• https://<base_url>/users/{id} generated a userById() method

Note that only the HTTP GET methods from the API were generated here. It means that DataGraph allow only queries, not mutations.

After reviewing the schema, go to Edit Schema, which is the next step.

In this step you can remove/add any method from the Query type or any field of the other Types. If you remove a field, it won't be included in the unified schema. There are other configurations that we are going to see in a few seconds.

Once you finish the changes that you believe are necessary in terms of omit or show methods or fields, it is time to add this schema to the unified one. After clicking on the button to add this to the unified schema, it will take some time in order to publish it.

Do the same for the Address API. One thing you will realize is that the Query type will show more methods. It will merge the UserAPI generated methods with AddressAPI generated ones. Unified schema and one single endpoint, remember?

Once it finishes, it is time to test it. Go back to the overview menu item and click on "Explore Schema" them, on the top right, click on "{;} Run a Query" button.

On the first time you try to run a query, DataGraph will ask you to inform a client application or a Client ID + Secret to manage your access. The same concept of external application to consume your APIs, when you configure it on the API Manager.

Let's create a new one:

After filling the required fields, click on "Create & Request Access". Once I am the admin of the environment, I will get access instantly and will be redirected to the Query page, another nice feature.

You can test the queries directly on the platform and explore the schema as well. If you press "Ctrl + Space" you can see the options to write the query. Let's see it working.

Worked like a charm! The request was sent to the single endpoint and the result was according to I expected from my query.

But wait! What if I need to query for the street name and number for this user? How can I do that? There is only the ID of the address in my User object. Do I need to perform another request to Address API get it? The answer is NO and let's learn another feature.

Enable Collaboration

On the GraphQL session, when we were modeling the schema, there was a relationship between User and Address object directly. In our APIs, the relationship is made by an "foreign key" from user to address. We need to "join" the AddressAPI in order to get more detail about it for a specific user.

Anypoint DataGraph has a feature called "Enable Collaboration" that indicates that a specific Type from the schema can be used to enrich the query, providing information to another type.

It is clear on the documentation :

To enable collaboration on a type, you must configure a default query method and a primary key for that type to access its fields.

The default query method for an object type is the method that returns a single record of that object type for which you want to enable collaboration. In our case, the method "addressById" that returns a type Address, is exactly what we need.

Go to the side panel and click on "List of APIs added" menu. Click on AddressAPI and them, click on Address type. Click on "Enable collaboration" button.

Now you need to select the default query method for this type. This method will be called to enrich the response when one of the fields from this type are included in the Query.

Click in "Next". Now you need to inform what field represents the primary key of this type. In our case, the primary key of Address type is "id".

After selecting, click in "Confirm" and update the Unified Schema applying the changes.

Now it is time to create a relationship between UserAPI and AddressAPI.

To do that, edit the schema of UserAPI. Go to User type, scroll down until a section called "Link to another type". When you click on the drop down menu to select the target type, you will see the Address type and its corresponding primary key. Only types with "Enable Collaboration" option enabled, will be shown here. Select Address type.

After selecting, you need to inform which field from User type, is the "foreign key" of Address type. In our case, select "idAddress".

That’s it! Our User type is now linked to Address type and the field "idAddress" will be used to identify the corresponding address from Address type, using the its id field! If you go back to User type, you will see that a new field called "address" of type Address is shown and "idAddress" is hidden. Make sense to be hidden because this information already exists on the new field. If you prefer, you can make it visible again.

Time to test our changes!

In this example, I will test the "usersById" method. To see it working and its performance, there is a nice feature on this query editor where you can trace the query response time. Click on "Trace query" button and execute a query.

The query I sent basically retrieves the name and Id for a specific user with Id "1111". Have a look on the trace.

Now, if you press "Ctrl + Space", you will see the new attribute "address" that will return a type Address. Inform the fields and execute the query again.

Note that, now you can see on the trace that the Address API was executed in order to retrieve the details of the address for the user.

If you prefer, you can test using Postman or another Rest Client of your preference. To discover the endpoint, click on "Copy endpoint" button. The following screen will be shown with all the necessary information to send a query to the single endpoint. The GraphQL standard is that the endpoints ends with "graphql".

Let's copy the information and execute via Postman. One thing that I haven't mentioned: despite you are calling the API to retrieve information, where REST principles suggest us to use HTTP Get method, to consume GraphQL endpoint on DataGraph, you must use "Post" method. And that is the result:

Merging types

Sometimes more than one API of your Application Network will have information of the same entity.

For example: an account on Salesforce may have its representation on SAP as well, which complementary information, linked by an external ID. Probably you will have two Systems API that will generate two methods on the Query type.

To merge it in a single method, calling the APIs only when a specific field on the query is informed, you can merge the types on DataGraph. To do this, edit the schema, scroll down until "Merge with another type" section and select the type you want to merge. It is not the case here, but make sure to use this option if it is your case, to build an even smarter solution.

Important considerations

This is the end of this article and I′m listing some important considerations I understand you need to know:

• Take a look on the documentation and understand the restrictions and limitations of DataGraph
• Before suggest it to your customers, understand the requirements first
• A good API modeling, following API-Led Connectivity approach will result in a good application network to be consumed using graphql. Do not think in DataGraph first, think in modeling your apis correctly
• DataGraph isn't a silver bullet. It isn't the end of ProcessAPI or ExperienceAPI. Use carefuly.

Hope you enjoyed. ??