How to integrate ZOHO CRM with a 3rd party Database thru REST API

Case Study:

BigBite (a fictional entity :p) is an f&b company that serves fast food Deliveries to its Clients base. Client's records are managed on ZOHO CRM under Contacts module.

In order to safeguard these clients data, BigBite would like to backup the Contacts records to its local database, the backup must be in real time, whenever a record is created/edited in ZOHO CRM, it should instantly be sent to database.

BigBite might need to use these Records for other internal business processes managed by other apps.

BigBite IT guy would like to avoid using paid integration services such as "Zappier" to be cost efficient.

Constraints:

• ZOHO CRM integrates strictly thru RESTAPI
• BigBite is using a local mongodb that has no REST API end points exposed

Structure:

Assume the following model for a Contact record:

• ZCRM_ID [record id generated by ZOHO CRM]
• first_name [string]
• last_name [string]
• email [string]
• mobile [number]
• phone [number]

IT Admin will be using Mongodb as a data source (you can use other database such as MySQL, MsSQL)

Step 1:

I will be using Loopback4 framework to build the API micro-service that will expose mongodb. On local server install the following:

• NodeJs/NPM (you can find installation process from relevant pages)
• Loopback4 framework (Loopback4 is a framework built on top of express, it helps you build an API service as easy as 123, you can extend LB4 to be your core back-end and you can be linking your app to lot of data source options. LB4 is downloadable from loopback.io)
• mongodb (or you can have mongodb on a different instance to mitigate risk of "single point of failure")

Step 2:

After finishing step 1, create the integration app by following below steps:

1. open cmd for windows (or shell for linux)
2. create a directory (mkdir) in your desired folder.
3. cd to created dir
4. create the app:

lb4 app

1. enter desired project name
2. enter description
3. enter root directory of the app
4. enter application class name (usually kept same as suggested for better syntax alignment)
5. keep all features selected (you can opt out the docker file, I usually do)
6. cd into the root directory

Now time to setup the app:

1. we need first to define the data source by using the following command:

lb4 datasource

1. enter source name
2. select the desired connector for the database you are using, in our case will select mongodb since that's what we mentioned above, you can choose other connectors such as mysql, mssql, redis etc.
3. if you have the connection string ready you can enter it, else
4. enter host [the IP of the mongodb ]
5. enter port [u can keep it empty it will automatically take 27017]
6. enter user [username]
7. enter password
8. enter database name [ this is the name of the database created on mongodb where the records will be stored, I will name it Contacts]
9. Now that the data source is defined, time to define the model, a model is the entity that forms the record created in database, in our case it is "Contact", you can refer to the Contact model structure described above.
10. in the root directory, enter the following command:

lb4 model

1. enter model class name: contact
2. select model base as "entity"
3. "allow free form properties" select No
4. now time to create the properties of the model, enter property name: zcrm_id
5. zcrm_id is a string auto generated by zoho therefore it should be flagged as the ID property, it shouldn't be generated automatically in mongodb, hence generate automatically must be false, and it should be required
6. Do the same for the rest of properties, make sure you select the right property type
7. keep default value empty for all properties
8. Now that we have created the Model we need to create the repository for that model, a repository artifact is responsible to manipulate (CRUD) the related model table defined on the data source, enter the following command:

lb4 repository

1. select the related data source
2. select the model (in that case contact model)
3. select the base class as defaultCRUDrepository
4. not last but least you need to create the relevant controller of the model, a controller manages any business logic related to the model, in our case it will include the different routes and methods for the RESTAPI , enter the following command:

lb4 controller

1. enter controller class name (for better alignment name it is same as model name , in our case "contact")
2. select controller type as REST controller
3. select the model contact
4. select the model repository
5. enter "id" as the as the ID property of type string, it should not be omitted when a new instance is created since the id is the primary key of a contact record created in ZOHO.
6. set the path of CRUD operations as the plural format of contact, in our case it is /Contacts
7. assuming you have followed above steps correctly, test your app by running:

 npm start

1. go to the ip of the instance on which you are doing the setup using port 3000 , click on explorer, and Voila , your APP with proper API end points to fire

Step 3:

Now its time to secure the app to prevent every Tom, Dick & Harry to abuse your application APIs, it is essential by ZOHO to have a secure connection with 3rd party apps, in our case we will be using JWT.

1. Using your best text editor [ I use vscode ] open the the root directory
2. install the loopback JWT package with npm using the following command:

npm i --save @loopback/authentication @loopback/authentication-jwt

1. we need first to bind the authentication components to our app, open the "application.ts" file
2. we need to import the following libraries:

import {AuthenticationComponent} from '@loopback/authentication';

import {JWTAuthenticationComponent,SECURITY_SCHEME_SPEC, UserServiceBindings,} 
from '@loopback/authentication-jwt';

• we also need to import the datasource class name created in datasrouce.ts under data source folder:

import { [class name here]} from './datasources';

• next we need to include inside the constructor the following functions:

this.addSecuritySpec(); // it will add the security specs on the API this.component(AuthenticationComponent); //it will mount the authentication system this.component(JWTAuthenticationComponent); //it will mount the JWT components this.dataSource([class name here], UserServiceBindings.DATASOURCE_NAME);         //"[class name here]" is the class name defined in the datasource.ts 

• outside the constructor we need to define the "addSecuritySpec" function as below:

addSecuritySpec(): void {

                             this.api({

                                openapi: '3.0.0',

                                       info: {

                                     title: 'test application',

                                     version: '1.0.0',

                                       },

                                     paths: {},

                           components: {securitySchemes: SECURITY_SCHEME_SPEC},

                            security: [

                                             {

                                            // secure all endpoints with 'jwt'

                                              jwt: [],

                                            },

                                           ],

                         servers: [{url: '/'}],

                 });

        }

• next step we need to define the authentication process in the sequence.ts file, a sequence is a group of actions that control how the app will respond to requests, open the sequence.ts file in root directory and add below snippet code:

import { AuthenticateFn,
 AuthenticationBindings, 
AUTHENTICATION_STRATEGY_NOT_FOUND,
 USER_PROFILE_NOT_FOUND, } 
from '@loopback/authentication';             
                                                                              //this imports proper functions from authentication library

• next we need to inject the authentication binding parameter in the constructor :

@inject(AuthenticationBindings.AUTH_ACTION) protected authenticateRequest: AuthenticateFn,

• then we need to add the following line in the async function right after the findRoute method:

await this.authenticateRequest(request);                                                                  //this will enable the JWT authentication and call the authentication action

• in order to flag errors coming from JWT authentication failures, we need to add the 401 unauthorized statement in the error catch as follows:

if (

                   err.code === AUTHENTICATION_STRATEGY_NOT_FOUND ||

                   err.code === USER_PROFILE_NOT_FOUND

                    ) {

                     Object.assign(err, {statusCode: 401 /* Unauthorized */});

              }

• Now we need to create User Controller in order to be able to create the bearer token, we will use a typical controller used in many apps that will provide the following API end points:

1. Signup // will allow us to signup for a user that can generate relevant token to use the Contact API end points
2. Login // use the credentials created at signup to generate token
3. whoami //will be used to get the user info

use the following command to create the User controller: lb4 controller , then name the controller as "User"

• make sure you create an empty controller, then open the relevant ts file and replace the generic code with the one below:

import {
	  authenticate,
	  TokenService,
	  UserService,
	} from '@loopback/authentication';
	import {inject} from '@loopback/core';
	import {get, post, requestBody, getModelSchemaRef} from '@loopback/rest';
	import {SecurityBindings, securityId, UserProfile} from '@loopback/security';
	import {
	  TokenServiceBindings,
	  UserServiceBindings,
	  User,
	  Credentials,
	  MyUserService,
	  UserRepository,
	} from '@loopback/authentication-jwt';
	import {model, property, repository} from '@loopback/repository';
	import {hash, genSalt} from 'bcryptjs';
	import _ from 'lodash';
	

	@model()
	export class NewUserRequest extends User {
	  @property({
	    type: 'string',
	    required: true,
	  })
	  password: string;
	}
	

	const CredentialsSchema = {
	  type: 'object',
	  required: ['email', 'password'],
	  properties: {
	    email: {
	      type: 'string',
	      format: 'email',
	    },
	    password: {
	      type: 'string',
	      minLength: 8,
	    },
	  },
	};
	

	export const CredentialsRequestBody = {
	  description: 'The input of login function',
	  required: true,
	  content: {
	    'application/json': {schema: CredentialsSchema},
	  },
	};
	

	export class UserController {
	  constructor(
	    @inject(TokenServiceBindings.TOKEN_SERVICE)
	    public jwtService: TokenService,
	    @inject(UserServiceBindings.USER_SERVICE)
	    public userService: MyUserService,
	    @inject(SecurityBindings.USER, {optional: true})
	    public user: UserProfile,
	    @repository(UserRepository) protected userRepository: UserRepository,
	  ) {}
	

	  @post('/users/login', {
	    responses: {
	      '200': {
	        description: 'Token',
	        content: {
	          'application/json': {
	            schema: {
	              type: 'object',
	              properties: {
	                token: {
	                  type: 'string',
	                },
	              },
	            },
	          },
	        },
	      },
	    },
	  })
	  async login(
	    @requestBody(CredentialsRequestBody) credentials: Credentials,
	  ): Promise<{token: string}> {
	    // ensure the user exists, and the password is correct
	    const user = await this.userService.verifyCredentials(credentials);
	    // convert a User object into a UserProfile object (reduced set of properties)
	    const userProfile = this.userService.convertToUserProfile(user);
	

	    // create a JSON Web Token based on the user profile
	    const token = await this.jwtService.generateToken(userProfile);
	    return {token};
	  }
	

	  @authenticate('jwt')
	  @get('/whoAmI', {
	    responses: {
	      '200': {
	        description: '',
	        schema: {
	          type: 'string',
	        },
	      },
	    },
	  })
	  async whoAmI(
	    @inject(SecurityBindings.USER)
	    currentUserProfile: UserProfile,
	  ): Promise<string> {
	    return currentUserProfile[securityId];
	  }
	

	  @post('/signup', {
	    responses: {
	      '200': {
	        description: 'User',
	        content: {
	          'application/json': {
	            schema: {
	              'x-ts-type': User,
	            },
	          },
	        },
	      },
	    },
	  })
	  async signUp(
	    @requestBody({
	      content: {
	        'application/json': {
	          schema: getModelSchemaRef(NewUserRequest, {
	            title: 'NewUser',
	          }),
	        },
	      },
	    })
	    newUserRequest: NewUserRequest,
	  ): Promise<User> {
	    const password = await hash(newUserRequest.password, await genSalt());
	    const savedUser = await this.userRepository.create(
	      _.omit(newUserRequest, 'password'),
	    );
	

	    await this.userRepository.userCredentials(savedUser.id).create({password});
	

	    return savedUser;
	  }
	}

• next we need to make sure the contact controller is decorated with the JWT authenticate by adding " @authenticate('jwt') " above the class e.g.

@authenticate('jwt')                                                                                                export class ContactController {...

• If you need to exempt an end point from authentication, just add the following decoration above it:

@authenticate.skip()

Step 4:

Testing the App:

1. run the app with "npm start" , and browse to https://[ip]:3000/explorer/
2. go to the the "Signup" end point and signup for a User
3. then go the "Login" end point and login with created User, in the response body you will see a token generated, copy it and browse up to "authorize" button, enter the token and click on "Authorize".
4. Now your secured end points are ready to be used, try the post end point to create a new contact, if it is successful and reflecting in your database that means "It works!"

Step 5:

Setting up connection on ZOHO CRM to invoke the Contact Created into the Database.

1. go to "developer space" >> connections , and create a custom service
2. after naming the service, select auth type as API key, & param type as "header"
3. add header key "Authorization" (same in display name)
4. name you connection, and click on "create & connect"
5. a popup window will open, enter the generated JWT token and submit.
6. the service will have status "connected"
7. go to settings >> workflow rules, and create a new Rule.
8. select Module "Contacts", name the rule , and enter some description (optional)
9. for the "when" condition, select on record action >> Create.
10. select "All Contacts" for condition
11. then for "instant action" create a custom function
12. add the following code in the function:

continfo = zoho.crm.getRecordById("Contacts",zcrm_id);

contact_info = Map();

contact_info.put("zcrm_id",zcrm_id.toString());

contact_info.put("first_name",continfo.get("First_Name"));

contact_info.put("last_name",continfo.get("Last_Name"));

contact_info.put("email",continfo.get("Email"));

contact_info.put("mobile",continfo.get("Mobile").toNumber());

contact_info.put("phone",continfo.get("Phone").toNumber());

header_data = Map();

header_data.put("Authorization","Bearer [jwt token]");

header_data.put("Content-Type","application/json");

response = invokeurl

[

	url :"https://[ip]:3000/contacts"

	type :POST

	parameters:contact_info.toString()

	headers:header_data

	connection: [connection name here between ""]

];

info response;

Note: make sure you define "zcrm_id" as argument mapped to "contact id" in "Contacts" module.

Repeat the above workflow creation for the "on record edit" scenario and create a custom function with below code:

continfo = zoho.crm.getRecordById("Contacts",zcrm_id);
contact_info = Map();
contact_info.put("zcrm_id",zcrm_id.toString());
contact_info.put("first_name",continfo.get("First_Name"));
contact_info.put("last_name",continfo.get("Last_Name"));
contact_info.put("email",continfo.get("Email"));
contact_info.put("mobile",continfo.get("Mobile").toNumber());
contact_info.put("phone",continfo.get("Phone").toNumber());

header_data = Map();
header_data.put("Authorization","Bearer [JWT token]");
header_data.put("Content-Type","application/json");
response = invokeurl
[
	url :"https://[IP]:3000/contacts/" + zcrm_id
	type :PUT
	parameters:contact_info.toString()
	headers:header_data
	connection:[connection name here between ""]
];
info response;

Note: make sure you define "zcrm_id" as argument mapped to "contact id" in "Contacts" module.

You can also create a 3rd work flow to trigger the deletion of the record by using above function code with "type:DELETE" in the "invokeurl" array.

I hope this post would be beneficial for you, stay tuned for more.

Cheerz,