Pay Nearby's Data science Stack

We at Nearby Technologies strongly believe in data first methodology before going ahead with any business decision and this is now embedded in our DNA to "test-measure-implement" anything and everything!

We are one of the early fintech leaders who have their own in-house Data Science Team, which enables the business to take decisions on the go with supportive data and intelligent insights.

Thanks to our in-house team of expert Data scientist and analysts, who achieve our complex goals keeping the entire system up and running 24X7

Data is crucial and to make high impact decisions, we at Nearby tech started collecting a huge amount of data from different sources but to put everything in one place there was a need to get a team that can drive value out of the data. This was when the Data Science team came into existence. Two years back we started this journey and today we help the business to use this data in decision making, KPI tracking, Marketing, and sales.

Current System

our present system can be broadly explained under 6 segments

Source :

There are ~200 data sources in different systems!

So, we can imagine the challenge to put all the data in one place and create a single source of records which is just the first step towards building a full-fledged data science stack.

ETL Pipelines:

At Nearby, we ingest ~ 200 Million transactional entries in a month with over 200 active pipelines!

Data security is at the core of Nearby technologies and that is one the reasons we created our ETL platform on GCP. This not only helped us to secure data but also helped us in saving money, which could have been a huge cost if we would have gone for any third-party ETL tool.

Data warehouse and Data Lake:

We use two types of Data Warehouses to solve different use cases.

1. Google's BigQuery - This is a columnar data warehouse tool and is based on pay per scan costing model
2. Cloud storage - Here we store cold data and keep cleaning this frequently

Below listed are a few of the common suggestions, you should take into consideration when setting up and using Bigquery

• Avoid using Select * instead, specify the column you need this method will help you reduce cost drastically
• Use data partitioning cluster method while storing data, this helps in lowering query cost even if you have to scan the heaviest table
• Keep cold data in a separate table as active and cold data is priced differently in GCP

Data Visualisation, Dashboards & Reporting :

We use PowerBI for dashboarding, visualization and reporting. This is the one-stop source for all the product and business metric and it's being used extensively across the organization to track product performance and business health. This enables the Senior management and CXOs to be aware of all the key metrics and also gives them a UI from where based on the filters they can extract metadata.

We also use Automailers to send across many reports direct to the mailbox of the stakeholders using R/python

BigQuery is the single source for all data access needs of business users, the only prerequisite being basic SQL knowledge.

Machine Learning :

We have a dedicated team of Data scientists who works together to make robust systems which help to achieve our business goals easily

Here are a few systems build by our data scientists:

• Product cross-sell system bases on product usage, location, and line of business
• PAN and Adhaar OCR using google vision and ML layer - With this system we were able to read data directly from image thereby reducing human errors
• Churn prediction model -we were able to predict who all will leave the system in next 15 days thereby making retention strategy well in advance which in return helped us to achieve ~85%-90% second-month retention of the agents
• Price optimization
• Market estimation using stacked ML model which helped us in understanding the new area where the market can be created which helped the business to strategize their sales functions
• Sales gamification: What is Gamification for a non-gaming industry?

"The use of characteristic game elements in typically non-game contexts to create gameful experiences that enhance value creation by users "

That's exactly what we did and created a gameful experience for our customers which in turn will increase user engagement. We named it "Leaderboard", it's a platform where we are allowing our customers to earn more and a platform which gives them an adrenaline rush to get in the top in the locality by instilling a feeling of competition.

Business Analytics :

"One size fits all" is something which does not apply to any Data science team as in most of the cases a good machine learning guy might not be good at quick business analysis but would be a great fit to solve the complicated business problems which can not be solved by traditional methods. Keeping this in mind we have a team of business analytics ninjas who are quick like ninjas and help us to be on our toes always.

Here are a few items build by our BA ninjas:

• Entire PowerBI dashboarding for each service (We have ~ 35 dashboards, each tracking ~50 metrics of product/business! )
• Data governance - to ensure the data is in the right format to enable users the get it at any time
• They are the custodians of all the reports delivered in the mailboxes of all the stakeholders on time and in the right format
• Enablers for marketing campaigns because they provide the data and insights for the campaigns and then measure them too!
• Helps business/products to give visibility of the business impact of any change before actually making any strategy/feature change

We are still evolving and are looking for ways to make this setup more robust and ready to handle 10X volume!

Watch this space for more on how we evolve our Data Science stack to meet the demands of a fast-growing business.

Website: https://paynearby.in/

LinkedIn: https://www.dhirubhai.net/company/paynearby/

Data blog post:https://www.dqindia.com/digging-new-oil-well-data/