Unified Data Analytics Platform is a must for Successful Analytics Projects
This post in in support of using a proper low code Unified Data Analytics Platform - I am writing here a copy book approach that will work always.
Even if one is developing the most complex solution a Low Code Platform is always the best (esp. the one which allows white labelling). A Platform will push first phase in production in 6 months and after that it is free of cost as it will give the return! I have seen following approach works and we have done it multiple peta byte scale implementations on our platform?
1. Do a POC for 3-4 weeks to understand the mindset of a customer. Gauge their problem statement, Maturity, Infrastructure etc. with their expectations and try to give them a quick flow using the approach you want to take. This gives lot of ideas & understanding of problems going to come ahead.?
2. Start the MVP phase with actual Production. At this point, charge the customers at least Dev License + Efforts. Focus on making the Data Lake working. Choose most common 3-5 Ingestion sources and push into the Raw Zone quickly. Your data architects would see new kind of messed up data and they need to use
i) Data Preparation Module + write SQL or Python scripts to further enrich it - Use Low Code Mode of Platform through Script writers?
ii)Push the partly enriched code into a Data Virtualization layer to understand the data (should be provided by Platform itself)
iii) Use Self Service Reports to build 10-15 quick reports in 3-4 days to start discussing about the data, expected KPIs, Look n Feel etc. with Customer
iv)Review core Pipelines which is getting real time, batch, micro batch data by doing basic Integration tests
V) Start working on your NLP, Computer Vision, Regression etc. models that are applicable using the Data Science Lab (Should be part of the platform)?
Vi) Start building some of the complex dashboards and show the design to customer teams
vii) Close on the requirements and stabilise the system
viii) After UAT, Integration Tests - Start Load Test & focus on Deployments and User Training
ix) Monitoring, Maintainability will be the Key as soon as you start the production run or Parallel run to Production?
This should be over in 4-6 Months. The production Env should give ROI from this moment itself.?
Once MVP is in Production?
Extensibility of Platform is critical feature that is required
领英推荐
Start phases of the actual Project - Divide in multiple phases of 3 months each?
Every quarter more Databases are supported
More Visuals are created
More Models are pushed into production and their accuracy should be enhanced through Feature Store
All Should happen in the Same Platform which is white labelled. User can be created and use training can be given
Platform should be able to work on multiple cloud platforms + On Premise (together)
Create Different kind of Tenants and extend the solution for Monetisation?
This way internal teams, R&D teams, Quality teams, Finance, HR etc. can be given analytics along with external ecosystem like Partners, Vendors, Re-sellers, Component manufactures, Governments etc.?
A true Unified E2E Platform should be able to shorten your overall execution by 50% - 75% of time as compared to using broken tools (50%) or self development of everything (75%).?
With a right kind of platform - we need to have true Data Champions designing the data model. This is a section if done wrongly, will cost very high in the end.?
Customer involvement is always critical esp. they’re Domain experts who validates what’s being built. Engage them quickly and engage them with visualisation.?
Ask them to play with your outputs. ?
A True Platform has all the modules -