You’re struggling to find the right data discovery solution for your BI project. What should you do next?

Na minha opini?o, compreender profundamente o negócio e suas necessidades específicas é um passo crítico antes de mergulhar em qualquer projeto de BI. Isso é crucial, BI é poderoso, mas sua eficácia está diretamente ligada à clareza dos objetivos que se deseja alcan?ar. Sem um entendimento sólido do que o negócio precisa, você pode acabar coletando dados irrelevantes, ou pior, interpretando os dados de maneira errada, levando a decis?es mal informadas.

In addition.. Integration with existing platforms and packages is also a key thing to keep in mind. Training is one more aspect that come to my mind

Finding the right data discovery solution for our business intelligence (BI) project starts with defining our requirements. Identify specific use cases, necessary data sources, and understand the needs of our end-users. This ensures we select a user-friendly solution that aligns with their expertise and our project goals. Conduct thorough research to shortlist potential solutions using industry reports, user reviews, and vendor websites. Consider total cost of ownership and potential ROI to make an informed decision. Implement a pilot project with the shortlisted solution to assess real-world performance and gather user feedback. Review findings to choose the best fit for our requirements, budget, and strategic goals.

Define your objective to start with discuss with the stakeholders and lock your requirements and document the same in the form of an SRS. Evaluate the data sources and ensure data access and security. Define how you will arrange your data. Then you compare and chose various solutions available based on your business size, complexity and budget. Test, validate and implement. The job is not over yet, keep on improving by monitoring on a regular basis. Also ensure that a proper framework for data governance is in place to reduce data breaches and enhance security with data catalogue, data lineage. Make good use of data collaboration tools to keep everyone on the same page. In all this process, keep you stake holders involved in every stage.

My approach is pretty straightforward- Talk to stakeholders and confirm the requirements. It will give you clarity how data modeling has to be done and what KPIs required. Data extraction pipeline and condition depends whether real time data required or near real time (from replica DB) Write Business logic to extract data, confirm retrived records and all features (columns) Note - Avoid data extraction on every page refresh until required data can be fetched in milliseconds. Make analysis view , get this confirmed with stakeholders. Note : Title should be meaningful for all panels and sheets. Further you can create one pre prod instance and Prod instance of dashboard.

Uma boa prática é utilizar o O Crisp-DM (Cross Industry Standard Process for Data Mining) é uma metodologia padr?o para projetos de dados. Ele fornece uma estrutura compreensiva, dividida em fases como, compreens?o do negócio, compreens?o dos dados, prepara??o dos dados, modelagem, avalia??o e implementa??o. Utilizar o Crisp-DM ajuda a organizar e guiar eficientemente o processo de minera??o de dados, garantindo uma abordagem sistemática para resolver problemas complexos e extrair insights valiosos.

Depois da primeira fase bem feita, agora é hora de entender onde encontrar os dados para realizar aquela análise! Esta em SQL Server? No Oracle? Excel? Em API? MongoBD? Documente ao máximo todas as fases!

Todo projeto de BI come?a com o levantamento de requisitos, é com ele que identificamos os principais indicadores, e com esta informa??o inicia o mapeamento das fontes de dados, que é uma etapa fundamental, pois é aonde iremos buscar os dados, tem banco de dados? API? será necessário fazer Web Scraping? Todas essas respostas podem tornar um projeto viável ou n?o. E após a extra??o desses dados avaliar o volume de dados e com base nisso come?ar a pensar na arquitetura que será implementada.

One of the key aspects here is really the necessity behind the large volume of historical data that is necessary to train the AI. This section delves into the complexities around the "data quality" traits (completeness, accuracy, consistent), but, I want to emphasize volume. If your business does 200 sales transactions per year for the last 10 years, is 2,000 sales records enough to extrapolate predictability in buying? Or is your business to niche to know the "emotional" aspects of a purchase that can't be represented in data. A lot of folks think, I'm turning AI on it's a whiz-bang impact overnight. The analytics rely on that substantial history accumulated over time.

For building right data solutions you need to understand your data well , data profiling is something you need to do , evaluate data sources and then take informed decision

To find the right data discovery tool for your BI project, start by defining your needs, budget, and technical requirements. Research and shortlist tools like Tableau, Power BI, and others, considering user reviews and case studies. Utilize free trials to test their fit with your workflow. Create a comparison matrix focusing on features, pricing, data connectors, security, and support, prioritizing based on your project's specific needs. Consult with BI experts and vendor communities for deeper insights. Choose the solution that aligns best with your requirements and budget, ensuring it enables your team to extract valuable insights and make informed decisions. This thorough comparison process will guide you to the right tool for project.

Em posse dos dados e do entendimento da necessidade do cliente, de como ele gosta de visualizar as coisas... chegou a hora de procurar ferramentas que atenda a necessidade! Quando possível, compare ferramentas de mercado (levando em conta além de funcionalidades, o seu custo e benefício para o negócio em quest?o!) E falei acima quando possível, pois muitas as vezes já tem a ferramenta que teremos que utilizar! Hoje vejo muito no mercado, Power BI, Looker e Quicksight

Identify and list the key features and functionalities required for your BI project. This may include data visualization, integration capabilities, scalability, user interface, and security features. Consider the user-friendliness of each solution. Examine the scalability of each solution to ensure it can handle your current data volumes and accommodate future growth.

After identifying your needs and establishing a methodology to prioritize them, it's essential not to overlook the resources you already have at your disposal. Frequently, organizations possess solutions that are part of a broader suite of products, not all of which are actively utilized. Therefore, I recommend engaging with your vendors to fully understand the scope of services and products you're currently paying for. You may discover that the solution you seek is already available to you.

Crafting large-scale strategic solutions demands thorough vetting. Always test 3-4 options with a sample dataset to assess their performance and integration capabilities. Key factors include connectivity, seamless integration, response times, computational strength, scalability costs, security, and access management. eg Azure and Fabric are known for implementation of large-scale solutions and are considered one of the best solutions for end-to-end model. But in my experience, I have witnessed it struggling with SAP connections and integration. Hence, it's very important to compare different solutions for your use case before finalizing.

Na minha vis?o... testar solu??es de descoberta de dados é crucial para garantir que escolha a mais adequada. Ao simular cenários reais com seus próprios dados, você avalia o desempenho, usabilidade e capacidade de resolver problemas reais. Comparar resultados com expectativas e obter feedback dos usuários finais s?o passos essenciais para garantir a eficácia da solu??o selecionada. Em resumo, testar é fundamental para tomar uma decis?o informada e garantir a satisfa??o das partes interessadas. Valeu!

One important factor is to have a benchmark or baseline for each core factor, as most solutions come with their limitations, thus keeping a benchmark pattern based on organizational needs will ease the testing evaluation.

Criar cenários realistas e avaliar o desempenho da solu??o em termos de explora??o, análise, visualiza??o e colabora??o de dados é fundamental. Além disso, comparar os resultados com as expectativas e solicitar feedback dos usuários finais ajudará a tomar uma decis?o informada e garantir a eficácia da solu??o escolhida. Esta abordagem orientada por dados e feedback é essencial para o sucesso do projeto.

Now testing and validating narrowed-down data discovery solutions with your own data and use cases is a crucial step. Creating realistic scenarios aligned with your objectives and challenges allows you to assess each solution's performance in terms of data exploration, analysis, visualization, and collaboration. Measuring results against expectations and benchmarks, along with gathering feedback from stakeholders and end-users, ensures a thorough evaluation of how well each solution meets your project's specific needs and user satisfaction.

There is no substitute for quality testing. Hopefully, your organization has an independent testing function. This is crucial for all types of testing. When developers test, they focus on whether the solution does what was intended. In addition to this, independent testing groups try to break the system. What happens when unexpected data is encountered, or users do not follow the defined process? Thorough testing increases confidence.

Escolha sempre a solu??o que melhor atendeu as necessidades do negócio, independente se ela é sua preferida ou n?o! Custo e benefício ao cliente! Foco nisso!

Good approach and good additional comments. I would add make sure all stakeholders have visibility. Companies can have multiple teams working in silos modeling and analyzing data.

Upon the completion of testing and validation, it’s time to select the optimal data discovery solution. It’s crucial several factors: -Technical Aspects: Evaluate the technical features and functionalities thoroughly. -Business Value: Assess the potential business value and the impact that each solution may have. -Implementation Strategy: Develop a comprehensive plan for deploying the chosen solution. -Training & Support: Ensure you have a plan for training users and providing ongoing support to stakeholders. -Continuous Evaluation: Commit to regular monitoring and evaluation of your solution to ensure it meets your needs. -Iterative Improvement: Be prepared to make necessary adjustments and enhancements based on feedback and performance.

This is one of the most important aspects you can find the right tool and budget the appropriate amount of money to spend but if you can't implement both the vendor and your company are spending time, resources and money on something neither can use or bill for.

This is good general direction around the decision matrix driving the conversation and path forward. Don't glance over too quickly the training of the end users and stake holders. If you build a model, solution that the business can't take over, maintain, adjust quickly in response to market needs, etc., then you're going to create too much overhead maintaining a solution and loose agility to development cycles. The BI project may not yield the ROI expected if the time to value is impeded this way.

You should start with why you are “struggling”? - Do you have the right team that understands sourcing, modelling and visualisation of data? - Do you have common data sources? - Since you are struggling, it’s very likely that you don't have one. Then do you have the data governance in place? Where are the sources of truth for duplicated data across the organisation? Who owns them? - Who will be the key users of this solution? What are their management level and intelligence needs?

When choosing a data discovery solution for BI, consider the full data journey, focusing on long-term scalability and security. Investigate the vendor's ecosystem, including support, community, and integrations with existing tools. Evaluate training resources for smooth team adoption. Don’t shy away from hybrid or open-source solutions to meet specific needs, and consider pilot projects for practical evaluation. Prioritize solutions that address your business challenges, involve stakeholders to ensure buy-in, and remain adaptable in your search, refining your choice based on insights gained throughout the process. This comprehensive approach will guide you to a decision that supports your BI project’s success.

BI is not just business intelligence but also considered as better intelligence Tools are matter of help which help you to frame data It is more important to see on how best you can work with it Try be a person who can use anytool anywhere

Aqui só refor?o um ponto... fazer as perguntas certas é essencial para direcionar a busca pela solu??o de descoberta de dados adequada. Isso envolve entender exatamente quais s?o seus requisitos e desafios específicos, para que possa avaliar as solu??es em rela??o a essas necessidades. Ao fazer perguntas bem formuladas, você pode garantir que está focando nos aspectos mais relevantes e encontrando a solu??o que atenda aos seus objetivos de BI. #perguntascertas

Em minha vis?o, a implementa??o bem-sucedida do BI demanda n?o apenas a escolha adequada das ferramentas, mas também uma cultura organizacional que valorize a análise de dados. é crucial fomentar uma mentalidade orientada por dados em toda a empresa, incentivando a colabora??o entre as equipes e promovendo a transparência no compartilhamento de informa??es. A tecnologia do BI se destaca plenamente quando integrada a uma cultura que valoriza e compreende a importancia estratégica dos insights derivados dos dados.