What do you do if your company is struggling to select a data warehousing platform?

I'd recommend starting by delving into the specific needs and goals of the business. Understanding the nature of the data, its volumes, and the intended analyses and reporting is crucial groundwork. This insight forms the foundation for making an informed decision when choosing the appropriate data warehousing platform. Following this, conducting proof of concepts (POCs) where feasible can provide invaluable clarity and confidence in the decision-making process.

I'd suggest understanding the specific needs and objectives of the business. What kind of data are we dealing with, what are the expected volumes of data and What types of analyses and reporting will we be undertaking? Understanding the requirements will always help to make an informed decision when selecting the right data warehousing platform. After this, setting up some POCs where possible can also help a great deal in making the correct decision.

Understand the nature of your data (structured, semi-structured, or unstructured), the volume of data, the frequency of data updates, and the complexity of your analytics needs. Align these requirements with your business objectives and growth plans.

Conduct a thorough assessment of your organization's data needs, including data volume, variety, velocity, and sources. Define clear business objectives and use cases that the data warehousing platform should support. Engage stakeholders from various departments to gather requirements and understand their specific data analytics needs.

In my opinion; this is part of feature list that i will evaluate in data warehousing platform to choose one: > IA/ML built-in to delivery some results. > Security: Access control on column, table and row, network constraint, permision and roles management, backups, etc. > Privacy features: can use anonymization techniques? > Operational processing: CRUD operations, Views, CTE, compression, indexing, etc. > Integration & development capacity: can integrate with external and internal service?, Can use SDK or API way to develop around it?, etc. > Available: How replace failed component?. > Another capacities: what table format can use, how distribute the data on storage, What way to process the data, etc. These points help you to choose.

Research and evaluate multiple data warehousing platforms, considering factors such as scalability, performance, security, ease of use, and integration capabilities. Create a checklist or matrix to compare the features and benefits of each platform against your organization's requirements and priorities. Consider additional features such as built-in analytics, machine learning capabilities, and support for real-time processing, depending on your business needs.

Cloud vs. On-Premises: Decide whether a cloud-based or on-premises solution is more suitable for your organization. Cloud data warehouses offer scalability, flexibility, and ease of management. On-premises solutions may be necessary if you have legacy databases that don’t integrate well with cloud-based warehouses or if regulatory requirements mandate an on-premises approach

Research and compare different data warehousing platforms based on their scalability, performance, security features, ease of use, integration capabilities, and support for advanced analytics such as machine learning and AI.

When choosing a data warehousing platform, first map out your current and future needs. Check if the platform can scale with your data and if it's compatible with your existing tech stack. Performance is key—ensure it can handle your queries efficiently. Also, factor in adaptability to new tech trends, total cost of ownership, and security capabilities. Don't forget to evaluate your team's expertise and the potential need for specialized resources. Balancing these considerations will help you select a platform that meets today's needs and tomorrow's growth, ensuring you stay competitive.

Understand the long-term costs associated with the chosen platform. Consider licensing fees, storage costs, and maintenance expenses. Factor in the total cost of ownership (TCO) over time

Consider the upfront costs, ongoing maintenance costs, licensing fees, and any additional costs for features or services. Evaluate the long-term benefits and ROI of each platform.

Analyze the pricing models, licensing fees, and total cost of ownership (TCO) for each data warehousing platform under consideration. Consider not only upfront costs but also ongoing maintenance, support, and scalability expenses. Evaluate the trade-offs between cost and features, ensuring that the selected platform provides the best value for your organization's budget and long-term goals.

Conduct proof-of-concept (POC) tests with a subset of your data to evaluate the performance, ease of use, and compatibility of each platform with your existing systems and tools. Involve key stakeholders and data experts in the evaluation process.

Vendor reputation and reliability: Evaluate the track record and reputation of each platform vendor, including customer reviews, industry recognition, and service-level agreements (SLAs). Vendor ecosystem and support: Assess the availability of third-party integrations, community support, and vendor-provided documentation, training, and technical support. Regulatory compliance and data governance: Ensure that the selected platform complies with industry regulations and data privacy standards relevant to your organization, and provides robust data governance and security features.

Look into the availability of support resources and a strong user community. Robust support ensures timely assistance when issues arise. Active communities provide valuable insights, best practices, and troubleshooting tips

Scalability: Ensure that the platform can scale with your data and business growth. Compatibility: Check if the platform integrates seamlessly with your existing data sources, BI tools, and other systems. Security and Compliance: Evaluate the platform's security features, data encryption capabilities, and compliance with relevant regulations (e.g., GDPR, HIPAA). Support and Training: Consider the availability of vendor support, documentation, training, and community forums. Vendor Reputation: Research the reputation and track record of the vendors offering the data warehousing platforms. Future-proofing: Choose a platform that can adapt to evolving technology trends and future business requirements.

Mostly in early stage the organization needs to understand business needs, the current stage we are in with 3V’s volume velocity and variations of data. Secondly we need to decide on on premise or cloud based warehouse based on need ,scalability,latency ,resources ,existing contracts and security aspects.Later comes the tradeoffs based on cost and dire need to either opt for Snowflake , Aws redshift ,azure synapse analytics and others considering the fact of integration options, cost per compute or storage , security , easy deployments. Databricks could be explored despite being a data lake due to its ml capabilities companies need to be strategic in this aspect given the aspects of emerging technologies and proliferation of ml models.