You’re struggling to understand your data. How can you find the right data visualization tools?

In my experience with working with data I have two approaches: 1. Small data: Use spreadsheet if possible, excel is still an excellent tool for data analysis and visualization. If you can aggregate and get insights on your large data and put that into smaller set and then later visualize it that's excellent 2. Big data: use tools like PySpark to process and then visualize using python or if dashboarding is required then use PowerBI or Tableau So knowing your data is really the first step in this whole process, how big it is, what is the frequency of incoming data. Static vs Dynamic data (streaming). For streaming data the dashboard approach is more suitable due to it's inherent benefits and automation capabilities.

When you have a relational data, it is always better to explore the relationships among the entities within the data. Any tool that helps you realise the data in terms of a graph/network can be used. For example Neo4J is a GraphDB which can be used for this purpose.

By gaining a deep knowledge of data, you can make informed decisions about the most suitable visualization techniques and tools. For example, numerical data may benefit from histograms or scatter plots, while categorical data may be effectively represented using bar charts or pie charts. Time-series data might require line charts or heatmaps, and spatial data could be visualized using maps or geospatial plots. Knowing your data allows you to tailor your visualization choices to effectively communicate insights and uncover meaningful patterns, ultimately aiding in a more comprehensive understanding of the underlying information.

Knowing your audience is extremely important. By knowing you need to know the technical knowledge and hierarchy of the person among other things. If the audience is technically sound, then you have multiple options and usually can make the visualization more interactive and hacky. Otherwise keep it as simple and less cluttered as possible. Secondly, the importance of use of this information to them and other people. If the person is very senior level and will make crucial strategic decisions based on your work, then it needs to be really concise, to the point and self explanatory and easy to understand and use.

In my opinion, a fundamental step in navigating data understanding and choosing the right visualization tools is to distinctly "know your purpose." It's crucial to articulate specific objectives—are you after trends, patterns, outliers, or relationships in the data? This clarity becomes the North Star for selecting visualization tools. For illustrating data distribution, my go-to choices are histograms or box plots; when exploring correlations, I lean towards scatter plots or heatmaps; and for conveying proportions, I find pie charts or bar graphs effective. Having a clear purpose ensures that the chosen tools align seamlessly with goals, providing a strategic roadmap amid the diverse landscape of available options.

There are tons of options available and as explained in prior sections it depends heavily on type of data, audience and your purpose in deciding which one to use. I will share a few: 1. Sheet based: excel, google sheets, etc. Small data, mix audience, internally used 2. Pro Dashboarding: Tableau, PowerBI etc. Big data, mix audience, internally or externally used by many people on daily basis 3. Notebooks with code: again multitude of python packages, matplotlib, plotly, etc. big data, technically sound audience, used for experimentation or internal purposes One final way is to present findings using PowerPoint or other similar tool. That's for more strategic audience and purposes.

There are some guiding principles to follow in the field of Visual Analytics. The Visual Information-Seeking Mantra [Shneiderman, 1996] is the best place to start with. One can also look into the concept of 'SenseMaking' in Data Visualization when it comes to realizing pragmatic data visualization.

In addition to understanding your data and defining your purpose, consider data quality, audience understanding, interactivity, temporal aspects, scalability, ethical considerations, storytelling, accessibility, feedback loops, and the iterative nature of the process. Documentation is crucial for future reference and collaboration. These factors collectively enhance the effectiveness of data exploration and visualization.