Data and the dopamine of failure

Hi everyone, hope you found the title inquisitive enough ??

Today, I would like to take you through my ~1 year journey of starting this newsletter - through some numbers.

This is a dual post - both a reflection of personal insights and ideas on presenting data.

How it started

It all started as a new year resolution for 2022. Since I am taciturn when it comes to speaking, I thought I should find a medium to express myself occasionally - through writing.

I evaluated many portals and zero-ed in on https://charann.substack.com as it allowed:

• People to subscribe the blog to their mailboxes (recurring viewers) and
• At the same time, anyone could be read without signing in (minimal inertia to view)

But it did not have a viral angle. I then tried to share the link on WhatsApp status (~100 views), LinkedIn (~2000 views) and Twitter (~50 views).

LinkedIn seemed to be the most relevant medium but felt it didn’t seem to display the post enough as it was an external link. And at the same time, LinkedIn had launched a “newsletter” format with subscribers getting a mail and notification for each post.

Hence, since then I have been posting the content on both Substack and LinkedIn platforms.

For the past one year, even though I tracked the responses, I never seriously tabulated it. So, I thought why not try it from capture to insight and take everyone through the process.

Step 1: Data Capture

Started the process by capturing the following metrics in a Google Sheet (manually typed in from LinkedIn)

For all the fancy world of Big Data, ML, AI & algorithms starts in the “boring” world of actually capturing data in a systematic manner!

The classic example of machine learning to identify cats was possible because someone tagged all the photos correctly to begin with! ??

Step 2: ETL (Extract Transform Load)

Extract - Connecting to the data (server, username, password)

Transform - Create new columns, join multiple tables, summarise.

Load - Load it to a common database for analysis

As it is a single google sheet - there is not much need for it explicitly.

I created a free Tableau Public account and linked my google sheet to it. Please be careful that Tableau Public cannot be used for private data, but it is excellent for learning!

Step 3: Analyse

I wanted to know what worked for me and what didn’t.

Questions:

Is my viewership increasing?

What type of topics were better received?

Is there a pattern to those that have been well received?

Any repeating patterns?

How has my publishing pattern been?

Metrics / KPIs: Since LinkedIn never gives metrics like how many fully read an article or time taken reading etc. The simplest proxies we have are Views and Likes. (There have been hardly any shares or comments to even analyse - statistically insignificant??)

Dimensions: The “dimensions” are those through which we can slice data or run a ML algorithm on. Primary ones for me here were date of posting and an arbitrary categorisation of topics (Data, Personal, Human Behaviour)

Step 4: The Trends which emerge.

Though people tend to like building voluminous and fancy dashboards to show off their visualisation abilities. I prefer to keep it utilitarian. How much I am able to convey with lesser graphs ??

The “bar chart with colours” in Tableau has been my favourite chart. I can convey 2 messages in one graph! For example, a bar chart with revenue growing with profit margin % going in red.

If you see the two graphs below (Link here).

The one on the left is a timeline. The bar shows the number of views, the colour shows the category and the gap between bars shows the gap between my posts.

The one on the right are the top posts. The bar showing the relative proportion of views and the colour showing the category.

Step 5: Insights

What are the insights I could draw?

1. My post on moving from Mumbai to Bengaluru has been the all-time high
2. The “Personal” posts have greater views than the rest.
3. I was more frequent initially and enabled it to get greater reach. Social media platforms like to promote people who post more! ??
4. My viewership seems to have plateaued after the first quarter.
5. The timeline seems to be following a product life cycle graph with the early adopters being the only users ??

Step 6: Action

What action can I take based on the insights.

1. There is no need to always “act” ??
2. Am I interpreting right? There might some other information
3. In my case, I realise I need to be more consistent with my frequency and quality of posts
4. More posts with a personal touch
5. Find ways to increase the reach. I attended a session by Saloni Mehta which gave me more insights to some dos and don’ts while it comes to engaging on LinkedIn

A Few Takeaways

Though I have managed to do all this, I would like to call it a “dopamine of failure” as I don’t think my experiments have shown any better results. I don’t think I have been able to reach beyond my close set of friends and colleagues. ??

But the experiments keep me going with the hope that it will reach its right place.

In the age of social media, there always seems to be a constant challenge between “Impressing a few deeply” or “reaching many virally”. Or in rare cases, both.

The need for virality sometimes might make us compromise on what we are naturally. Click bait posts, the top 5 things you must do, Secret to ever lasting things, etc. always bring more eyeballs. But is that what we need to aspire for?

More the # of views or any other metric, the greatest thing which eggs me on is how I “feel” about writing a post and some great feedback I get from friends that it made them think!

When we enjoy the journey, the result becomes the enjoyment itself ??

I have been an aspiring leg spinner for more than 30 years but would still find it difficult to get into any gully cricket team, but still the quest continues! ??

Thanks a lot for reading this post. Do let me know your feedback!