What emerging trends in statistics should you learn?

When our planet have largest number of Visual Learners (about 65%), it is imperative that we present the Data in a Visually appealing manner. The key objective is to communicate the information to get action. Keeping this in mind Data Visualization is the way to go. I am a strong advocate of Citizen Analyst and Data Democracy, since it helps we humans to evolve to next stage. Tools like PowerBI and Tableau helps people with Zero coding knowledge to become an Analyst and take quick decisions.

Without clear data visualization, even the most insightful findings may remain concealed. To truly harness the power of data, one must not only conduct the analysis but also master the art of presenting it visually. The purpose of your analysis, its significance, can easily be lost in the noise of data if not effectively communicated. Data visualization tools such as Tableau, Power BI, or R act as your instruments to orchestrate this symphony of information. They enable you to shape your data into a coherent narrative but a good visualization isn't just about the tools; it's about having a clear aim and purpose in mind, ensuring that your message is not just heard but understood and appreciated.

Which data trends are "the best" emerging trends in data visualization are constantly evolving, but some key areas to explore include interactive and immersive visualizations, which allow users to engage with data in more dynamic ways. Another trend is the use of storytelling techniques in data visualization, where data is presented in a narrative format to make it more compelling and easier to understand. Additionally, the integration of artificial intelligence and machine learning into data visualization tools is becoming more prevalent, enabling more sophisticated analysis and insights. Overall, staying abreast of these trends can help professionals create more impactful and engaging data visualizations.

Data visualization is simply an effective and efficient way to communicate diverse feelings, impressions, and suggestions to a wide range of audiences. This can be through charts, dashboards, and any diagrammatic presentations that aid towards better communication. This technical skill is needed in almost every industry as it is the foundation in aid for data-driven decision-making.

Apply AI, machine learning, and data analytics. The trend is in visualization to communicate with organization executives who have little understanding of statistical terms

Bayesian models give you posterior distributions as a result, which are so much richer than maximum likelihood estimators. But they are also more complex and computationally intensive. Think twice whether the models are the best product for your audience.

Coming from a frequentist perspective, learning Bayesian statistics can be challenging because it forces you to reshape some of your thinking. Many texts focus on applications and examples where a frequentist approach would be fine; skip those books and find better ones. Still, your eyes will glaze over reading about conjugate priors, likelihood functions, and posterior distributions. Fortunately, that is handled automatically by software nowadays, though you must understand what the software is doing. Among the most common applications of Bayesian methods are multilevel and heirarchical modeling and random coefficient models. In economics, Bayesian methods have proven extremely useful in estimating DSGE macroeconomic models.

Bayesian models highlight the importance of combining existing information with models. More specifically the use of data to inform the models. It is paramount to consider the impact the marginal changes in information have on the outputs.

The Bayesian statistical is capable to model the knowledge accumulation over time in a very natural way (since the posterior of one step is the prior of the next). The Bayesian theory is more simple to understand, although the math involved can be more difficult that the equivalent in the classic statistics. But today there are a lot of softwares that can help you to bypass this mathematical complexity; from analytical calculus software as Wolfram Mathematica to MCMC simulation software as JAGS.

-Importance: Provides a more flexible framework for model estimation and hypothesis testing. -Applications: Economic forecasting, financial risk assessment, and modeling uncertainty.

Before learning about causal inference it’s important to understand why it’s necessary and why causal impact is not the same as predictive impact. For example the feature importance you get from an XGBoost tells you how important a feature is in predicting the target but does not tell you the direction. In order to say Smoking causes cancer you have to keep all else as is. For example if I am taking two groups that are healthy and same age group where one smokes and the other doesn’t then I can see how much smoking alone impacts health. If I see a higher cancer rate among the smokers then i can say smoking causes cancer. However, if the group that smokes already contains people with other diseases then you can’t assess the impact of smoking

- Importance: Moving beyond correlation to understand causation. - Applications: Evaluating the effectiveness of economic policies, understanding drivers of economic events, and experimental economics.

Causal inference is unfortunately a relatively less visited field among statisticians despite its profound significance. Certain times it is the best way to model phenomena, if not the only way. At the end of the day, we live in a deterministic and causal universe; that everything is connected in intercausal relationships like trillions of billard balls. Hint : when implementing instrumental variable methods, ensure the instrument is not only correlated with the treatment but exclusively affects the outcome also, to avoid bias.

Econometrics has been around for a long while (therefore it is not an emerging trend). Nonetheless, in my experience, it is the most rewarding application of statistics, if you are willing to go down this rabbit hole.

Causal inference is the discipline dedicated to the investigation of how to recognize and estimate causal connections between variables. It's a swiftly advancing area with numerous practical uses in sectors like healthcare, economics, and public policy. Causal inference methodologies empower us to comprehend the genuine impacts of interventions and policies, facilitating improved decision-making.

Another consideration for data ethics is the full utilisation of data by making it open access, where appropriate. This can reduce the need for repeating studies where data is already available, helping to save time, resources and burden on participants.

Let’s take election polling for example. In the past few elections for both parties the polls have gotten it wrong. Most polls over the last two elections were way off. The analysis using the data was correct but the collection appears to be flawed. As an example it appears the primary method of polling is using landlines, which tends to be used more by older voters who generally vote for Republicans. Younger audience rarely uses landlines. Last election saw a huge spike in young voters that skew towards the Democrats. Hence the polling suggested big Republican wins but many elections went the other way. This is an example of collecting samples that did not represent the overall voting population correctly.

ETHICS!!!! Are you collecting data? Are your respondents aware of the necessity of this data? Is the response captured anonymously? I believe that data ethics should NOT be lightly taken. For a complete cycle of this principle, the findings from the collected data should be shared by the respective respondents. This is to show openness between you and the respondents

Understanding data ethics is crucial for statisticians and data scientists as it allows them to navigate the complex ethical landscape that comes with handling data. By grasping data ethics, you can effectively address issues such as bias, privacy, consent, accountability, and fairness in your data-related work. It's not just a matter of moral responsibility; it's also a legal and professional obligation to ensure that data is collected and used ethically. To delve into this field, consider following the guidelines and codes of conduct provided by organizations like the ones stated above. Embracing data ethics is not only a trend but a fundamental necessity in today's data-driven society.

Ethics is key... we have a duty to address crucial issues like bias, privacy, consent, and fairness, consistently and professionally...

Open data repositories such as Zenodo @ CERN or figshare.com are excellent tools for reproducible research. They allow you to archive not only code but also various other types of research data. Further, these services provide permanent DOI links to each archive, allowing others to conveniently cite your research results or software.

Reproducible research is a mindset. If you cannot even rerun your own analyses after six months, your mindset is what needs to change. A good source-control system like git is useful but not strictly necessary. Most importantly, do not tie yourself to solutions that may disappear in 10 years. Shun the latest IDE and learn a powerful editor like vi (vim); the payoff is worth it. Learn to structure your work by using sub-directories for raw data, processed data, library code that implements tools you use in your analysis. Create a directory called Backups and put copies of your working code in there at least daily. Reproducible research was possible long before the latest tools--it's a mindset, not a set of tools.

Reproducible research is important because it allows other researchers to build on your work, and to verify that your results are correct. Reproducible research can be challenging, but there are many tools and resources available to help you. Reproducible research is becoming increasingly important in many fields, as funders and publishers are increasingly requiring researchers to make their data and code available. With R 'rUM' package it is very easy to make a research reproducible.

In the dynamic world of scientific research, a meticulously crafted experimental design is the linchpin that connects data collection to reproducibility. Far too often, researchers embark on data collection journeys without a well-defined blueprint. This practice not only muddles research objectives but also unleashes uncontrollable variables, thus adding a layer of disruptive noise to the data. To uphold the integrity of research findings and ensure reproducibility provides a clear pathway for data collection, fostering transparency, credibility, and the potential for others to replicate and validate the results.

Unless you are part of a prominent research cycle proposing a greatly different scheme, if your research conforms to a related category, it is the correct way to use the common benchmarks. In plain words, if you are developing a new image classification architecture, then first step of ensuring reproducibility is to test it on common benchmarks.

When we talk about statistical literacy, we got to shape our intentions and expectations realistically. It has been an ongoing debate how general public lacks basic statistical literacy and data evaluation skills. However, I think, we cannot expect every citizen on earth to know what a p-value is. However again, and a very big however, we should expect everyone to know what a p-value is, if that particular person is tasked to read reports containing p-values, even in small footnotes of graphs.

It can look kind of old, but gaining statistical literacy is a matter of facing theory. Without theory, one can easily get misconceptions and misuse statistics. A basic statistics book (even those that are said "for dummies") can help you to start. Like mathematics, statistics also requires some exercises.

I would place much more emphasis on critical thinking which is a skill that can be applied in much more broader context. Critical thinking is the ability to analyze, evaluate and synthesize information in logical and systematic manner. In this age of misinformation critical thinking would enable one recognize common tactics that are often used to manipulate or distort information such as cherry-picking data, using misleading statistics, or appealing to emotions rather than reason.

Learning about how to combine what you see and what you write is one of the emerging trends in the statistical industry. Basically, for AI - a statistician needs to understand transformers and breaking those into smaller steps.

One thing that helped me in my early career days was to learning how to present factual information. Once you research statistics on any given topic and start compiling graphs (or any other way of portraying facts and numbers), you quickly realize how many pitfalls there are. The same information can be perceived completely differently depending on the way you present it. Learning through practice was more effective for me than simply reading avout these challenges.

You should learn at least 2 programming languages, such as R, Python, Stan, SAS, JavaScript, C++ or others. These are essential in most of the aspects of advanced statistics discussed in the article. Also knowing these allow for customisation of the analysis with tens of thousands of statistical packages. Further Statistical software development is also important and these are the main programming languages for it.

Good to study: (1) ways to control Type 1 error associated with DNA particles being therapy selective, (2) prediction modeling (vs prognostic modeling) for customized treatment plans, and (3) biomarker development strategies to promote health equity. Guaranteed to be forever busy if you master these topics.

Just do not be afraid of statistics and have fun. That's the most important thing I have to say. Statistics is not difficult, it is complex, but once you start to handle the basics, the more complicated concepts come with more ease. Don't forget that, nowdays, we have a lot of people learning how to program softwares and using ready-to-go models. What will differentiate professionals of this field in the future is the ability to look at problems from an statistical and more broad approach instead of just applying models that one just does not know the meaning.

To really understand statistics at a useful level requires much study and continual learning. Do not shy away from mathematics, but do shy away from people who claim you do not need to understand the mathematics behind statistics. In order to perform statistical analyses at a rigorous level you must understand key concepts like expectations, distribution functions, characteristic functions and their many uses, asymptotic theory including central limit theorems and laws of large numbers, and hypothesis testing in the spirit of Neyman and Pearson. Those are hard topics, and you may not use or understand many of them right away. But I guarantee that throughout your career you will face challenges that require mastery of those fundamentals.

In many areas, the use of synthetic data is increasingly important. Synthetic data addresses privacy and portability issues that are very common in many areas of data analytics, especially health data. In my own experience, the performance on synthetic data is quite strong and the models are more predictive then those using federated models (trained on data sets sequentially with parameters being updated with each new data set)