What are the best tools and techniques to collect and analyze data for your research project?

The first step is to determine how broad or specific is my research question. If it's broad, then perhaps I may benefit from narrowing in and defining the scope a bit more. If the scope is too narrow, then perhaps I may benefit to broadening my aperature. A good example is financial literacy. Perhaps I want to start broad and ask, what does financial mean to people? Do they consider themselves financially literate? That is quite broad. But what if I narrow in to look at financial literacy and its impact on how college students make decisions money decisions. That's a more refined and narrow scope. I can even step back and ask what types of financial topics do younger consumers care about? All of these are exampes of how to shift scope.

When choosing your data source, it is important to think about how you will refresh the data. For example, if you decide to go with implementing a survey, what is the cadence in which you will send out future surveys to track changes in responses? When deciding on a data source, it is important to keep sustainability and the ability to iterate in mind.

Your research questions really ought to justify your data collection sources. Some people may think, "I need a survey" when their research questions may lend themselves to more descriptive techniques such as observations, interviews, and focus groups. Other people may want to speak with end users because the surveys are not providing much insight, and so may think "I need a focus group" when it reality the topic might be too sensitive in nature and they would benefit from a different approach. One technique I use is really to diagnose the problem and research question, then use a process of elimination to arrive at the best method or combination of methods to arrive at an approach.

If collecting survey data, Qualtrics is great. There are many ways to customize surveys, and it has a user friendly interface that is easy to use and navigate. If using an online sample, whether via Qualtrics, MTurk, or other methods, it might be a good idea to restrict access to a particular geographical region. If you are US-based, consider restricting access to those who have US based IP addresses, unless you are interested in international responses. This can reduce the risk of too many "junk" responses from survey farms or not getting enough responses from one's population of interest.

When using clinical data such as lab values, it is important to consult with those with expertise. For example, if we see potential "outliers" in Hba1c values, these values may be accurate for those patients. It's good to have a physician to consult on whether to delete such values or keep them as-is.

SPSS is a very common data analysis tool, but it is not free. If you already have access to it, or you are in a position to pay for it, then it's a great program with a variety of features and extensive tutorials on YouTube (virtually any type of analysis you can find a tutorial for). Various free software also exists, R being very common. R is open source, and there are a wide variety of packages for many different types of analysis. The only downside is that it is a bit less user friendly than SAS/SPSS so some basic familiarity with programming is required. Still, many tutorials also exist and chatGPT can troubleshoot code. Depending on how basic the analysis, Excel might also be an option, as many analyses can be done in Excel.

When analyzing a survey, comparing groups is a common practice and so can be used to determine statistical differences. A chi-square test can also tell me if group is independent based on the expected number of counts within a cell for a larger sample. When comparing numerical data, there are 2-sample t-tests, paired t-tests, correlations, and ANOVA to compare differences. I can even use a regression to understand significant predictors. For a qualitative data analysis, typically one is analyzing text and looking for common patterns and themes within participants and across participants. Thematic saturation is an important consideration as well as there tends to be diminishing return in new themes as more participant data is analyzed.

Whatever tools you use (SPSS, STATA, SAS, R, ...), it's always a great practice to always go back to the objectives of the project as well as the hypothesis of that project. Depending on the scope of the project, some simple descriptive analyses will still work such as using Tableau, or even excel graphs/figures.

This is easily the most truly time consuming piece of the process and the best advice is to "live in the data". By living in the data you can truly come to understand it best and how to best present those findings to your audience.

With reporting, always keep your audience in mind. Not everyone knows what an ANOVA is, and not everyone is going to understand why a structural equation model was used, so it's best to avoid jargon unless you are publishing in an academic journal. I'd also suggest sticking to plain language as much as possible. If the report is intended for a wide, general audience, it might be helpful to pop it into word and look at the Flesh-Kincaid reading ease score through the document stats. The average American reads at around the 7th grade level or lower, so communications intended for a general audience should aim for plain language. Details can always be included in an appendix or supplemental material for those so inclined to peruse them.

In my experience, there is a good practice that we keep all the syntax with a good description of any projects. Some projects we may get back to those after years for many different purposes (teaching or additional analyses based on the original ones). Also, keeping a detailed syntax may help keep the quality of your work with a second eye from your colleagues/co-workers.

To collect data for analysis: utilize surveys, web scraping, collaborations, social media monitoring, historical data, and relevant public datasets. In real-world scenarios, Kaggle datasets are often unsuitable. For instance, in my recent gift items sales analysis, I conducted online surveys with gift buyers, scraped data from e-commerce platforms, collaborated with local retailers for sales figures, and analyzed social media trends. This multi-faceted approach ensured robust data for comprehensive analysis.