Here's how you can tackle a data analysis project with unstructured or messy data during an interview.

Na minha vis?o, o primeiro passo é avaliar os dados disponíveis. Examine os dados para entender seu formato, a natureza das informa??es e quaisquer padr?es ou irregularidades. Identifique o tipo de dados (texto, imagem, som, etc.) e a origem dos dados. Isso ajuda a determinar as ferramentas e técnicas necessárias para processar e analisar os dados.

One thing I have found useful is to find take a look at the data types in each column, look for missing values and see if there is a general trend for a certain type of missing data. For example, if you notice that 70% of the data missing is timestamp data this might suggest an issue with how the data is collected and can inform your future decisions. I would also be looking for any inconsistent data such as some data points being title case, upper case or lower case. If there is a mixed bag it could mean that there is some manual entry involved which will undoubtedly lead to errors such as 'Tuesday' being spelled 'Tuseday'.

This will help guide the analysis and focus on the most important aspects of the data. Next, it is important to thoroughly clean and preprocess the data to ensure accuracy and reliability in the analysis. This may involve removing duplicates, filling in missing values, and standardizing data formats. Once the data is cleaned, it can be explored using various techniques such as visualization, statistical analysis, and machine learning algorithms to extract meaningful insights. Finally, it is crucial to effectively communicate the findings and recommendations in a clear and concise manner to demonstrate analytical skills and problem-solving capabilities to potential employers.

En mi experiencia, la evaluación inicial de los datos es importante porque nos permite entender con qué estamos trabajando, lo que facilita todo el proceso de análisis posterior; pudiendo identificar las variables, así como el tipo de datos que contiene y su estructura. Además, una evaluación inicial adecuada nos prepara mejor para realizar un análisis eficiente y efectivo, evitando problemas y asegurando la calidad de los resultados.

Familiarize Yourself with the Data: Start by thoroughly understanding the data you’re working with. Explore its structure, format, and any potential issues. Identify the sources of data and any relevant context.

It's unusual to clean data during an interview unless a dataset is provided beforehand. Knowing the employer's preferred tools for data preparation, such as SQL, Alteryx, or Dataiku, is essential to tailor your explanation. This enables you to describe your cleaning process using their terminology and methods. Understanding their tools showcases your adaptability and ensures you can effectively communicate your approach, demonstrating both technical proficiency and alignment with their workflow, which is critical for presenting a clean, reliable dataset.

When tackling a data analysis project with unstructured and messy data, start by evaluating the data landscape. Ask yourself: - What type of data is it (qualitative/quantitative)? - Type (qualitative/quantitative)? - Multiple sources? - Apparent structure? This initial assessment helps identify tools and techniques needed for data cleaning and organization. By taking a moment to understand the data, you'll set yourself up for success in transforming raw data into a format ready for analysis. Remember, a clear understanding of the data landscape is the foundation of any successful data analysis project.

I would not expect to clean data during an interview, however if I had to clean data I would be asking questions about where this data is being used, shown and how it is being collated. If we have more context surrounding the data we would be able to determine what is the best way to remove innaccuracies and missing values. For example, filling in the missing values with the mean average of all values in the column can be a great idea or a bad idea. With context we can make better decisions. If a dataset is just plain wrong, it can sometimes be helpful to find out if there is inherent bias because of the sampling pool. Healthcare data in an area where the average age is 70 compared to 34 will tell two different stories.

Como analista de datos, puedo afirmar que la limpieza de datos es una etapa esencial en cualquier proyecto de análisis. Después de evaluar los datos, es fundamental eliminar imprecisiones, rellenar valores faltantes y corregir incoherencias. Utilizar expresiones regulares para extraer información y técnicas de imputación para manejar las brechas es parte del proceso. Explicar claramente cada paso de la limpieza no solo demuestra atención al detalle, sino que también asegura la integridad de los datos. Un conjunto de datos limpio es la base de cualquier análisis fiable y preciso. Sin una limpieza adecuada, los resultados del análisis pueden ser enga?osos o incorrectos, comprometiendo decisiones basadas en esos datos.

When working with qualitative data you should also think about enriching the dataset. This will allow you to focus and select the most interesting data points. This can usually be done by leverage pertained models. For text you could use language models to add the topics present in each text, identify different entities that are present like persons, companies or dates, or you could even ask models to remove all PII. For vision having an idea of which and how many objects are present and the characters could be useful. For example do we have pictures of cats or dogs. How many dogs do you have and which breed. Pre-trained models can help you with that.

Here are a few things I do: - Using derived indexes to combine two related data points. - Renaming column names with a consistent naming convention. - Seperate timestamps into year, month, week, day and sometimes even hour. - Use a correlation matrix to find out any hidden trends in the data. (Does not always work) - Create segments within the data with K-Means clustering or similar. (Does not always work)

Organizar los datos después de limpiarlos es crucial para un análisis efectivo. Técnicas como normalización, que ajusta los datos a una escala común, y agrupación, que clasifica datos similares, son esenciales. Transformar variables puede simplificar análisis complejos, mientras que indexar datos mejora la velocidad de las consultas. Integrar datos de diferentes fuentes proporciona una visión más completa. Estas prácticas aseguran que los datos estén listos para un análisis preciso, revelando patrones y facilitando decisiones informadas. Sin una buena organización, el análisis se vuelve más difícil y menos fiable.

Si bien los gráficos para el EDA varían según los datos, los que uso frecuentemente son: -Histogramas: Revelan la distribución de una variable y detectan sesgos y outliers. -Diagramas de dispersión: Muestran relaciones entre dos variables continuas, útil para detectar correlaciones. -Boxplots: Identifican outliers y muestran la dispersión de los datos. -Matrices de correlación: Visualizan relaciones entre múltiples variables, ayudando a encontrar correlaciones significativas. -Gráficos de barras: Comparan categorías, mostrando diferencias claras entre grupos. -Gráficos de línea: Analizan tendencias temporales. -Mapas de calor: Visualizan densidades y patrones en datos matriciales.

During the interview, It is easy for candidates to rush into their own conclusions. There are many paths to find the answer in data analytics and you don't have to give all of the answers but it is good to suggest 2-3 choices. 1. Turn business question & problem into analytical framework by identifying the metrics or quantitative features to explore. 2. Implement EDA with Charting, Pivoting, and Dicing the data to check data integrity is guaranteed and enough to build the reasonable test & model. 3. Ready for the follow-up question with accurate number and reasonable logic. Even though the rationale is not enough to reveal the entire scene, it is always good to provide the reason why it seems like.

Algunos puntos que me han funcionado para la interpretación de los datos después del EDA: -Contextualiza: Relaciona los hallazgos con las preguntas de investigación y el contexto del negocio. -Identifica patrones: Busca correlaciones, distribuciones y anomalías que confirmen o refuten hipótesis. -Descubrimientos sorprendentes: Se?ala y analiza hallazgos inesperados. -Crítica: Evalúa los datos críticamente, considerando sesgos y limitaciones. -Comunica claramente: Usa visualizaciones efectivas y explicaciones simples. -Insights accionables: Proporciona recomendaciones útiles para decisiones. -Validación: Revisa y valida tus interpretaciones con colegas. -Documenta: Mantén un registro claro del proceso y las decisiones tomadas.

Be as concrete as possible and start with your conclusion. Tell the business what you would do. Afterwards tell them your reasoning that is supported with the data. They want to know what they should do, so you should start with this. Often they will not require you to explain them all the detailed topics of your analysis. You should have them at hand when they ask for this but usually you'll discuss this with the data team