What do you do if you need to hire new members for your data engineering team?

In my point of view, some points should be taken into consideration, such as: Define roles and responsibilities, Create job descriptions, Use job platforms and networks, Analyze resumes and applications to identify candidates who meet the criteria described in the job descriptions, Conduct technical interviews, Assess cultural fit and team dynamics, Provide candidates with hands-on assignments or projects to assess their ability to apply their skills in real-world scenarios, and Once candidates accept job offers, facilitate the onboarding process .

Hiring the data engineering staff involves assessing whether the candidates posses both hard technical and soft skills. Code knowledge in important languages such as Python and Scala, as well as a profound comprehension of database systems and SQL should be taken into consideration. Familiarly with big data frameworks - Hadoop, Spark or others - is significant prerequisite for dealing with complex data processing. An assessment of the candidates into their technical capabilities, problem-solving skills, and effective communication. Soft skills such as ability to work in a team, good communication and problem solving contribute to teamwork, engaging in projects and every other work among different departments.

I do prefer sending the candidate a test for him/her to do at home, he/she will have all the tools available to completed the test but once the test is finished/completed I have a second interview where I can ask/understand why something was created that way and not the other way around, in that process I will capture how the participant will express about the solutions and help me to conclude if he/she is a good candidate, an experienced candidate is noticeable or a bad one can be discarded.

In my case I always try to lead the candidate to emulate as much as possible a normal day-to-day work as a Data Engineer. My Code Challenges are completely focused on exercises that the person will encounter on a daily basis and I allow them to ask questions, to google, to make mistakes and also if it is the case, that they don't finish the test, yes, I don't care if they don't finishes it, I evaluate how the process was and validate the reasons why he couldn't finish it in the established time. Additionally, I like the off-screen Code Challenges more, although I do both (live coding) but the first one seems more real to me in terms of a task and then he explains step by step how he approached his solution and I ask questions from there.

A data engineering team should first evaluate its present needs and identify any skill shortages before hiring new members. Establish specific openings and criteria, then use a variety of media to market your hiring plan in an effort to draw in suitable applicants. After screening resumes and applications to create a shortlist of individuals who best fit the job specifications, in-depth interviews are held to evaluate each candidate's problem-solving, technical, and cultural fit. Make employment offers to the most qualified applicants after evaluating references to confirm credentials and performance.

Evalua que responsabilidades necesitas dentro del equipo y agrupalos en roles especificos según al industria, el mercado y la madurez del equipo. Si hay responsabilidades que no encajan con los roles pero que no son critico, pon ello como opcionales en los roles definidos: es más probable que se trabajen más adelante com poco impacto. Si hay responsabilidades que no encajan con los roles pero que son critico; puede usar un metodo de outsourcing or freelance para cubrir el puesto temporalmente.

Designing an effective interview process for hiring data engineers is crucial for finding the right candidates. Begin with a phone screen to evaluate basic qualifications and communication skills. Follow up with technical interviews that include practical assessments such as solving real-world problems using SQL queries or designing data pipelines. Incorporate team members in the interview process to assess cultural fit and provide candidates with insights into your team's working style and values

Based on my experience when I am tasked with finding a new member for the team, I like to ask them what projects they have to show and what value they brought to their company, finally I encourage them to solve a simple but real case study as close to the real role they will have in our company (no unnecessary code testing, no boring technical interviews) and this helps me know how they solve the problem, how efficient and how quickly they learn something new in case they have not had the opportunity to work. with the stack technology we use. I can finally say that this has worked for us and reduced the time to hire our next team members.

Candidates flexible attitude and approach. Along with development willingly he/she should mentor others. Only Coding challenge or technical aspects Never help build a good strong team. It’s one of required skills but candidates who can spot the right strategies for data engineering project and provide the suggestions to build scalable, cost effective and single blueprint solution. So put more scenarios based questions in data engineering interview process. Use case based questions. I would say interview should be like discussion exchanging and sharing knowledge.

In my opinion, determining whether an individual has the appropriate level of expertise during a selection process is quite challenging, even when assessments are conducted. I personally believe that these tests, except for very specific cases, can sometimes be counterproductive, potentially excluding highly qualified individuals. Instead, I think the best approach is to assess the minimum requirements during interviews and then have an onboarding process that allows for a more secure evaluation of whether the person is truly the right fit, both professionally and personally.