Skill : Azure Data Engineer Architect 9y+
Paras Kothari
Possessor at IT Company (Projects, Recruitment, Subcon ). 46000+ Followers Talks AI,IT, Recruitment,Subcon,#jobs, #hiring, #jobalert, #recruitment, and #hiringalerts. Please follow back, subscribe to Newsletter for job
Job?Description:?Lead?Databricks?Data?Architect?(Technical?Architect)?
?
Responsible?for?designing,?developing,?and?optimizing?cloud-based?data?architectures?using?Azure?
Databricks,?Azure?Data?Factory,?and?Azure?SQL?DB.?This?role?involves?working?closely?with?data?
engineering,?analytics,?and?operations?teams?to?deliver?scalable,?efficient,?and?high-performance?data?
pipelines.?The?Lead?Databricks?Data?Architect?will?be?responsible?for?implementing?best?practices,?
mentoring?teams,?and?ensuring?seamless?data?integration,?ingestion,?and?transformation?workflows.?
?
Key?Responsibilities:?
?Architect?and?design?end-to-end?data?solutions?using?Azure?Databricks,?Azure?Data?Factory,?and?
Azure?SQL?DB,?focusing?on?high-performance,?scalable,?and?secure?data?architectures.?
?Lead?the?design?and?development?of?data?ingestion?pipelines?utilizing?Delta?Live?Tables?(DLT)?and?
Structured?Streaming?to?handle?both?batch?and?real-time?data?ingestion?requirements.?
?Optimize?the?performance?of?Spark?jobs?and?Databricks?workflows?using?advanced?Spark?
internals,?performance?tuning,?and?job?optimization?techniques.?
?Develop?and?maintain?metadata-driven?data?ingestion?frameworks?to?improve?automation,?
scalability,?and?data?pipeline?management.?
?Collaborate?with?cross-functional?teams,?including?data?scientists,?analysts,?and?business?
stakeholders,?to?define?data?requirements?and?deliver?solutions?that?align?with?business?goals.?
?Provide?leadership,?technical?guidance,?and?mentorship?to?data?engineers,?ensuring?adherence?to?
best?practices?and?continuous?learning?within?the?team.?
?Implement?and?enforce?data?quality?checks,?governance?policies,?and?data?security?measures?to?
ensure?the?integrity,?availability,?and?confidentiality?of?data.?
?Continuously?monitor,?optimize,?and?enhance?the?performance?of?cloud-based?data?infrastructure,?
while?ensuring?cost?efficiency.?
?Keep?up?to?date?with?emerging?trends,?tools,?and?technologies?in?the?data?engineering?space,?
particularly?in?the?Databricks?and?Azure?ecosystems.?
Skills/Qualifications:?
?Bachelor's?degree?in?Computer?Science,?Information?Technology,?Data?Engineering,?or?a?related?
field.?
?10+?years?of?experience?in?data?engineering?or?data?architecture?roles.?
?6+?years?of?experience?designing?and?implementing?solutions?using?Azure?Databricks,?Azure?Data?
Factory,?and?Azure?SQL?DB.?
?Extensive?experience?working?with?Databricks,?Spark?internals,?and?optimizing?Spark?jobs?for?
performance.?
?Strong?proficiency?in?PySpark,?SQL,?and?Python?for?building,?maintaining,?and?optimizing?data?
pipelines.?
?Extensive?experience?in?designing?and?developing?data?ingestion?pipelines?using?Delta?Live?Tables?
(DLT)?and?Structured?Streaming.?
?Experience?in?developing?metadata-driven?frameworks?for?automating?data?ingestion?and?
transformation.?
?
?
?
?Strong?understanding?of?data?warehousing?concepts,?including?star?schema?and?snowflake?schema?
design.?
?Azure?Data?Engineer,?Azure?Solutions?Architect,?or?Databricks?certifications?(e.g.,?Databricks?
Certified?Associate/Professional)?are?preferred.?
?Proven?leadership?abilities,?including?leading?data?engineering?teams,?mentoring?team?members,?
and?guiding?complex?technical?projects.?
?Strong?communication?skills,?with?the?ability?to?translate?technical?concepts?to?non-technical?
stakeholders?and?present?solutions?effectively.?
08355934128