Senior Azure Data Engineer Job at eTeam, Redmond, WA

  • eTeam
  • Redmond, WA

Job Description

Job Title: Senior Azure Data Engineer

Location: Redmond, WA 98052

Work Setup: Hybrid

Duration: 12 Months

Experience: 8-10 Years


Job Description


We are looking for an experienced Senior Azure Data Engineer with 8-10 years of experience in data engineering and strong hands-on expertise in Python, SQL, Apache Spark, PySpark, and Azure Databricks .


The candidate will be responsible for developing and supporting scalable data pipelines, transforming large datasets, and optimizing data processing solutions using Azure cloud technologies.


Must Have Skills

  • Strong experience with Python and SQL
  • Hands-on experience with Apache Spark, PySpark, and Spark SQL
  • Strong experience with Azure Databricks , including:
    • Notebooks
    • Jobs/Workflows
    • Clusters
  • Experience with Azure Synapse
  • Strong knowledge of Delta Lake
  • Experience working with Parquet
  • Hands-on experience with ADLS Gen2
  • Strong understanding of data engineering and ETL/ELT concepts
Nice to Have Skills
  • Experience with Scala
  • ETL/ELT development
  • Experience with MERGE/Upsert operations
  • Knowledge of data partitioning and partition pruning
  • Strong Spark performance tuning experience
  • Experience handling data skew, shuffles, caching, broadcast joins, and partition optimization
  • Experience with Git
  • Experience with Azure DevOps and CI/CD
Key Responsibilities
  • Develop and maintain scalable ETL/ELT data pipelines
  • Extract, transform, clean, and load data from various source systems
  • Build data solutions using Azure Databricks, Spark, Delta Lake, Synapse, and ADLS Gen2
  • Implement MERGE/Upsert logic using Delta Lake
  • Design effective partitioning strategies for large datasets
  • Optimize Spark jobs for performance and cost
  • Manage data skew, shuffles, caching, partitions, and cluster resources
  • Collaborate with development and data teams to deliver reliable data solutions
  • Follow best practices for Git, Azure DevOps, and CI/CD where applicable

Job Tags

Similar Jobs

Pacer Group

Azure Data Engineer Job at Pacer Group

 ...Tailored Job Description Azure Data Engineer Data Modelling & Architecture Strong understanding of semantic models used in enterprise reporting and analytics. Expertise in dimensional modelling, including star and snowflake schema design. Practical experience... 

Dream Team Family Dentistry

RDA or EFDA Job at Dream Team Family Dentistry

 ...At Dream Team Family Dentistry, your smile is our priorityand were here to help you keep it bright for years to come. Title RDA or EFDA Job Description Key Responsibilities: Prepare treatment rooms by sterilizing instruments and setting up equipment Assist... 

The Judge Group

Director of Patient Access Job at The Judge Group

 ...Our client is currently seeking a Director of Patient Access Role Description: The Director of Patient Access provides strategic...  ...Patient Access processes to improve efficiency and patient experience. Revenue Cycle Performance Ensure accurate and... 

Boyse Dentistry

Registered Dental Assistant (RDA) Job at Boyse Dentistry

 ...Where Every Tray Prepped is a Smile Saved: Be Our Next Registered Dental Assistant (RDA) ! At Boyse Dentistry, were more than just a dental office, were a tight-knit team of professionals passionate about creating positive, lasting experiences for our patients. We believe... 

3M HEALTHCARE

Associate Director, Patient and Provider Analytics - Neuroscience Job at 3M HEALTHCARE

 ...empower smarter, datadriven decisionmaking. Reporting into Director, Patient and Provider Analytics Neuroscience, this role is crucial in...  .../or MBA strongly preferred minimum. Minimum 6+ years of experience in the pharmaceutical or healthcare industry, with a deep understanding...