Skip to main content

Data Engineer

Technology
Programmers.io
Today
Full timeFully remote

Job description

Programmers.io is currently looking for an "Azure Data Engineer"

Remote Role

.

Full Time Role with benefits

Role-

Key Responsibilities

  • Develop scalable ETL/ELT pipelines using Azure Data Factory.
  • Build data processing and transformation solutions using PySpark and Python.
  • Design and implement data lake solutions using ADLS Gen2 and Delta Lake.
  • Develop optimized Spark jobs for large-volume data processing.
  • Implement data quality, validation, logging, monitoring, and error-handling frameworks.
  • Troubleshoot and optimize data pipelines for reliability and performance.
  • Work with business, analytics, BI, and architecture teams to deliver end-to-end data solutions.
  • Follow development standards, Git workflows, code reviews, and CI/CD practices.

Mandatory Skills

  • 6+ years of Data Engineering experience.
  • Strong hands-on PySpark development experience.
  • Strong experience with Azure Data Factory (ADF).
  • Hands-on experience with Delta Lake and ADLS Gen2.
  • Strong Python and SQL programming skills.
  • Experience with ETL/ELT, data ingestion, transformation, and performance optimization.
  • Good understanding of Azure-based data engineering architecture.

Good to Have

  • Azure Databricks
  • Apache Spark
  • Azure DevOps / CI/CD
  • Medallion architecture
  • Power BI / Snowflake / Microsoft Fabric exposure
If you are interested, please apply or feel free to share your updated resume at anas.khan@programmers.ai
Keywords
Data EngineeringPySparkAzure Data FactoryDelta LakeAzure Data Lake Storage Gen2PythonSQLETL/ELTData IngestionData TransformationApache SparkAzure DatabricksAzure DevOpsCI/CDMedallion ArchitecturePerformance OptimizationADLS Gen2Spark JobsData QualityData Validation

Interested in this role?