Skip to main content

Data Engineer

Technology
Data Engineering
Bengaluru, India3 weeks ago
Full timeOn-site

Job description

Description

Primary Focus: SQL-to-Snowflake Data Transfer, ELT Pipelines & File-to-ADLS Ingestion

Role Overview

We are seeking an experienced Data Engineer to support a high-priority data integration and cloud enablement initiative. The primary focus of this role is designing and executing end-to-end ELT pipelines and data transfer workflows—migrating structured data from SQL sources into Snowflake and orchestrating bulk file transfers into Azure Data Lake Storage (ADLS).

Responsibilities

  • Design, build, and maintain scalable ELT/ETL pipelines and data workflows for ingestion and transformation.
  • Execute structured data extraction, movement, and landing from relational SQL databases directly into Snowflake staging and core layers.
  • Build, execute, and monitor file movement tasks to efficiently transfer flat files, logs, or unstructured formats into Azure Data Lake Storage (ADLS).
  • Write and tune high-performance SQL queries for data modeling, validation, data verification, and staging transformations.
  • Conduct data reconciliation and completeness checks to ensure zero loss across data pipelines during bulk migration.
  • Collaborate closely with the Lead Data Integration Expert and client engineering teams to align with platform connectivity, security, and governance protocols.

Required Skills

  • ELT / Data Pipelines: Proven, hands-on experience building, optimizing, and monitoring production ELT/ETL pipelines and data workflows.
  • SQL & Relational Databases: Strong proficiency in SQL (writing complex queries, performance tuning, indexing) for extracting and validating large datasets across relational engines.
  • Snowflake: Hands-on experience loading and modeling data in Snowflake using staging strategies, COPY commands, or bulk loading utilities.
  • Azure Cloud Storage: Solid background working with Azure Data Lake Storage (ADLS Gen2), Blob Storage, and associated file transfer/ingestion patterns.
  • File Processing: Practical experience with bulk file ingestion formats (CSV, Parquet, JSON) and file movement tooling.
  • Agile Execution: Ability to deliver rapid, high-quality results within structured project timelines.

Preferred Skills

  • Experience with orchestration and transformation tools (e.g., Airflow, dbt, Azure Data Factory, or Python-driven pipeline movers).
  • Familiarity with enterprise or industrial data integration platforms and SAP.
  • Version control using Git.
Keywords
SQLSnowflakeAzure Data Lake StorageELTETLData PipelinesData ModelingData IngestionPerformance TuningData ReconciliationCSVParquetJSONAgileGitData EngineerADLS Gen2AirflowdbtAzure Data Factory

Interested in this role?