Data Engineer

🚨 Hiring: Data Engineer | 5 Years | Indore

We are looking for an experienced Data Engineer with 5 years of hands-on experience in designing, developing, and maintaining scalable data pipelines and data platforms across cloud and big data ecosystems.

📍 Location: Indore

💼 Experience: 5 Years

🏢 Department: Data Engineering

📌 Employment Type: Full-Time

🔹 Role Overview The ideal candidate should have strong expertise in:

✅ Hadoop

✅ Apache Spark / PySpark

✅ Snowflake

✅ Advanced SQL

✅ Data Modeling

✅ Apache Airflow

✅ Cloud Data Platforms

✅ ETL / ELT Pipelines

✅ Data Warehousing

✅ Data Quality & Pipeline Optimization

🔧 Key Responsibilities

• Design, develop and maintain scalable ETL/ELT data pipelines
• Work with large-scale distributed data processing systems including Hadoop and Apache Spark
• Develop and optimize pipelines using PySpark and Spark SQL
• Build, maintain and optimize Snowflake data warehouses
• Implement data models including Star Schema, Snowflake Schema and SCD Types 1, 2 & 3
• Design and manage workflow orchestration using Apache Airflow
• Work with cloud platforms such as AWS, Azure or GCP
• Optimize pipelines for performance, scalability, reliability and cost efficiency
• Implement data quality checks, validation, monitoring and error handling
• Work with structured and semi-structured data including JSON, Parquet, Avro, ORC and CSV
• Collaborate with Data Analysts, Data Scientists and business stakeholders
• Participate in data architecture decisions, documentation and code reviews
• Troubleshoot complex data pipeline issues and drive continuous improvement

💻 Mandatory / Core Technical Skills

Big Data & Hadoop

• Hadoop – HDFS, YARN, MapReduce
• Apache Spark
• PySpark
• Spark SQL
• Hive
• Kafka – working knowledge
• HBase – optional

Cloud – At Least One Required

• AWS – S3, Glue, EMR, Redshift, Lambda, Athena
• OR Azure – ADLS, Azure Data Factory, Synapse, Databricks
• OR GCP – GCS, BigQuery, Dataproc, Dataflow, Composer

Data Warehousing

• Snowflake
• Redshift / BigQuery / Synapse – good to have

Data Modeling

• Star Schema
• Snowflake Schema
• Slowly Changing Dimensions – SCD Type 1, 2 & 3
• Fact and Dimension Tables

Orchestration

• Apache Airflow
• Azure Data Factory
• AWS Step Functions / Prefect / Luigi

Programming

• Python
• PySpark
• Advanced SQL
• Basic Shell Scripting

File Formats

• Parquet
• Avro
• ORC
• JSON
• CSV

Version Control & CI/CD

• Git
• GitHub Actions / Jenkins / Azure DevOps

⭐ Preferred Skills

• Databricks
• Delta Lake
• Apache Iceberg
• Data Lake Architecture
• Data Governance & Security
• Docker
• Kubernetes

🎯 Candidate Expectations

• 5 years of relevant Data Engineering experience
• Proven experience building and optimizing production-grade data pipelines
• Strong analytical and problem-solving skills
• Ability to independently troubleshoot complex data engineering challenges
• Strong cross-functional collaboration skills
• Experience delivering scalable and maintainable data solutions

#Hiring #DataEngineer #DataEngineering #DataEngineeringJobs #DataEngineerJobs #BigData #BigDataEngineer #BigDataJobs #Hadoop #HadoopDeveloper #HDFS #YARN #MapReduce #ApacheSpark #Spark #SparkDeveloper #PySpark #SparkSQL #Hive #Kafka #HBase #Snowflake #SnowflakeDeveloper #SnowflakeJobs #DataWarehouse #DataWarehousing #DataModeling #DataModel #ETL #ELT #ETLDeveloper #ETLJobs #DataPipeline #DataPipelines #DataPipelineEngineer #DataArchitecture #DataArchitect #DataLake #DataLakeArchitecture #ApacheAirflow #Airflow #AirflowJobs #WorkflowOrchestration #AdvancedSQL #SQL #SQLDeveloper #Python #PythonDeveloper #PythonJobs #CloudData #CloudDataEngineering #CloudEngineer #AWS #AWSData #AWSGlue #AmazonS3 #AmazonEMR #AmazonRedshift #AWSLambda #Athena #Azure #AzureDataFactory #AzureSynapse #AzureDatabricks #ADLS #GCP #GoogleCloud #BigQuery #Dataproc #Dataflow #Composer #Databricks #DeltaLake #ApacheIceberg #DataGovernance #DataSecurity #Docker #Kubernetes #Git #CICD #DevOps #GitHubActions #Jenkins #AzureDevOps #Parquet #Avro #ORC #JSON #CSV #SCD #SlowlyChangingDimensions #StarSchema #SnowflakeSchema #FactTables #DimensionTables #DataQuality #DataValidation #DataAnalytics #Analytics #DataSolutions #DataPlatform #DataPlatforms #ProductionData #CloudComputing #CloudTechnology #TechJobs #ITJobs #TechnologyJobs #SoftwareJobs #EngineeringJobs #DataCareers #TechCareers #HiringNow #ImmediateHiring #JobOpening #JobOpportunity #CareerOpportunity #IndoreJobs #JobsInIndore #IndoreHiring #MadhyaPradeshJobs #IndiaJobs #JobsInIndia #IndiaHiring #TechHiring #ITHiring #ExperiencedProfessionals #5YearsExperience #DataProfessionals