Skip to main content

Senior Data Engineer - Python/SQL

Technology
Good Co
₹25,00,000 - ₹55,00,000 /year1 week ago

Job description

Job Summary :

We are looking for an experienced Senior Data Engineer to design, develop and maintain scalable data pipelines, data platforms and cloud-based data infrastructure. The ideal candidate should have strong expertise in data engineering, ETL/ELT processes, big data technologies, cloud platforms and data warehousing.

Key Responsibilities :

  • Design, develop and maintain scalable data pipelines and ETL/ELT workflows.
  • Build and optimize data processing frameworks for structured and unstructured data.
  • Develop data integration solutions across multiple internal and external systems.
  • Work with large datasets using distributed computing technologies.
  • Design and maintain data warehouses, data lakes and lakehouse architectures.
  • Implement batch and real-time data processing pipelines.
  • Optimize data storage, processing performance and query execution.
  • Collaborate with Data Scientists, AI Engineers, Business Analysts and Software Engineers.
  • Ensure data quality, governance, security and compliance.
  • Implement automated data pipeline monitoring and error-handling mechanisms.
  • Manage cloud-based data infrastructure and deployment workflows.
  • Support AI/ML teams by providing reliable, high-quality data pipelines.

Required Technical Skills :

  • Programming : Python, SQL, PySpark, Scala
  • Big Data Technologies : Apache Spark, Hadoop, Kafka
  • ETL/ELT Tools : Apache Airflow, dbt, Informatica
  • Cloud Platforms : AWS, Microsoft Azure, GCP
  • Data Warehousing : Snowflake, Amazon Redshift, Google BigQuery, Azure Synapse
  • Databases : PostgreSQL, MySQL, MongoDB
  • Data Lakes : Delta Lake, Apache Iceberg, AWS S3
  • Data Processing : Batch Processing, Stream Processing, Real-Time Analytics
  • DevOps & Deployment : Docker, Kubernetes, CI/CD, Terraform
  • Data Governance : Data Quality, Data Lineage, Metadata Management

Experience &

Qualifications :

  • 6 - 11 years of overall experience in Data Engineering or related fields.
  • Minimum 4 - 5 years of hands-on experience in data engineering.
  • Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering or a related field.

Preferred Certifications :

  • AWS Certified Data Engineer - Associate
  • Google Professional Data Engineer
  • Microsoft Certified : Fabric Data Engineer Associate
  • Snowflake SnowPro Core Certification
  • Databricks Certified Data Engineer Associate
Keywords
ScalaApache HadoopApache KafkaApache SparkRedshiftMongodbMetadataApache AirflowDevOpsAirflowMysqlPostgresqlPythonSqlHadoopApache Http ServerCI/CDBig dataBigQuery

Interested in this role?