Skip to main content

Data Engineer/Senior Data engineer-Python, PySpark, SQL, Databricks

Technology
RAPSYS TECHNOLOGIES PTE LTD
Singapore, Singapore2 weeks ago
Full time

Job description

🌟 We're Hiring: Data Engineer/Senior Data Engineer! 🌟

We are looking for a skilled and driven Data Engineer/Senior Data Engineer to join our team in Singapore. The ideal candidate will have at least 5 years of experience designing, building, and maintaining robust data pipelines and architectures. You will play a key role in ensuring the reliability, scalability, and performance of our data infrastructure to support business intelligence and analytics initiatives.

📍 Location: Singapore, Singapore

⏰ Work Mode: Work from Office

💼 Role: Data Engineer/Senior Data Engineer

What You'll Do:B. Technical & Professional Competencies Essential (Must-Have) • Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing.

• Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering.

• Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, Event Bridge, CloudWatch, Redshift and MSK/Kinesis.

• Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional or domain data modelling and reusable curated datasets.

• Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery.

• Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines.

• Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such as Terraform, and environment promotion controls.

• Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis and service recovery.

• Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secret management, data classification and audit logging.

• Ability to review technical designs and code, establish engineering standards, and guide less-experienced engineers.e

Eager to shape the future of data? 🚀 Apply now and become part of our innovative team!

Keywords
OrchestrationApache SparkRedshiftScalabilityMetadataPythonSqlApache Http ServerCI/CDDebugger

Interested in this role?