Skip to main content

Practo - Data Platform Engineer

Technology
Practo
1 week ago

Job description

About the Role :

We are looking for a highly hands-on Data Platform Engineer to join our AI Labs team in Bengaluru. This role will be responsible for architecting, building, and scaling our end-to-end data platform, bridging traditional data engineering with modern AI infrastructure.

You will own the design and implementation of data pipelines, lakehouse architecture, multi-region data systems, and AI/ML-powered data processing workflows.

From Change Data

Capture (CDC) and ETL/ELT pipelines to LLM inference and multimodal data redaction, you will build reliable, scalable, and cost-efficient data infrastructure.

Roles &

Responsibilities :

1. End-to-End Data Architecture &

  • Engineering :
  • Design, build, and maintain end-to-end data pipelines, covering CDC ingestion from production databases to production-grade lakehouse storage.
  • Develop and scale workflow orchestration using Temporal and data transformation pipelines using dbt.
  • Build standardized dimensional data models and optimized relational/materialized views to support analytics and business intelligence.
  • Modernize legacy data workflows by replacing traditional DAG runners and cron-based batch jobs with scalable orchestration frameworks.

2. Multi-Region Architecture &

  • Data Residency :
  • Design and implement multi-region data architectures across cloud environments, ensuring compliance with data residency and sovereignty requirements.
  • Establish secure data isolation mechanisms to ensure sensitive patient and transaction data remains within permitted geographical boundaries.
  • Build compliant cross-region replication, synchronization, failover, and data-routing mechanisms.

3. AI/ML Integration &

  • Multimodal Data Processing :
  • Integrate machine learning models into data pipelines to identify and redact sensitive PII/PHI across text, images, audio, and video.
  • Build and operationalize embedding generation, LLM inference, and semantic search pipelines.
  • Develop scalable workflows for AI-powered data processing and multimodal content analysis.

4.

Lakehouse

Infrastructure &

  • Platform Enablement :
  • Design and optimize open lakehouse architectures using Apache Iceberg.
  • Implement efficient partitioning, ACID transactions, and time-travel capabilities for large-scale datasets.
  • Provision and manage secure, scalable Jupyter Notebook environments for data science and exploratory analytics.

5.

Cloud

Infrastructure, DevOps &

  • FinOps :
  • Design, deploy, and manage AWS infrastructure using Infrastructure as Code (Terraform or AWS CDK).
  • Establish cost-monitoring frameworks to track cost-per-outcome across data and AI pipelines.
  • Own production reliability through monitoring, alerting, data quality checks, and pipeline freshness tracking.

Required Skills &

  • Technical Expertise :
  • Strong experience in data engineering, data platform development, and large-scale distributed data systems.
  • Proficiency in SQL and Python for data engineering and pipeline development.
  • Hands-on experience building CDC-based ingestion and ETL/ELT pipelines.
  • Experience with workflow orchestration frameworks such as Temporal.
  • Working knowledge of dbt for data transformation and modeling.
  • Experience with Apache Iceberg or other modern lakehouse technologies.
  • Experience with AWS cloud infrastructure and Infrastructure as Code tools such as Terraform or AWS CDK.
  • Exposure to ML/LLM inference pipelines, embedding generation, and AI-powered data processing.

Hiring Note :

We are hiring for two complementary SE II positions from the same candidate pool. Both positions are individual contributor roles focused on deep technical architecture and execution, with no formal people-management or team-mentoring responsibilities.

Keywords
OrchestrationCDKCloud computingPartitionDevOpsPythonSqlApache Http Server

Interested in this role?