Skip to main content

Data engineer

Technology
1 week ago
Service contract

Job description

Key responsibilities for the Data Engineer:

  • Optimize data pipelines that read from Kinesis and Write into Oracle improve exception handling for schema/data type mismatches and implement alerting for data quality issues.
  • Build services to read/write data to NetApp object store using the S3 API.
  • Create analytical datasets in Parquet or Iceberg format.
  • Develop housekeeping scripts for cleanup, archiving, and compression of logs, Object store files, etc.
Core skill requirements:
  • Strong Python/Java and data engineering expertise.
  • Experience building and deploying data pipelines using frameworks like Spark, Flink and understanding of distributed data systems.
  • Hands-on experience with real-time streaming architectures, REST APIs, and database connectors.
  • Proficient with and Relational Databases (e.g. Oracle).
  • Experience working with Amazon S3 APIs for data storage and integration.
Preferred (nice-to-have):
  • Experience with Apache Iceberg.

Interested in this role?