Skip to main content

Data Engineer

Technology
3 weeks ago
Full time

Job description

  • Experience: Minimum of 2-3 years relevant experience
  • Technical Requirement: Strong expertise in AWS
Data Pipeline & Infrastructure Development
  • Design, develop, and maintain scalable data pipelines for data ingestion, transformation, and storage.
  • Build and manage ETL/ELT workflows to integrate data from multiple internal and external systems.
  • Develop and maintain data warehouses, data lakes, and hybrid storage solutions.
  • Monitor and optimize pipeline performance, reliability, and cost efficiency.

Data Quality & Reliability

  • Implement data quality checks, validations, and monitoring mechanisms.
  • Identify and resolve data issues, anomalies, and pipeline failures.
  • Ensure data accuracy, completeness, and consistency across platforms.
  • Support continuous improvement of data reliability and availability.

Collaboration & Analytics Enablement

  • Collaborate with data scientists, analysts, and business users to understand data requirements.
  • Deliver curated, analytics-ready datasets to support reporting, dashboards, and advanced analytics.
  • Partner with Data Architects and Enterprise Architecture teams to ensure alignment with architectural standards.

Data Security & Governance Support

  • Implement data engineering solutions that comply with data governance, security, and privacy policies.
  • Apply access controls, data masking, and other protective measures where required.
  • Support metadata management, documentation, and lineage tracking.

Documentation & Automation

  • Create and maintain documentation such as data flow diagrams, pipeline designs, and data dictionaries.
  • Automate data processing and operational tasks to reduce manual intervention and errors.
  • Document operational procedures and troubleshooting guidelines.

Technology Evaluation & Continuous Improvement

  • Evaluate and adopt new data tools, platforms, and engineering practices.
  • Stay current with trends in big data, cloud platforms, and data engineering frameworks.
  • Recommend improvements to data infrastructure and processing efficiency.
Keywords
Data maskingMetadataBig dataReplication

Interested in this role?