Python, Data Engineer, AWS

Python, Data Engineer, AWS

Key responsibilities

• Develop and maintain data pipelines Build, test, and maintain ETL (Extract, Transform, Load) processes to move and transform data.
• Design and manage data architecture Construct and manage data infrastructure, including data warehouses, data lakes, and databases.
• Automate data processes Use Python to automate tasks and build data processing applications.
• Integrate data sources Connect internal and external data sources and APIs, and ensure system compatibility.
• Ensure data quality and security Implement data quality checks, security controls, and access management policies.
• Collaborate with stakeholders Work with data scientists, analysts, and other teams to understand requirements and deliver solutions.
• Optimize systems Monitor system performance, troubleshoot issues, and implement optimizations for reliability and efficiency.

Required skills and qualifications

• Programming Proficiency in Python is a must. SQL and other languages like Java or Scala are also valuable.
• Databases Strong knowledge of database management systems (relational and NoSQL) and data modeling.
• Big Data Technologies Experience with big data frameworks such as Apache Spark and Hadoop.
• Cloud Platforms Familiarity with cloud services is often required, such as AWS, GCP, or Azure.
• Version Control Experience with version control systems like Git.
• Problem-Solving Strong analytical and problem-solving skills.
• Communication Excellent communication skills to collaborate with technical and non-technical teams.
• Education A bachelors degree in Computer Science, Engineering, or a related field is typically required.