Sr. AWS Data Engineer

Job Title: Sr. AWS Data Engineer

Location: Remote (Mexico)

Work Type: Full Time

About Smart IT Frame:

At Smart IT Frame, we connect top talent with leading organizations across the USA. With over a decade of staffing excellence, we specialize in IT, healthcare, and professional roles, empowering both clients and candidates to grow together.

Job Description:

We are seeking a Senior Data Engineer to build and optimize data systems that power batch processing, real-time streaming, pipeline orchestration, data lake management, and data cataloging. You will have the opportunity to use your expertise in solving big data problems and apply design thinking, coding and analytical skills to develop data pipelines that support marketing data products. We’re looking for talented Data Engineers passionate about building new data-driven solutions with the latest Big Data technology.

What you’ll Do:

• You will build services that will integrate directly with client’s products and with external vendors.
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

About you:

• 5+ years of experience and a bachelor’s degree in computer science or a related field, or equivalent work experience
• In-depth working experience of distributed systems Hadoop, Spark, Hive, DBT and Airflow/Dagster
• At least 4 years of solid production quality coding experience in data pipeline implementation in Python
• Experience working with public cloud platforms, preferably AWS
• Experience working with Databricks and/or Snowflake
• Experience in Git, JIRA, Jenkins, shell scripting
• Familiarity with Agile methodology, test-driven development, source control management and test automation
• Experience supporting and working with cross-functional teams in a dynamic environment
• You have excellent collaborative problem solving and communication skills and are empathetic to others
• You have a track record of building fast, reliable, and high-quality data pipelines

Nice to have skills:

• Experience building Data pipelines for Digital Marketing will be a big plus