AgileEngineAgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. If you’re looking for a place to grow, make an impact, and work with people who care, we’d love to meet you! About the Role
As a Data Engineer specializing in Java and Apache Spark, you will help build and evolve large-scale data processing systems that power experimentation and user insights. Working within a cloud-based AWS EMR environment, you’ll contribute to improving data infrastructure reliability, scalability, and cost efficiency. This role offers the opportunity to shape critical datasets and analytics capabilities while collaborating with platform and data teams to support high-impact experimentation and decision-making.s.
What you will do Develop and maintain index builder products, including user session index builders, user session–derived index builders, and experimentation platform index builders;
Investigate and resolve reported issues related to index builders;
Assist with user inquiries regarding the platform and its datasets;
Improve index builder stability and reliability;
Support efforts to optimize compute costs across the platform;
Contribute to the Central Exposure Dataset effort, including building a consolidated dataset for experiment analysis;
Work toward meeting code freshness goals;
Persist Yarn logs and Spark history for terminated clusters;
Capture metrics from UserCohort;
Optimize resource allocation for platform infrastructure;
Help reduce the number of core instances for platform clusters;
Support the deprecation of legacy index builders used for experiment analysis. Must haves
4+ years experience in software development;
Bachelor’s degree in Computer Science or equivalent practical experience;
Significant practical experience with Java (4+ years);
Practical experience implementing Apache Spark jobs, including partitioning, grouping, joins, importing data into the cluster, and exporting data from the cluster;
Practical experience working with AWS, specifically AWS EMR (or ability to pick it up fast);
Upper-intermediate English level. Nice to have
Basic knowledge of Kubernetes;
Experience with Spark Operator;
Experience with Airflow;
Experience with Scala. Perks and Benefits
Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps
We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities A selection of exciting projects
Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands
Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.
¿Te interesa este puesto?