Data Engineer

We are seeking a talented Data Engineer to help design and develop modern data platforms that support analytics, machine learning, AI, and other data-driven applications. This role will focus on building scalable, reliable data infrastructure capable of handling complex datasets across a variety of sources and workloads.

The ideal candidate brings strong experience working with modern cloud data technologies and distributed data environments. You will work across data engineering, analytics, and machine learning initiatives while partnering with data scientists, ML engineers, and business stakeholders to turn complex data into reliable, actionable solutions.

Our engineering approach is highly code-focused, with an emphasis on treating data as a product. Quality, scalability, observability, security, and performance are key considerations throughout the development lifecycle.

Key Responsibilities

• Design and develop scalable data platforms capable of bringing together large and complex datasets from multiple agencies, infrastructure systems, and other sources.
• Build cloud-native data architectures capable of supporting batch, streaming, and near-real-time processing requirements.
• Develop robust data ingestion and transformation pipelines connecting legacy applications, APIs, IoT/telemetry platforms, and structured databases.
• Build and maintain distributed data processing and event-driven solutions using technologies such as Spark, Kafka, or equivalent platforms.
• Develop and implement data quality, monitoring, observability, lineage, and reliability practices across the data platform.
• Establish secure data architectures incorporating role-based access controls, encryption, auditing, and appropriate data governance standards.
• Develop modern lakehouse architectures that support analytics, GIS applications, machine learning, and other advanced data use cases.
• Collaborate with data scientists, ML engineers, and senior stakeholders to create the infrastructure necessary to deploy AI and analytics solutions into production.
• Identify opportunities to improve data platform performance, scalability, reliability, and cost efficiency.
• Participate in architectural planning and help establish long-term standards and best practices for the data engineering organization.
• Provide technical guidance and mentorship to other engineers as the platform and team continue to grow.

Requirements

• 3+ years of experience designing, developing, and supporting large-scale data platforms within cloud or hybrid environments.
• Demonstrated experience building modern data architectures, including lakehouse, data mesh, or comparable architectures, supporting both analytical and operational use cases.
• Strong hands-on experience with distributed data processing technologies such as Spark and streaming/event platforms such as Kafka or similar technologies.
• Experience implementing secure and governed data environments with appropriate access controls, encryption, data lineage, and auditing capabilities.
• Background designing and supporting data platforms within regulated or government environments, including an understanding of compliance, security, and data protection requirements.
• Experience working with a variety of data sources, including legacy applications, APIs, IoT/telemetry systems, and relational databases.
• Proven ability to build highly available, reliable, observable, and production-ready data systems.
• Experience developing data infrastructure that supports machine learning and advanced analytics, including feature pipelines.
• Strong proficiency in Python and SQL, with experience using dbt preferred.
• Demonstrated ability to write clean, maintainable, production-quality code.
• Experience working with AWS, Azure, or GCP cloud environments.
• Familiarity with CI/CD and infrastructure-as-code practices is beneficial.
• Strong communication and collaboration skills with the ability to work effectively across technical and non-technical stakeholder groups.
• A systems-oriented approach with an emphasis on scalability, modularity, maintainability, and long-term platform development.
• Experience designing data platforms within U.S. public-sector or highly regulated environments.
• Working knowledge of applicable federal and state data privacy and security requirements, including HIPAA, CJIS, FERPA, and state-level privacy acts.
• Ability to incorporate compliance, governance, security, and auditability considerations into data architecture and engineering practices.