Senior Data Engineer with AWS Python FastAPI

Client

Our client is a leading legal recruiting company aiming to build a data-driven platform specifically designed for lawyers and law firms. The platform brings everything together in one place — news and analytics, real-time deal and case tracking from multiple sources, firm and lawyer profiles enriched with cross-linked insights, rankings, and more.

Project overview The platform aggregates data from hundreds of public sources including law firm websites, deal announcements, legal databases, and media publications creating a unified ecosystem of structured and interconnected legal data. It combines AI-driven enrichment, automated data processing, and scalable infrastructure to ensure comprehensive and reliable coverage of the legal market.

Position overview

We are seeking a Senior Data Engineer to join our team to design, build, and scale robust data pipelines for collecting, transforming, and structuring large volumes of legal and financial data collected via scrapers. You will collaborate closely with AI/ML engineers, DevOps, Front-end and Back-end teams to ensure smooth and efficient data workflows integral to the platform.

Responsibilities

• Design and implement data ingestion pipelines to collect and process structured and unstructured data from multiple online sources (web scraping, APIs, feeds, etc.).

• Develop and optimize ETL/ELT workflows using Python and SQL.

• Build and orchestrate scalable data workflows leveraging AWS services such as Batch and S3.

• Develop and deploy internal data APIs and utilities supporting platform data access and manipulation.

• Implement robust text extraction and parsing logic to handle diverse data formats.

• Ensure data quality through validation, deduplication, normalization, and lineage tracking across Raw ➝ Curated ➝ Enriched data layers.