Mit Bharodiya

Senior Software Engineer specializing in Data Engineering and AI-driven systems, with 10 years of experience designing ETL/ELT pipelines, data warehouses, data lakes, and lakehouse architectures on AWS. Expert in Python, SQL, PySpark, Databricks, Airflow, Dagster, and dbt, with strong experience building data ingestion frameworks, web data extraction and scraping pipelines, API-based data collection systems, and processing large-scale structured and unstructured datasets. Experienced in LLM API integration (OpenAI, Claude), embedding-based search and recommendations, RAG and vector search pipelines, and ML-ready datasets and feature pipelines for data science teams.

Applied machine learning in real-time domains, including anomaly detection and forecasting on streaming financial market data. Strong record of infrastructure automation with Terraform and GitHub Actions CI/CD, distributed processing at billions-of-records scale, and cross-functional delivery across fintech, marketing, and enterprise environments.

AI Training Data Contributor at Alignerr (2025-03 – 2026-07)

• Authored 100+ domain-specific evaluation tasks with verified reference solutions, designed to surface reasoning failures in frontier language models.
• Evaluated paired model responses across helpfulness, factual accuracy, safety, and instruction-following rubrics, producing written rationales for each judgment.
• Identified recurring failure modes, including confidently stated but jurisdiction-dependent claims and unrequested assumptions on underspecified prompts, and documented them for downstream training use.
• Maintained a 95% first-pass acceptance rate against reviewer quality standards across 500+ completed tasks.
• Reviewed and calibrated submissions from other contributors, providing correction feedback and flagging ambiguous or unverifiable items before delivery.
• Applied structured annotation guidelines consistently at volume, sustaining inter-annotator agreement above [client threshold] on held-out calibration sets.
• Wrote unambiguous problem statements with self-contained context and single defensible answers, reducing rework caused by specification ambiguity.
• Collaborated with project leads to refine rubric language where guidelines produced inconsistent judgments on edge cases.

Senior Software Engineer at BIGSCAL TECHNOLOGIES Pvt. Ltd (2022-03 – 2026-06)

• Design and maintain ETL/ELT pipelines ingesting data from APIs, relational databases, event streams, external vendors, and web-scraped sources into enterprise analytics platforms.
• Architect dimensional models, star schemas, and curated data marts, with automated validation, cleansing, and enrichment frameworks that keep datasets trusted for self-service analytics.
• Orchestrate hundreds of production pipelines with Airflow and Dagster, backed by monitoring, alerting, and observability practices.
• Build distributed processing pipelines on Databricks, PySpark, and AWS EMR for high-volume batch and near-real-time workloads; tune Spark jobs through partitioning, execution-plan optimization, and resource management.
• Design secure cloud architectures on AWS (ECS, Lambda, EC2, IAM, Step Functions) with automated infrastructure deployment and operational monitoring.
• Develop web scraping and data collection pipelines in Python - HTML parsing, pagination, content normalization, metadata enrichment, and quality validation - feeding analytics and AI applications.
• Prepared datasets, features, and pipelines consumed by data scientists and ML models, ensuring data quality and reproducibility for training and analytics workloads.
• Built embedding-based search and recommendation logic and integrated LLM APIs (OpenAI, Claude) into internal services and tooling.
• Build RAG data pipelines: document preprocessing, chunking, embedding generation, and vector indexing that power semantic search and LLM question-answering over enterprise knowledge repositories.
• Maintain feature engineering pipelines for predictive analytics and ML training, with automated data validation and monitoring to safeguard model performance.
• Optimize warehouse performance through query tuning, indexing, and partitioning; partner with data scientists and analysts to deliver production-ready datasets.

Backend Engineer at Modvise (2020-05 – 2022-03)

• Designed and developed Python and Node.js services supporting data ingestion, processing, and API-driven applications across research and enterprise environments.
• Built and optimized PostgreSQL and MySQL databases - schemas, indexes, and query strategies that improved performance and scalability for large datasets and high-traffic workloads.
• Led development of data warehouses, data lakes, and lakehouse architectures on AWS, enabling efficient analytics across billions of records and terabytes of data.
• Developed ETL and ELT pipelines using SQL, Python, dbt, PySpark, and AWS services to automate data movement, transformation, and validation across multiple systems.
• Automated infrastructure provisioning and deployment workflows with Terraform and GitHub Actions, reducing manual operations and accelerating delivery cycles.
• Worked closely with product, analytics, and business teams to design data models, reporting layers, and backend systems aligned with operational and analytical requirements.
• Improved platform reliability through monitoring, observability, automated testing, and data-quality frameworks; optimized distributed workloads and database performance, cutting execution times and infrastructure costs.
• Contributed across the full software development lifecycle - architecture, implementation, deployment, support, and continuous improvement of production systems.

Backend developer at Tata consultancy services (2018-11 – 2020-03)

• Developed a high-demand, user-friendly configuration management feature for Laiqa customers using React and JSON forms on the front-end, and Python and MySQL on the back end.
• Upgraded the project significantly, making extensive modifications to accommodate the new feature, resulting in a 95% reduction in configuration time, from 20 minutes to a few seconds.
• Designed a full-stack application using Django, MySQL, HTML, CSS, and JavaScript for a responsive web app, featuring role-based access, face recognition for attendance, location-based work verification, automation dependency alerts, and tracking of login/clock-out times to streamline workforce operations for employees and vendors.

Master's Degree in MCA – Veer Narmad South Gujarat University (2016 – 2018)

Bachelor's Degree in BCA – Veer Narmad South Gujarat University (2013 – 2016)