Moorthi Kumar

Results-driven Data Engineer with 6+ years of experience building large-scale ETL pipelines, cloud-native data platforms, and automated data delivery systems. Proven track record of architecting end-to-end AWS-based data infrastructure processing tens of millions of records. Strong background in Python, AWS, and web-scale data engineering.

Actively upskilling in PySpark and Spark optimization, with practical exposure through AWS Glue projects. Seeking a role that provides hands-on production experience with modern data warehousing (Redshift, Snowflake, BigQuery), Airflow, and streaming architectures.

Data Engineer - Forage AI - Remote

(2022-10)

Promoted from Data Automation Engineer — recognition of expanded scope and technical ownership

• Architected and maintained large-scale ETL pipelines delivering 50M+ people records and 200K+ company records to enterprise clients on recurring schedules
• Led AWS Glue ETL project: ingested ~7 million rows from S3 (CSV), applied PySpark transformations (foundational usage), and delivered final JSON output to client SFTP — end-to-end ownership
• Built internal Google crawling platform using browser rendering, cookie/session management — reduced dependency on 3rd-party providers and cut crawling costs significantly; processes 10,000 URLs in ~20 minutes
• Developed LinkedIn Company and People URL crawlers for US market data, solving a long-standing pipeline gap
• Built Twitter crawling infrastructure: logged-out crawler, historical tweet retrieval, company post monitoring, and tweet alerting systems
• Implemented AWS-native job orchestration using ECS, Lambda, CloudWatch, and API Gateway; integrated Slack webhooks for real-time pipeline monitoring and status alerts
• Used LLMs to parse complex product feature strings (HP CTO/STO), extracting structured CPU, RAM, and configuration data from unstructured text
• Implemented basic PySpark ETL for Bisnow pipeline: read raw data from database, applied transformations, and stored final output as CSV in S3 — building practical Spark experience
• Improved Bisnow delivery pipeline: automated QA checks, reduced turnaround time, and migrated crawling infrastructure to AWS
• Acted as internal expert for complex crawling challenges — resolved session handling, CAPTCHA bypass, and document retrieval issues for 3 team members; go-to resource for difficult websites
• Contributed to NASDAQ Key Professionals project, Expert Institute monitoring (doubled update frequency to 2x daily), Preqin social media pipeline, and BLK/ADV/DHC data pipelines

Product Development Engineer & Data Engineer - Datinfi Technologies - Coimbatore, India

(2019-06 - 2022-06)

• Built web scraping framework for 70+ UK sneaker websites (Sole Suppliers UK) using Scrapy; delivered near real-time restock alerts via Kafka, Apache NiFi, Mongo, and Algolia
• Developed government data extraction pipelines (LGA Enterprise UK): harvested structured data from PDFs, DOC files, images, Twitter, and Facebook
• Built product matching and counterfeit detection system (Wersel CounterFeitX) using Annoy, Computer Vision, NER, and sentiment analysis on product reviews
• Extracted and transformed product data from 180+ marketplaces (Corvid UK) including Amazon and Currys; matched CPU/GPU codes using text matching algorithms; stored reports in GBQ
• Developed Flask REST APIs for integrating scraped data into client platforms

B.E. - Mechanical Engineering - Sri Subramanya College of Engineering and Technology (2019-05)