Ajar Khan

Results-driven Data Engineer with 6+ years of experience designing and building scalable data pipelines, ETL frameworks, and cloud-native solutions across AWS and Azure ecosystems. Adept at leveraging tools like PySpark, Kafka, Snowflake, and Airflow to process large-scale structured and unstructured data for analytics and real-time decision-making. Experienced in developing robust Python applications with PyTest-based unit testing, ensuring code quality and reliable ETL deployments.

Skilled in AWS tools like Glue, EC2, EMR, Redshift, Lambda and S3 for scalable cloud-based data solutions. Hands-on expertise in Python and PySpark for building ETL pipelines, developing custom UDFs, and validating data across multiple systems. Extensive experience in Snowflake Data Warehouse, including designing data models using star and snowflake schemas and operationalizing analytics solutions.

Hands-on experience with Azure Databricks, Delta Live Tables (DLT), and Unity Catalog for building scalable, governed, and secure enterprise data platforms. Experienced in real-time data streaming and processing using Kafka, developing custom producers and consumers, and integrating with Spark and Scala. Strong knowledge of TSQL and NoSQL databases (HBase, MongoDB) with experience in transferring, transforming, and validating data across various formats.

Created reusable data assets and pipelines for AI/ML experimentation, improving cross-team collaboration and reducing redundant effort. Developed and deployed machine learning and deep learning models using TensorFlow, PyTorch, and NLP techniques, improving prediction accuracy for key business use cases.

Used Azure

DevOps to plan, track, and deliver iterative data engineering solutions, integrating CI/CD pipelines for automated deployment and testing. Maintained and optimized a monolithic ETL/ELT pipeline built on Matillion/Airflow, supporting ingestion, transformation, and reporting for multiple business domains.

Experience in automating data validation scripts and creating regression workflows between multiple databases like Oracle, SQL Server, and Hive using Python. Built infrastructure-as-code using Terraform and CloudFormation to provision and manage AWS resources including ECS, S3, and EC2 GPU instances.

Data Engineer-Financial Domain - Freshbooks

(2023-07)

• Designed and implemented end-to-end data lake and warehouse solutions using S3, Glue, Redshift, and EMR forlarge-scale financial and customer analytics.
• Developed ETL pipelines in AWS Glue to transform raw JSON/CSV into optimized Parquet formats, improving query performance by 40%.
• Developed PyTest-based automated unit and integration test cases for Python ETL pipelines, improving deployment quality and reducing production defects.
• Implemented Lambda functions for real-time fraud detection and event-driven alerts, reducing detection latency from hours to seconds.
• Orchestrated serverless ETL workflows with AWS Glue and Step Functions, reducing pipeline runtime by 35% and improving reliability. Built data lake architecture on S3 with raw, curated, and analytics zones for structured and semi-structured data.
• Leveraged Unity Catalog to implement centralized data governance, access control, metadata management, and secure data sharing across multiple environments.
• Designed and implemented TypeScript-based microservices for serving ML models and integrating AI-driven features into cloud applications.
• Architected star and snowflake schemas, SCDs, facts, and cubes, enabling faster BI insights and reporting.
• Monitored application health and performance using DataDog and Sentry, ensuring high availability and quick incident resolution.
• Developed real-time data ingestion pipelines with Kinesis + Lambda, enabling fraud detection and customer notifications within seconds.
• Designed and implemented Delta Live Tables (DLT) pipelines in Databricks to automate data ingestion, transformation, and quality validation for batch and streaming workloads.
• Developed and maintained Python/Django applications with RESTful APIs, integrating Redis for caching and Celery for background task processing

Data Platform Engineer - CI financial

(2022-01 - 2023-06)

• Developed and managed data pipelines using AWS Glue and Step Functions, ensuring efficient ingestion and transformation of 50+ TB of data monthly from on-premises systems into the AWS Cloud environment, addressing technical challenges and maintaining clear communication with stakeholders.
• Leveraged AWS Glue and Redshift Spectrum to transform raw data from S3 data lakes into relational formats for storage and analytics in Redshift, improving data storage efficiency by 40% and processing times by 25%, while providing status updates and ensuring alignment with business goals.
• Developed secure, scalable data ingestion pipelines into AWS S3, facilitating downstream analytics and machine learning processes, contributing to a 15% fastertime-to-insight.
• Configured Unity Catalog for centralized governance, data lineage, schema management, and role-based access control across Databricks workspaces.
• Developed automated validation suites using PyTest to verify ETL logic, schema consistency, and data quality before production deployments.
• Designed and implemented Databricks ETL pipelines using notebooks, Spark Data Frames, SPARK SQL, Python scripting, Delta Live Tables, and Databricks Workflows for scalable enterprise data processing.
• Designed and deployed scalable AWS cloud architectures leveraging Docker containers for efficient CI/CD and microservice isolation.
• Played a key role in scaling systems for finance applications, emphasizing self-driven ownership and continuous development.
• Collaborated with cross-functional teams, including product, finance, and analytics, to solve complex data problems and address key business issues.

Data Analytical Engineer - PointClickCare

(2020-07 - 2021-12)

• Developed JSON scripts for deploying Azure Data Factory (ADF) pipelines to streamline data processing.
• Integrated Azure Data Factory with Azure Databricks, implementing automated orchestration and scalable ETL pipelines.
• Applied statistical and quantitative techniques to develop predictive models supporting market trend analysis and business forecasting initiatives.
• Built scalable ETL frameworks using Informatica and SSIS to load large-scale data into Azure Data Lake and Synapse Analytics.
• Implemented data validation, profiling, and metadata management to ensure compliance with governance standards.
• Automated manual ETL tasks using Python and Shell scripts, reducing operational overhead by 50%.
• Partnered with business teams to gather requirements and translate

BI/Data Analyst - Lightspeed Commerce

(2019-01 - 2020-06)

• Created stored procedures using PL/SQL and tuned the databases and backend process.
• Determined data rules and conducted Logical and Physical design reviews with business analysts, developers, and DBAs.
• Used data mining to extractinformation from data sets and identify correlations and patterns.
• Enforced standards to ensure that the data elements and attributes are properly named in the data model and the Source to Target mapping.
• Organized and transformed information into comprehensible structures.
• Used MDX Expressions, DAX Expressions, Power BI Power Pivot, Power integrated with Share Point and in creating dashboards in Power Bl and Tableau visualization tools.
• Involved in project cycle plan for the data warehouse, source data analysis, data extraction process, transformation and ETL loading strategy designing.
• Collaborated with the enterprise Data Modeling team to create Logical models.

Bachelor of Engineering - Mechanical engineering - Jawaharlal Nehru Technological University Hyderabad (2011 - 2015)