Sai Lokesh

Having 8+ Years of IT experience, which incorporates designing and developing Data Models and orchestrating Data Pipelines for ingesting and transforming structured and semi-structured data to various destinations like Azure SQL, Synapse or Datalake gen2 which can be further used for BI, data science or machine-learning workloads. Good understanding of Data Modeling, Data warehousing and performance tunning ETL processes. Extensive knowledge in SSIS, ADFv2, Databricks, PySpark, SparkSQL for building data ingestion and processing frameworks which can be reused.

Have good exposure to data governance and implementation of data governance in datalakehouse architecture using unity catalogue in azure databricks. Worked with micro batch processing using DLT pipelines for high frequency data refresh. Along with data ingestion and transformation built ADFv2 pipelines to refresh power bi datasets, built a automated workday integration monitoring system to raise service now tickets (used databricks) and ADFv2 and alert systems to send email notifications (used logic apps).

All secretes in ADFv2 pipelines were secured in Azure Key Vault. Extensive knowledge on cloud technologies Azure Data Factory, Azure Key Vault, Azure Data Lake Storage Gen2, Azure Synapse Analytics (Warehouse), Azure Synapse (pipelines), Azure SQL Database, Azure Databricks, Azure Active Directory, SQL. Expertise in Microsoft T-SQL, DDL, DML, views, stored procedure, functions and modifying database schemas.

Fair handson on cutting edge analytics reporting tools like Power BI, Tableau and other power platform tools like Power apps and Power flows. Committed to give impeccable result and performing under minimal supervision, multi-tasking, meet deadlines as an individual contributor and a committed team player.

Azure Data Engineer - Northern Trust - Chicago

(2024-06)

• Built 20+ data pipelines using Azure Data Factory, integrating data from SQL, REST APIs, and on-prem systems to Data Lake Storage Gen2.
• Developed complex transformation notebooks in Azure Databricks (PySpark), optimizing performance with Delta Lake and partitioning strategies.
• Automated ingestion using ADF triggers, parameterized pipelines, and dynamic datasets.
• Integrated ADF with Azure Key Vault for secure credentials and monitored pipeline health using Azure Monitor.
• Implemented data quality checks and validation rules using Databricks notebooks to ensure accuracy before data ingestion into curated layers.
• Created event-based triggers in ADF to automate data processing based on blob creation, reducing latency in real-time scenarios.
• Utilized Mount Points and Service Principals in Databricks to securely connect to Data Lake Gen2 and streamline read/write access.
• Designed and published Power BI dashboards for sales, finance, and operations with row-level security and scheduled refresh.
• Created Power BI Dataflows to centralize transformation logic and reduce duplication.
• Implemented incremental refresh for large datasets (100M+ rows) by setting up Range Start and Range End parameters, improving refresh performance by 75%.
• Ensured query folding was preserved in Power Query to support incremental refresh and efficient transformations.
• Applied row-level security (RLS) and integrated with Power BI Premium capacity for scalable enterprise reporting.

Azure Data Engineer - Optum - 1325 Boylston St, Boston, MA 02215

(2020-08 - 2024-05)

Metadata Driven Framework to Build a Lakehouse: Developed metadata driven Data pipelines using ADFv2 to load data from different file formats to data Lakehouse. Developed generic databricks notebooks to load data from landing layer to curated layer. Also built generic databricks notebooks to populate SCD Type 1 and SCD Type 2-dimension tables. Framework has self-logging mechanism to catch run details and failure details. In case of failure, framework is intelligent enough to raise a service-now ticket based on configured capability. Wrote custom fact notebooks to built fact. All custom and generic notebooks execute vacuum command for better space management. Enabled cluster pooling for better startup time. Helped analytics teams to create tokens and configure simba spark connector for power bi.

Workday Data ingestion Using Azure SFTP: Existing workday to ADLS gen2 connectivity was through Mulesoft which needed additional development efforts. Explored and configured ADLS Gen2's SFTP to get data files directly from Workday which saved 5 FTEs for each data pull considering Mulesoft developers efforts. Data pulled from Workday further processed by Metadata framework explained above. Wo

• Designed and developed Power BI graphical and visualization solutions with business requirement documents and plans for creating interactive dashboards.
• Created Dataflows in Power BI in order with Dimensional tables that can help users to build self-service reports, that in fact increased the re-usability of data.
• Utilized Power BI to create various analytical dashboards that depicts critical KPIs such as legal case matter, billing hours and case proceedings along with slicers and dicers enabling end-user to make filters.
• Created Paginated reports to support end users to extract huge volume of data

Sr. SQL BI Developer - HealthX - 9225 Priority Way W Dr #100, Indianapolis, IN 46240

(2020-05 - 2020-07)

• Created Power BI reports and dashboards over the course which mainly dealt with Inpatient data
• Worked on Power BI reports with complex DAX logics and Power query and helped the business users to build their own complex reports
• Rebuilding Indexes and Tables as part of Performance Tuning Exercise
• Used different Control Flow Tasks and Data flow Tasks for creating SSIS Packages. Designed packages using different types of Transformations for Data Conversion, Derived Columns with Multiple Data Flow tasks
• Created user variables, property expressions, script tasks in SSIS also responsible for Deploying, Scheduling Jobs, Alerting and Maintaining SSIS packages.

Software Engineer/Data Engineer - Indiana University Health - 1633 N Capitol Ave, Indianapolis, IN 46202

(2019-10 - 2020-05)

• Designed, developed, and maintained data processing solutions using Azure Databricks, Apache Spark, and Python.
• Built scalable data pipelines to ingest, transform, and load large volumes of data from various sources into Azure data lakes and data warehouses.
• Optimized Spark jobs and Databricks clusters to improve data processing efficiency and reduce costs.
• Implemented data security and compliance measures within Azure Databricks environments to ensure data privacy and regulatory compliance.
• Collaborated with cross-functional teams to gather requirements, define data architectures, and deliver solutions that meet business needs.
• Conducted performance tuning and optimization of Databricks notebooks and Spark SQL queries to enhance query performance and overall system efficiency.
• Integrated Custom Visuals based on business requirements using Power BI desktop.
• Developed profit analysis reports and dashboards by products and customers
• Creating Row level security with power bi and integration with power bi service portal to maintain and handle security of confidential data.

Data Engineer - Kroger - 1014 Vine St, Cincinnati, OH 45202

(2019-01 - 2019-10)

• Designed and implemented end-to-end data integration solutions using Azure Data Factory to ingest, transform, and load data from diverse sources into Azure data lakes, data warehouses, and analytics platforms.
• Developed and optimized data pipelines using Azure Data Factory to process large volumes of structured and unstructured data efficiently, ensuring high performance and scalability.
• Designed and implemented complex data transformation processes using Azure Data Factory to cleanse, validate, and transform data from various sources.
• Utilized Azure Data Factory's transformation activities such as Mapping Data Flows, Data Flow transformations, and SQL transformations to perform data wrang