Nithyanand Jukanna

Data Engineer with 2+ years of experience in Hadoop & Spark, ETL and data warehousing techniques.

• My expertise includes designing and developing data warehouses, building ETL pipelines, and working with cloud-based (AWS) data solutions.
• I am passionate about leveraging data to drive business insights and value, delivering high-quality solutions that meet business needs.
• Experienced in optimizing spark queries by tuning and configuring parameters.
• Expert in Client interaction, requirement analysis and system study and analysis.
• Experienced in working with framework like Cloudera distribution.
• Good understanding of Spark Architecture and underlying Spark framework for processing a large amount of data near to real-time.
• Good at implementing advanced concepts in hive queries like partitioning, bucketing and hive analytic functions and user-defined functions (UDF)
• A relentless appetite and forever student to learn and adopt new technologies and solve a variety of problems customers face today
• Strong engineering professional with a Master's degree in Data Science and Engineering from UNIVERSITÀ DEGLI STUDI DI GENOVA, ITALY
• Experience in software development methodologies like Waterfall and Agile (Kanban and scrum)

• Prepared large sets of data for further processing using Hadoop, Spark, Hive, Data transfer (Sqoop)
• Performed Batch processing & Stream Processing (KAFKA) using Spark
• Involved in building data warehouse and data marts Designs
• Closely monitored data warehouse functionality and performance
• Built end to end pipeline in AWS using Lambda, S3, Glue , Crawler, Redshift, DynamoDB
• Implemented a real-time data pipeline to process data by integrating millions of raw records from different data sources using Kafka and PySpark and stored processed data in Redshift
• Automated ETL processes across billions of rows of data which reduced manual workload by 33% monthly
• Used Docker containerization tool to pull required image from docker hub and working on the containers
• Worked with relational databases and NoSql databases
• Extracted data from MongoDB atlas, Cassandra, MySQL and further processed in Spark
• Performed necessary optimization techniques in Hive and Spark
• Designed ETL pipelines which involved in reading form API, performing necessary
• transformations and Storing the date in Database
• Used Airflow to Schedule and run the automated ETL pipelines
• Followed workflow in Agile methodology

• Assisted senior data engineers, data architects to design and implement data models that support business requirements
• Involved in building and maintaining data pipelines, ETL processes, and data integrations between different systems
• Involved in database management, including data migrations, backups, and performance tuning
• Filtered data, based on conditions and pushed them to RDBMS and used for report generation
• Worked on different big data technologies like Hadoop, Spark and NoSQL databases
• Exposed to Spark Data Frames, Datasets and writing UDFs in spark
• Assisted in Analysing large data sets and troubleshooting issues related to data processing, resulting in positive impact of company overall performance
• Worked with different data formats like CSV, Parquet, AVRO, ORC ensuring data accuracy, consistency, and completeness, as well as identifying and resolving data quality issues which resulted in 25% decrease in errors.
• Documented data workflow, system architecture, and ETL processes, and provide reports to the team