Data Engineer | Pyspark ,SQL, Python, Hive
Send a job offer directly to this candidate
4 years of Total IT experience includes requirements gathering, data analysis, design, development, testing various modules, deployments, and documentation. Hands-on experience in Big Data Solutions using Hadoop, HDFS, Spark, Hive, Impala, Python, Sqoop. Excellent knowledge on Hadoop architecture, Map Reduce programming paradigm, Spark Architecture including Spark Core, Spark SQL and monitoring systems. Good Understanding of GCP Services like GCS, Big Query, Dataproc, Composer etc. Experienced in working with structured data using Hive QL, Hive UDFs, partitions, bucketing, ACID tables. Experience in utilizing Spark for ETL, involving the ingestion of diverse data sources, performing data transformations, and loading the processed data into target systems. Good understanding and experience with Software Development methodologies like Agile, Waterfall and Testing. Good Knowledge on Extraction, Transformation, Loading (ETL and ELT) data from various sources into Data Warehouses and Data lakes with industry best practices. Hands on experience in building wrapper shell scripts (UNIX) and analysis shell commands in practice. Good at SQL, data analysis, unit testing, debugging data quality and performance issues.
4 years of Total IT experience includes requirements gathering, data analysis, design, development, testing various modules, deployments, and documentation. Hands-on experience in Big Data Solutions using Hadoop, HDFS, Spark, Hive, Impala, Python, Sqoop. Excellent knowledge on Hadoop architecture, Map Reduce programming paradigm, Spark Architecture including Spark Core, Spark SQL and monitoring systems. Good Understanding of GCP Services like GCS, Big Query, Dataproc, Composer etc. Experienced in working with structured data using Hive QL, Hive UDFs, partitions, bucketing, ACID tables. Experience in utilizing Spark for ETL, involving the ingestion of diverse data sources, performing data transformations, and loading the processed data into target systems. Good understanding and experience with Software Development methodologies like Agile, Waterfall and Testing. Good Knowledge on Extraction, Transformation, Loading (ETL and ELT) data from various sources into Data Warehouses and Data lakes with industry best practices. Hands on experience in building wrapper shell scripts (UNIX) and analysis shell commands in practice. Good at SQL, data analysis, unit testing, debugging data quality and performance issues.
B.Tech in Computer Science and Engineering