Sreeja P

I've been working in big data ecosystems like Spark, Hive, Athena, Python, Pyspark, Redshift, AWS kinases, and DynamoDB. I’m currently working in architecture where we ingest data both in batch and real-time. for batch, we ingest data from Teradata and SQL Server land data in S3, write Pyspark ETL scripts to read data from S3, and do all the computations in Spark and EMR, write data back into Hive and Athena, and I also export the latest snapshot data into Redshift for analytical purposes, where we have visualization developers that build Power bi / Tableau dashboards on top of that, for real-time I write Spark streaming applications where we ingest data from AWS kinases and write data into DynamoDB for API consumption. our downstream consumers are stakeholders, data scientists, and other engineering teams as well. We use airflow DAGs to schedule all the workflows we develop in Python, and that's pretty much about my experience in my current project as well.

I've been working in big data ecosystems like Spark, Hive, Athena, Python, Pyspark, Redshift, AWS kinases, and DynamoDB. I’m currently working in architecture where we ingest data both in batch and real-time. for batch, we ingest data from Teradata and SQL Server land data in S3, write Pyspark ETL scripts to read data from S3, and do all the computations in Spark and EMR, write data back into Hive and Athena, and I also export the latest snapshot data into Redshift for analytical purposes, where we have visualization developers that build Power bi / Tableau dashboards on top of that, for real-time I write Spark streaming applications where we ingest data from AWS kinases and write data into DynamoDB for API consumption. our downstream consumers are stakeholders, data scientists, and other engineering teams as well. We use airflow DAGs to schedule all the workflows we develop in Python, and that's pretty much about my experience in my current project as well.

masters in Data Science bachelor's in Computer Science and engineering