André Ichiro

Hi there! I am a data engineer and continuous learner, driven by a passion for leveraging data to drive meaningful insights and solutions. My focus is on designing and implementing ETL and ELT data pipelines and managing the end-to-end data lifecycle.

Transformations and Analysis

Skilled in Python, Pandas, Spark, Pyspark, dbt, and Seaborn for data transformations and insightful analysis.

Cloud

Utilized various AWS services such as Glue, S3, Lambda, Kinesis, documentDB, ECS, Fairgate and API Gateway, alongside with Databricks and Snowflake for analytics and data processing.

Data Storage

Working in SQL, Postgres and MongoDB databases to implement data storage solutions.

DevOps

Familiarity with CI/CD practices, containerization using Kubernetes and Docker, version control with Git & GitHub, and infrastructure automation with Terraform.

Messaging Systems

While my experience with Kafka is limited, I have some exposure to it for real-time data streaming and event-driven architectures.

Machine Learning

I have a foundational understanding of PyTorch, TensorFlow, exploratory data analysis and basic supervised, semi and unsupervised learning techniques, as well as recommender systems.

Back-end & Web:

I have gained experience in developing back-end systems using Python, Django, Django REST API, and following software engineering principles.

Architecture and Best Practices

Adhere to industry best practices such as Pep 8, SOLID, and Domain Driven Design principles.

Background

With a background in computer science, law (+10 years of experience in international private law and BA's degree with honors) and entrepreneurship (product, marketing, sales), I bring a unique perspective to problem-solving with the ability to collaborate effectively in interdisciplinary environments

In my most recent personal project, I developed an automated ELT data pipeline using a variety of tools and technologies including multiple AWS Services, Spark, Python, Pandas, Terraform, Docker, Ansible, Airflow, Databricks, GitHub Actions and dbt cloud. This project provided insightful video analysis and trained a Longformer Language Model (LLM) using YouTube and Whisper transcriptions.

Competências: Terraform · Databricks · Infrastructure as code (IaC) · Data Lake · AWS Glue · AWS Athena · AWS Lambda · Amazon Kinesis · dbt · AWS API Gateway · Data Warehousing · Data Engineering · Data Modeling · Docker · Pandas (Software) · Apache Spark · Ansible · GitHub · PySpark · Data Pipelines · Apache Airflow · SQL · Extract, Transform, Load (ETL) · Python · Design Patterns · DevOps

Universidade Presbiteriana MackenzieUniversidade Presbiteriana Mackenzie

Bachelor's degree, Computer Science