Lead Data Engineer

Lead Data Engineer

Brazil- Remote

Long Term Contract

Job Description:

• Proven experience in designing and implementing data solutions on the Databricks platform.
• Solid understanding of Databricks fundamentals/architecture and have hands on experience in setting up Databricks cluster, working in Databricks modules (Data Engineering, ML and SQL warehouse).
• Create and maintain documentation for data pipelines, processes, and best practices.
• Knowledge of Performance Optimization, Monitoring and Automation would be a plus.
• Understanding of data governance, compliance, and security best practices.
• Knowledge in anyone of the cloud platforms AWS is a must.
• Strong Proficiency in Pyspark, Databricks Notebooks, SQL, Python, Scala, or Java.
• Certifications in Databricks would be beneficial.
• Strong understanding of data engineering best practices, data modeling, Data warehousing and ETL processes.
• Staying up-to-date with the latest Databricks features and best practices.
• Proficiency in SQL for data querying and manipulation.
• Strong problem-solving and troubleshooting skills.
• Excellent communication and collaboration skills