Skip to main content

Python Databricks Data Engineer

Technology
Toronto, CanadaToday

Job description

Build Python and PySpark data pipelines at Electric Mind, supporting a financial services platform across development, quality assurance, and user acceptance testing environments.

This developer role is available permanently or by contract. You will maintain Databricks pipelines using SQL, medallion architecture, and curated data models. Working alongside a senior engineer, you will write clean transform code, resolve issues, and learn Databricks internals while supporting reliable platform operations.

Key Responsibilities:

  • Write PySpark transforms for silver and gold data layers
  • Apply Auto CDC and SCD Type 2 merge patterns
  • Query Unity Catalog using SQL warehouses
  • Debug pipeline logs, schema drift, and data quality issues
  • Maintain tests, scripts, documentation, and GitLab merge requests

Requirements:

  • Strong Python fundamentals, including functions, modules, and basic object-oriented programming
  • Databricks experience or willingness to learn quickly
  • Apache Spark, PySpark, or strong SQL exposure
  • Working knowledge of joins, aggregations, and window functions
  • Basic Git workflow and methodical debugging skills
Develop Python, PySpark, SQL, and Databricks expertise while improving financial services data pipelines at Electric Mind.

Interested in this role?