InfoObjects Software - Data Engineer - Azure Databricks/Apache Spark

Key Responsibilities :

• Design, develop, and manage data pipelines using Databricks (Spark, Delta Lake).
• Optimize large-scale data processing workflows for performance and reliability.
• Collaborate with Data Scientists, Analysts, and Business Stakeholders to gather requirements and deliver actionable insights.
• Maintain and enforce data quality and integrity across multiple data sources and systems.
• Work with cloud data platforms such as Azure, AWS, or GCP.
• Implement data governance and lineage tracking using tools like Unity Catalog, Great Expectations, or similar.
• Monitor, debug, and troubleshoot data pipelines and jobs.

Required Qualifications :

• 3+ years of professional experience in data engineering or similar roles.
• Strong experience with Databricks, Apache Spark, and Delta Lake.
• Proficient in SQL, Python, and distributed data processing concepts.
• Experience working with cloud platforms (Azure/AWS/GCP) and cloud-native tools.
• Hands-on experience with ETL/ELT processes, data warehousing, and modern data stack.
• Familiarity with CI/CD practices and version control tools (e.g., Git).
• Strong problem-solving skills and ability to work independently or in a team environment.