Solution Architect - Data Engineering

Experience :

• 10 years of overall IT / Consulting experience.
• 7 years of experience in Data Engineering, Big Data, Analytics, or related technologies.
• 68 end-to-end Databricks implementation projects.
• Proven experience in Solution Architecture with strong client-facing exposure.

Mandatory Technical Skills :

• Databricks: Strong hands-on experience and solution architecture expertise.
• Apache Spark / PySpark: Deep expertise, including Spark internals and performance tuning.
• Delta Lake / Lakehouse: Strong understanding of Lakehouse architecture and design patterns.
• Unity Catalog: Security, governance, access control, and data management.
• Databricks Platform: Workflows, Auto Loader, Databricks SQL, batch and streaming pipelines.
• Spark Performance Optimization: Partitioning, shuffle optimization, data skew, join optimization, AQE, and Catalyst.
• Databricks Compute: Cluster sizing, autoscaling, Photon, and workload optimization.
• Programming: Python / PySpark and Advanced SQL.
• Data Engineering: Large-scale ETL/ELT pipeline design and implementation.
• Cloud: Deep expertise in at least one cloud platform (AWS / Azure / GCP) and working exposure to a second cloud.
• CI/CD & DevOps: Git, Terraform, Databricks Asset Bundles, and at least one of Azure DevOps, GitHub, GitLab, or Jenkins.
• Security & Governance: Databricks security, governance, access management, and enterprise data controls.
• MLflow / MLOps: Working knowledge of MLflow and MLOps concepts.
• Certification: Databricks Certified Data Engineer Professional Preferred / Highly Preferred.

Solution Architecture & Client-Facing Skills:

• Strong experience conducting architecture workshops and requirement-gathering sessions.
• Ability to translate business requirements into scalable Databricks / Lakehouse solutions.
• Experience in solution design, architecture documentation, technical presentations, and client discussions.
• Strong troubleshooting, problem-solving, and stakeholder management skills.
• Ability to lead technical discussions with both business and engineering stakeholders.