Senior Data Engineer

Key ResponsibilitiesDesign, implement, and maintain a robust data platform on Databricks, supporting the ingestion, storage, transformation, and processing of large volumes of data from various sources.Design and develop Azure cloud data products (Data warehousing, data integrations, dashboards etc.) to enhance data services.Leverage Databricks' capabilities to enhance data workflows, implementing advanced features and automations to support predictive modeling and machine learning.Gather, document, and clarify requirements; design solutions; develop code using Microsoft BI stacks; test and support UAT; monitor product functionality.Improve the data product development and support process through automation, including access requests, data refreshes, releases, source control, and knowledge management.Ensure functional and non-functional quality through data profiling, automated and manual tests, and performance testing.Key SkillsProficiency in SQL, Python, and PySpark for data processing and transformation.Proven track record in using Databricks for data engineering, including cluster management, Delta Lake, and other Databricks-specific features.Devops: Experience automating aspects of product development and support such as automated releases and calling API's.Experience with cloud platforms (AWS, Azure, or GCP) and associated data services.Excellent communication and collaboration skills, with the ability to convey technical concepts to non-technical stakeholders.
#J-18808-Ljbffr