3 weeks ago Be among the first 25 applicants
About the Role:
We are looking for a talented Platform & Data Engineer to join our greenfield project, where you'll play a critical role in building and deploying a cutting-edge data platform in Azure Databricks. As part of a highly technical team, you'll be responsible for designing, implementing, and managing the entire infrastructure pipeline, from provisioning cloud resources in Azure to deploying scalable data solutions on Databricks.
This role is ideal for an engineer who thrives at the intersection of platform engineering, data engineering, and cloud infrastructure. You'll be working with modern Infrastructure as Code (IAC) practices, automating deployment workflows, and ensuring the robustness and scalability of the entire platform. Your expertise will be pivotal in ensuring the seamless operation of the platform, from cloud infrastructure provisioning to continuous deployment and monitoring on Databricks.
What Your Day Will Look Like: Designing, deploying, and managing cloud infrastructure on Azure, ensuring it supports the data platform and applications at scaleImplementing and managing Infrastructure as Code (IAC) using Terraform to automate provisioning and maintain cloud resourcesSupporting Databricks deployment, including the integration of data pipelines, notebooks, and cluster managementWorking with the broader data engineering team to optimise the data platform for scalability, performance, and securityCollaborating with other engineering teams to ensure a smooth deployment process and end-to-end delivery of the solutionAutomating continuous integration and continuous deployment (CI/CD) pipelines for the data platform, ensuring efficient and reliable delivery of updatesEnsuring platform security, monitoring, and alerting are in place to maintain a high level of uptime and performanceAssisting with troubleshooting and optimising infrastructure and data pipelines to meet operational and performance goalsRequirements: 5+ years of experience in platform and data engineering, with hands-on experience building and managing cloud-based platformsStrong experience with Azure cloud infrastructure, including provisioning resources, managing services, and configuring network securityExpertise in Infrastructure as Code (IaC) using Terraform or Bicep for automating cloud resource provisioning and infrastructure managementExperience deploying, managing, and optimising Databricks environments, including clusters, notebooks, jobs, and integrations with data pipelinesA solid understanding of CI/CD pipelines, automation, and deployment processes for cloud-based data platformsStrong technical background in data engineering, including building data pipelines, managing large-scale data systems, and working with cloud-native toolsFamiliarity with dbt and data modelling principles to work closely with the data engineering team in deploying solutionsExcellent communication and problem-solving skills, with the ability to collaborate effectively across multiple teamsAbility to quickly adapt to new technologies and solve complex infrastructure and deployment challengesBonus Points: Experience with machine learning pipelines or MLflow on DatabricksFamiliarity with containerisation technologies like Docker or Kubernetes for managing data workflowsExperience with monitoring tools, FinOps and cost management for cloud infrastructure and Databricks environmentsBenefits: Competitive Salary packagePick your equipmentGifted Day Off ("VersionUp Day")Generous parental leaveWell funded start up $$$Annual Training budgetMentorship ProgramClear promotion pathwaysFlexible workingMeet-ups & socials97% Rating on corporate social responsibilityIf you are passionate about building robust platforms and data solutions at scale, automating infrastructure, and ensuring smooth deployment workflows in Azure Databricks, we'd love to hear from you!
Seniority levelMid-Senior level
Employment typeFull-time
Job functionInformation Technology
IndustriesIT Services and IT Consulting
#J-18808-Ljbffr