Database Development & Administration (Information & Communication Technology)Compare the Market – yes, the one with the meerkats – is a high-growth FinTech company on a transformational journey. We are focused on designing, building, and delivering the very best home loan, energy, insurance, and health comparison experience in Australia.We are all about Customer first, teamwork, making a difference, and treating the business as our own – without compromise. These values are critical to delivering on our purpose: "to make a difference in the lives of Australians by simplifying the process of making sound financial decisions".About The Role:Are you ready to take your career to new heights? Compare the Market is searching for a Senior Data Engineer to join our innovative team in Toowong. As we continue to evolve our modern data platform on Databricks/AWS, we're dedicated to delivering scalable data solutions that drive actionable insights that support our customers and ongoing business growth.Reporting to the Head of Data Science, you'll have the unique opportunity to establish foundational engineering principles and provide technical leadership across business intelligence, data science, and machine learning initiatives.As a Senior Data Engineer, your expertise extends beyond hands-on development to include guiding architecture and solution design. You'll play a pivotal role in shaping our data strategy from an engineering perspective, contributing to the development of our data platform and ecosystem.Supported by a collaborative team including data scientists, business intelligence developers, data architects, and more, you'll collaborate closely with IT and business teams to drive innovation and excellence.What You'll Do:Design and develop robust, scalable, and high-performance data pipelines to process large volumes of data from heterogeneous sources.Optimise data pipelines for performance, reliability, and scalability using best practices in data engineering.Implement data models and schemas for efficient storage and retrieval of structured and unstructured data.Collaborate with commercial and other teams to understand data requirements and translate them into technical solutions.Collaborate with cross-functional teams to support data-driven decision-making and deliver actionable insights.Partner with architects to devise architectural blueprints for data pipelines, emphasising event-based real-time processing capabilities.Advocate for best practices in data management and governance, encompassing data modelling, ETL efficiency, data quality assurance, observability, and discoverability, as well as security protocols for handling sensitive data and ensuring codebase quality.Implement data governance processes and ensure data quality and integrity throughout the data lifecycle.Provide operational support during the evaluation and integration of new tools and technologies to enhance data processing capabilities and improve workflow efficiency.Mentor and provide technical guidance to junior members of the data engineering team.What You'll Bring:5 years plus proven experience as a senior data engineer.Extensive expertise in designing and developing ETL/ELT processes, data pipelines, and data lakehouse platforms.Extensive experience as a professional data engineer, specialising in developing ETL pipelines for production environments.Strong understanding of data architecture principles, data modeling, and schema design.Proficiency in programming languages such as Python, Java, or Scala, and experience with SQL and NoSQL databases.Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and data technologies in a cloud environment.Excellent problem-solving skills and ability to work independently as well as part of a team.Strong communication skills with the ability to collaborate effectively with stakeholders across different functions.Experience in Agile methodologies and familiarity with DevOps practices is a plus.Demonstrated ability to provide data engineering leadership and mentorship to junior data engineers.At Compare the Market, we offer a dynamic and collaborative work environment where your talents will be recognised and rewarded. You'll have the opportunity to make a meaningful impact and contribute to our continued success. As well as ample opportunities for growth and development, we also provide:Vibrant and social community who have annual celebrations, family fun days and regular events.Enjoy flexible work arrangements, including the option of one day working from home each week or a 9-day fortnight.Enjoy additional leave days - 'ME' leave and 'Volunteer Day' leave.Study support (up to 50% of tuition fees for relevant courses) and professional membership reimbursement.Employee discounts on Car, Home, Travel insurance.Income protection insurance provided to support you in the event of non-work-related illness or injury.Discounts with a broad range of providers, including gym memberships, accommodation, dental care and more.Compare the Market is an Equal Opportunity Employer, and we encourage all applicants to apply. We acknowledge that we are strengthened by diversity.Please note that this role is only available to candidates with unlimited Australian working rights, as sponsorship is not available.
#J-18808-Ljbffr
