Lead Data Engineer

Database Development & Administration (Information & Communication Technology)
Compare the Market – yes, the one with the meerkats – is a high-growth FinTech company on a transformational journey.
We are focused on designing, building, and delivering the very best home loan, energy, insurance, and health comparison experience in Australia.
Okay, in the world!
We are all about Customer first, teamwork, making a difference, and treating the business as our own – without compromise.
These values are critical to delivering on our purpose: "to make a difference in the lives of Australians by simplifying the process of making sound financial decisions". About The Role: Are you ready to elevate your career?
Compare the Market is seeking a skilled Lead Data Engineer to join our Enterprise Data Team.
In this role, you'll lead a team of talented data engineers and work closely with business stakeholders, data scientists, and developers to drive data-driven initiatives.
As the Lead Data Engineer, you will be a leader with strong data engineering and cloud competencies, committed to developing and mentoring your team.
This role is pivotal in shaping the way the team executes its tasks to deliver on business strategy and align with our architectural vision.
You will be responsible for designing, building, and maintaining robust data pipelines and cloud-based data solutions.
Your role will encompass strategic planning (defining the technical direction and best practices), operations (overseeing the implementation and maintenance of data systems), and stakeholder management (collaborating with various departments to understand their data needs and ensure effective data solutions).
Additionally, you will advocate for the use of advanced data technologies and promote a data-driven culture within the organisation.
This is a unique opportunity to combine strategic vision with hands-on leadership, ensuring that our data capabilities continue to advance and support our business goals.
What You'll Do: Serve as the technical expert with strong domain knowledge in data solutions and cloud implementations, leading the design and implementation of robust, scalable, and high-performance data pipelines to process and integrate large volumes of data from heterogenous sources.Take hands-on responsibility for the end-to-end delivery of data engineering initiatives / projects, ensuring timely and successful completion.Collaborate with cross-functional teams to support data-driven decision-making and deliver actionable insights.Partner with architects to devise architectural blueprints for data pipelines, emphasising event-based real-time processing capabilities.Implement data governance frameworks to ensure data quality, integrity, and security throughout the data lifecycle in collaboration with InfoSec and Data GRC stakeholders.Advocate for best practices in data management and governance, encompassing data modelling, ETL efficiency, data quality assurance, observability, and discoverability, as well as security protocols for handling sensitive data and ensuring codebase quality.Lead and mentor a team of data engineers and testers, building their capabilities and continuously improving standards and processes.Implement data governance frameworks to ensure data quality, integrity, and security throughout the data lifecycle.Provide operational support during the evaluation and integration of new tools and technologies to enhance data processing capabilities and improve system and workflow efficiency.Drive innovation in data engineering practices, advocating for best practices, automation, and scalability in data pipelines and infrastructure.Collaborate with IT and DevOps teams to ensure seamless deployment and operation of data systems in production environments.Mentor and provide technical guidance to other data engineers, fostering a culture of knowledge sharing and continuous learning.Conduct regular performance reviews and provide constructive feedback to data engineering team members, supporting their professional growth.What You'll Bring: Tertiary Qualifications in Computer Science, IT, or similar discipline.Relevant certifications in Databricks and related technologies are highly desirable.10 years plus proven experience as a lead or senior data engineer.Experience leading technical data projects to deliver on data architecture vision.Extensive expertise in designing and developing ETL/ELT processes, data pipelines, and data lakehouse platforms (Databricks highly regarded).Extensive experience as a professional data engineer, specialising in developing ETL pipelines for production environments.Strong understanding of data architecture principles, data modelling, and schema design.Proficiency in programming languages such as Python, Java, or Scala, and experience with SQL and NoSQL databases.Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and data technologies in a cloud environment.Excellent problem-solving skills and ability to work independently as well as part of a team.Strong communication skills with the ability to collaborate effectively with stakeholders across different functions.Experience in Agile methodologies and familiarity with DevOps practices is a plus.Demonstrated ability to provide data engineering leadership and mentorship to senior data engineers.At Compare the Market, we offer a dynamic and collaborative work environment where your talents will be recognised and rewarded.
You'll have the opportunity to make a meaningful impact and contribute to our continued success.
As well as ample opportunities for growth and development, we also provide:
Vibrant and social community who have annual celebrations, family fun days and regular events.Enjoy flexible work arrangements, including the option of one day working from home each week or a 9-day fortnight.Enjoy additional leave days - 'ME' leave and 'Volunteer Day' leave.Study support (up to 50% of tuition fees for relevant courses) and professional membership reimbursement.Employee discounts on Car, Home, Travel insurance.Income protection insurance provided to support you in the event of non-work-related illness or injury.Discounts with a broad range of providers, including gym memberships, accommodation, dental care and more.Compare the Market is an Equal Opportunity Employer, and we encourage all applicants to apply.
We acknowledge that we are strengthened by diversity.
Want to know more?
Then apply now!
It's Simples!
Please note that this role is only available to candidates with unlimited Australian working rights, as sponsorship is not available. *Note to Recruiters: We politely ask that you avoid making any approaches or sending any unsolicited resumes to our Recruitment Team or Hiring Leaders across our business. Comparethemarket.com.au is a comparison website.
What does that mean?
Well, pretty much just what the name implies: we provide quotes from a range of participating providers, helping consumers make an informed decision about what is best for them.
The value to you?
Comparison websites allow you to get quotes from a variety of sources in one go, rather than having to visit multiple websites and filling in lots of forms.
In very specific terms, what that means is that comparethemarket.com.au quickly and conveniently allows consumers to search for some products and services according to their individual needs, and then have a range of quotes from participating providers displayed for them in one place.
Source: This is an extract from the company's own website.
Comparethemarket.com.au is a comparison website.
What does that mean?
Well, pretty much just what the name implies: we provide quotes from a range of participating providers, helping consumers make an informed decision about what is best for them.
The value to you?
Comparison websites allow you to get quotes from a variety of sources in one go, rather than having to visit multiple websites and filling in lots of forms.
In very specific terms, what that means is that comparethemarket.com.au quickly and conveniently allows consumers to search for some products and services according to their individual needs, and then have a range of quotes from participating providers displayed for them in one place.
Source: This is an extract from the company's own website.
Don't provide your bank or credit card details when applying for jobs.

To help fast track investigation, please include here any other relevant details that prompted you to report this job ad as fraudulent / misleading / discriminatory.

#J-18808-Ljbffr