Big Data Engineer

Our client, one of the largest telecommunication companies in Australia, is requiring the skills of a Big Data Engineer. You will primarily be responsible for leading, architecture, design and implementation of data pipelines on Networks Big Data platforms (on premise and on cloud). This is a migration; moving from On-Prem to GCP Cloud.
Essential Requirements: Understand business requirement and develop solution designFine-tuning of new and existing data pipelinesSchedule and maintain data pipelinesDrive optimization, testing and tooling to improve data quality & data governanceAssemble large, complex data sets that meet functional / non-functional business requirementsIdentify, design, and implement internal process improvements: automating manual processes and optimizing data deliveryArchitect, design & build robust and scalable data infrastructure (both batch processing and real-time) to support needs from internal and external usersTroubleshoot Big Data platforms and software, including performance-tunes of BI applicationsBudgeting, tracking and forecasting capital & operational expenditureInterface with product vendors to keep abreast of new technologies, pricing and customer applicability. Participate in vendor evaluationsManage resource capacity and staffingIdentify capability for the team and provide guidance and training.Technical Stack: GCP (Big Query, Dataflow and ETL orchestration)HadoopExtensive ETL experience with an understanding of Big Data ArchitectureAirflowKafka migration (Dataflow to process data in real time).Flexible Work from Home Arrangements To be considered for the role click the 'apply' button or for more information about this and other opportunities please contact Jennifer Bohm on 02 9464 5518 or email: ****** and quote the above job reference number.
Paxus values diversity and welcomes applications from Indigenous Australians, people from diverse cultural and linguistic backgrounds and people living with a disability. If you require an adjustment to the recruitment process, including the application form in an alternate format, please contact me on the above contact details.

#J-18808-Ljbffr