Data Engineers (Multiple)
RFQ closing date: Wednesday, 06 November 2024
Client: Department of Agriculture, Fisheries and Forestry
Requirements
The Department of Agriculture, Fisheries and Forestry (DAFF) is looking for Data Engineers to join its Enterprise Analytics section to work across several data and analytics platforms. We are seeking candidates with strong experience in developing Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes, and/or the development of data products and complex data visualisations.
The role will be responsible for design, development and unit testing activities across several data movement, data transformation, and data visualisation processes within DAFF. The data movement and transformation processes focus on the preparation of data for use in decision-making processes across the department, utilising modern cloud technology (Azure) to enable operational analytics use cases.
The successful candidate will require experience with the following techniques and technologies:
MS Azure Stack
Data Integration - Data Factory, SQL Server Integration Services and/or Databricks
Data Store - SQL Server and/or Data Lake Storage
Analytics - Azure Databricks, Azure Machine Learning, ArcGIS Enterprise
Development tools - DevOps, Visual Studio, VS Code
Data technology solutions sourcing (Oracle, Ingres, Azure, SQL Server), collecting, ingesting and storing
Data Preparation
Transformation of data into formats tailored to analytics use cases - Parquet and/or Delta
Data Visualisation
Analyse and interpret complex data sets, and to identify trends and patterns.
Design principles to create appropriate visualisations for target audience.
Visualisation tools - Power BI
Initial contract period is 12 months.
Candidates must hold a valid Baseline security clearance or be willing and eligible to obtain one. Please provide your CSID on your resumé/CV (if known).
Estimated start date: Wednesday, 20 November 2024
Initial contract duration: 12 Months
Extension term: 12 months
Number of extensions: 2
Location of work: ACT, Offsite
Working arrangements: Most of the section is located at Agriculture House in the Canberra CBD. Our standard expectation is for staff to attend the office three (3) working days each week. The section is supportive of flexible arrangements and consideration will be made for individual and collaboration needs, including attendance at important Agile ceremonies and other team/section events.
Security clearance: Must have Baseline
Criteria
Essential criteria
Demonstrated experience developing ETL/ELT processes for complex and/or large data movement, transformation and/or visualisations, particularly in a cloud environment. (50%)
Experience preparing data optimised for query performance in cloud computed engines (e.g., Distributed computing engines (Spark), Graph Databases, Azure SQL) (25%)
Experience working with Engineering, Storage and Analytics services in cloud infrastructure. (25%)
Desirable criteria
Experience working with Azure Data Factory and Databricks.
Experience/Knowledge of working with Data Lake and Lakehouses.
#J-18808-Ljbffr