Overview
The Data Engineer is responsible for designing, delivering, and supporting robust data solutions and services to meet the Department’s strategic and operational goals. This role also contributes to the evolution of data architecture and management of on-premise data platforms and tools.
Key Responsibilities
- Collaborate with business users, data scientists, analysts, development teams and other stakeholders to design and implement data products and services, leveraging a modern data lakehouse architecture.
- Design, build and maintain data pipelines, integrating and managing large volumes of data from various data sources.
- Maintain and monitor the on-premise data architecture.
- Optimize data flows, database queries and conduct performance tuning.
- Keep abreast of changes in data management technologies, assessing emerging technologies for applicability within the work of the Division, prototyping potential approaches and presenting the results.
- Contribute to IT project and domain teams, Section activities, and process improvement initiatives.
Required Experience
- Minimum of five years of relevant professional experience working with complex data integration platforms supporting data analysis and data services.
- In-depth experience in data engineering and related platforms and tools such as Spark, Trino, Airflow, Iceberg, Jupyter, dbt core, Kafka, S3, elastic.
- Experience with managing on-premise data architectures is an asset.
Qualifications
Bachelor's Degree - University degree in computer science, computer engineering, information technology or a related field.
