Data Engineer
Overview
Placement Type:
Temporary
Compensation:
£64k per annum pro rata(PAYE)
Start Date:
23/10/2023
As our Data Engineering expert, you will have significant responsibility to drive the building out of cloud based multi-faceted datasets, ensuring their provenance and governance are upheld to a high standard.
- Build fault tolerant, scalable, quality data pipelines using cloud- based tools.
- Innovative solutions to help broader organization take decisive actions fast and efficiently.
- Be a data steward. Develop deep organization wide data understanding.
- Contribute to data engineering frameworks, tools, and processes
- Implement discipline around data operations and implement best practices to use resources in an optimum way.
- Architect data ingestion, data transformation, data consumption, data governance frameworks.
Required Skills :
- 5+ years of experience as a Data Engineer.
- 3+ years demonstrated ability in distributed data technologies e.g., Hadoop, Hive, Presto, Spark etc.
- 2+ years of experience with Cloud based technologies – Databricks, AWS EMR, S3, Azure Blob Storage, Notebooks etc.
- Familiarity and usage of different file formats in batch/streaming processing i.e., Delta/Parquet/Avro/ORC etc.
- Ability to write complex SQLs across platforms.
- Solid hands – on experience in Python/PySpark/Scala and ability to manipulate data using Pandas, NumPy, Koalas etc. and using APIs to transfer data.
- Experience working as an architect to design large scale distributed data platforms.
- Working experience with CI/CD tools i.e., GitHub, Jenkins etc.
- Working experience with Open- source orchestration tools i.e., Apache AirFlow / Azkaban etc.
Nice to have :
- Familiarity with Machine Learning Algorithms.
- Showcase your work if you are an open – source contributor. Passion to contribute to Open-source community is highly valued.
- Experience with Data Governance tools e.g. Collibra
- Experience with Collaboration tools e.g. JIRA, Confluence etc.
Client Description
A multinational cloud-based software company specialising in a series of products designed to drive creative innovation across multimedia. Used by millions around the world for personal and professional use across all industries.
Aquent is dedicated to improving inclusivity & is proudly an equal opportunities employer. We encourage applications from under-represented groups & are committed to providing support to applicants with disabilities. We aim to provide reasonable accommodation for any part of the employment process, to those with a medical condition, disability or neurodivergence.
