Job Details

Big Data Engineer (Azure)

MBN Solutions
Glasgow, Lanarkshire, United Kingdom

MBN are partnering with a global business founded and headquartered in Glasgow to help them continue their evolution within Data Engineering. After the creation and development of a brand new Azure big data platform, the business are looking to expand by bringing in a new member of the team to work on a variety of cutting edge big data projects.

Key Responsibilities:

Design and implement pipelines to ingest data into and consume data from the data lake

Source and transform data from historical and streaming data sources

Implement automated monitoring and alerting on pipelines to ensure consistent processing of data

Design systems to allow real time analytics on high velocity data

Proactively ensure optimum performance by revisiting design choices to check they are still fit for purpose

Collaborate with Data Scientists and BI Developers to ensure the wider team's objectives are being met by data lake design

Support wider team in consuming data from the data lake

Be able to provide technical leadership in the use and design of big data technologies

Key Skills & Experience Required:

  • Experience ingesting and streaming data in to a cloud environment
  • Experience creating robust well engineered pipelines (ideally using Python or PySpark)
  • Experience working in a big data environment or cloud platform (Azure being preferential but not essential)
  • Working knowledge of the Azure Stack is preferred using tools such as (CosmosDB, Data Factory, Logic Apps, Event Hub / IOT Hub, DataBricks) Highly advantageous

..... click apply for full job details

Send application

Mail this job to me so I can apply later

Apply With CV

You are not logged in. If you have an account, log in to your account. If you do not have an account, why not sign up? It only takes a minute!

latest videos

Upcoming Events