Job Details

Senior Big Data (Hadoop/Spark) Engineer

Advertiser
M&T Bank
Location
New York, United States
Rate
-

At M&T Tech, we're a team of makers, doers, and builders, working to create the most advanced technology solutions in banking. We're not your stereotypical suit and tie bankers: We're an innovative team of leading tech experts, pushing boundaries, and taking risks. We're building an agile team of the most skilled and creative, working to solve complex problems, architecting solutions, writing high-performance software, and charting our new path, all to make the lives of our customers, and the communities that we serve better. Join us and be part of something new as we build tomorrow's bank, today.

The Enterprise Architecture, Data, and Engineering Group is responsible with enabling our technology partners across the Bank with a suite of best-in-class Platforms, Tools, and Products; allowing them to deliver rich customer solutions all while decreasing their time to market. In addition, the Group creates significant value by providing an advisory and governance function on these capabilities, so technology delivery teams can focus on the "What" not the "How".

Overview:

This position is an opportunity to build the data framework with HDFS, Spark and Kafka technology for batch and streaming analytics. Data processes in BDM, if appropriate, include ingestion, standardization, metadata management, business rule curation, data enhancement, and statistical computation against data sources that include relational, XML, JSON, streaming, REST API, and unstructured data. BDM has provided a metadata injection layer for data ingestion across six SQL Server sources that will be leveraged for additional source ingestion.

Scope of Responsibilities:

Understand, prepare, process and analyze data to drive operational, analytical and strategic business decisions. You will create, modify and maintain both Sqoop and BDM code and complex SQL for BI/DW data flows. You will program in Spark and Python where BDM Spark co-generation is not adequate. Build end to end data flows from sources to fully curated and enhanced data sets. This can include the effort to locate and analyze source data, create data flows to extract, profile, and store ingested data, define and build data cleansing and imputation, map to a common data model, transform to satisfy business rules and statistical computations, and validate data content.

Tech Stack:

  • Hadoop
  • Informatica tools: PowerCenter, MDM, Big Data Manager
  • Python
  • Java
  • Scala
  • Spark
  • Kafka
  • Hive
  • HBase
  • Databases: SQL, NoSQL, Postgres, Cassandra
  • AWS cloud services: EC2, EMR, RDS, Redshift
  • Stream-processing systems: Storm, Spark-Streaming

The successful candidate will have:

  • Minimum of an associate degree and 8 years' systems analysis/application development experience, or in lieu of a degree, a combined minimum of 10 years' higher education and/or work experience
  • Experience with Agile Methodology
  • An ability to build out data products & product enhancements from idea through to launch
  • Strong collaboration with technology partners and customers on feature requirements and prioritization
  • A team player mindset with an ability to thrive and effectively communicate in a fast-paced, constantly evolving environment

About M&T:

M&T Bank is a Top 20 US bank holding company and one of the best performing and financial stable regional banks in the country, we offer our technology employees a wide range of performance-based career development opportunities. We have a strong commitment to our customers and the communities we serve, and we continue to grow with a focus on the future. So, when looking to advance your career, look to M&T. Grow with us.

#LI-KB1



Location

Cheektowaga, New York, United States of America

Send application

Mail this job to me so I can apply later

Apply With CV

You are not logged in. If you have an account, log in to your account. If you do not have an account, why not sign up? It only takes a minute!

latest videos

Upcoming Events