Job Details

Big Data Engineer

Texas City, Texas, United States

The Big Data Engineer - is accountable for leading a technical team of developers on the Master Data Management (MDM) team. This is both a hands-on devleopment and a tech leadership role charged with building a data platform and services to provide a single-source of truth for policy and quotes data. Engage with strategic partners to optimize the data quality function and identify strategic capabilities to keep up with data and insurance industry trends.

This role is responsible for driving multiple complex tracks of work to deliver Big Data solutions enabling advanced data science and analytics. This includes working with the team on new Big Data systems for analyzing data; the coding & development of advanced analytics solutions to make/optimize business decisions and processes; integrating new tools to improve descriptive, predictive, and prescriptive analytics; and discovery of new technical challenges that can be solved with existing and emerging Big Data hardware and software solutions.

This role contributes to the structured and unstructured Big Data/Data Science tools of Allstate from traditional to emerging analytics technologies and methods. The role is responsible for assisting in the selection and development of other team members.

Key Responsibilities

Executes complex functional work tracks for the team.
Partners with ATSV teams on Big Data efforts.
Partners closely with team members on Big Data solutions for our data science community and analytic users.
Leverages and uses Big Data best practices/lessons learned to develop technical solutions used for descriptive analytics, ETL, predictive modeling, and prescriptive Real Time decisions analytics
Influence within the team on the effectiveness of Big Data systems to solve their business problems.
Participates in the development of complex technical solutions using Big Data techniques in data & analytics processes.
Supports Innovation; regularly provides new ideas to help people, process, and technology that interact with analytic ecosystem.
Participates in the development of complex prototypes and department applications that integrate Big Data and advanced analytics to make business decisions.
Uses new areas of Big Data technologies, (ingestion, processing, distribution) and research delivery methods that can solve business problems.
Understands the Big Data related problems and requirements to identify the correct technical approach.
Works with key team members to ensure efforts within owned tracks of work will meet their needs.
Drives multiple tracks of work within the research group.
Identifies and develops Big Data sources & techniques to solve business problems.
Co-mingles data sources to lead work on data and problems across departments to drive improved business & technical results through designing, building, and partnering to implement models.
Manages various Big Data analytic tool development projects with midsize teams.
Executes on Big Data requests to improve the accuracy, quality, completeness, speed of data, and decisions made from Big Data analysis.
Uses, learns, teaches, and supports a wide variety of Big Data and Data Science tools to achieve results (ie Hadoop, SQL, no-SQL [MongoDB] and others).
Uses, learns, teaches, and supports a wide variety of programming languages on Big Data and Data Science work (ie Java, Python, and Perl).
Trains more junior engineers.

Job Qualifications

5-10 years of technologies experience (Tech Lead experience a plus)
Bachelor in Computer Science or related field, or equivalent experience
3-5 years experience with Spark, Scala, Java (Python is a plus)
3-5 years experience working with Hadoop, SQL, no-SQL platforms (MongoDB a plus)
3-5 years experience with various data types (AVRO, JSON, PARQUET a plus)
Experience with development, management, and manipulation of large, complex datasets
Demonstrated knowledge of data management competencies and implementation
Ability to operate and learn in a rapidly changing, high-visibility environment

Send application

Mail this job to me so I can apply later

Apply With CV

You are not logged in. If you have an account, log in to your account. If you do not have an account, why not sign up? It only takes a minute!

latest videos

Upcoming Events