Job Details

Senior Big Data Engineer

Advertiser
Allstate
Location
Texas City, Texas, United States
Rate
-

The Big Data Engineer - Senior Consultant II is accountable for leading a technical team of developers on the Enterprise Data Quality team to deliver innovative data quality solutions that generate business value, specifically focused on Hadoop-based, non-relational data. This role requires in-depth data experience to translate unique business requirements into technical data quality specifications that ensure data is fit-for-purpose. Engage with strategic partners to optimize the data quality function and identify strategic capabilities to keep up with data and insurance industry trends.

Job Description

Key Responsibilities

Serve as a subject matter expert and an advocate of the data quality strategic vision and enterprise data strategy
Design and drive the implementation of end-to-end data quality solutions for multiple data ecosystems that leverage non-relational data
Consistently contribute to process improvements and help lead the team to deliver working solutions and data quality standards
Participate and review documentation for self-service data quality practices, the process framework, and internal guides
Collaborate with various business and technical teams to gather requirements around data quality rules and propose the optimization of these rules if applicable, then design and develop these rules in Informatica Data Quality, Informatica Data Engineering Quality (DEQ), or other home-grown technologies
Perform thorough data profiling with multiple usage patterns, root cause analysis and data cleansing and develop scorecards utilizing Informatica, Excel and other data quality tools
Configure and manage connection process
Develop "matching" plans, help determine best matching algorithm, configure identity matching and analyze duplicates
Build workflows scorecards in Informatica Data Engineering Quality (DEQ) to support data remediation
Run data quality jobs (address standardization and validation, email cleanups, name cleanup, parsing, etc.) utilizing DEQ and other ETL tools
Serve as the primary resource to team members and data stewards for training, problem resolution, data profiling etc.
Analyze and provide data metrics to target personas to provide insight into data quality and help prioritize remediation efforts
Identify and resolve issues, bugs, and impediments

Job Qualifications

Education and Experience

Bachelor's Degree in Math, Statistics, Finance, Business, Economics, IT or related field, or equivalent experience
5-7+ years of data and/or analytics experience

Required Skills

Experience with various data types (ie relational, unstructured, semi-structured, hierarchical, linked graph data, streaming, biometrics)
Experience with development, management, and manipulation of large, complex datasets
Experience with database & ETL technologies
Demonstrated knowledge of data management competencies and implementation
Proficient at writing SQL queries and verifying the results
Familiar with organizational change management strategies
Strong communication skills, both written and verbal
Excellent time-management and organization skills
Ability to operate in a rapidly changing, high-visibility environment
Desire to learn and ability to learn quickly
Advanced analytical and problem-solving skills
Strong decision-making skills
Ability to draw meaningful conclusions and recommendations

Send application

Mail this job to me so I can apply later

Apply With CV

You are not logged in. If you have an account, log in to your account. If you do not have an account, why not sign up? It only takes a minute!

latest videos

Upcoming Events