Location: Alpharetta, GA
Description: Our client is currently seeking a BIG DATA ENGINEER - IV in Alpharetta, GA/Dallas, TX/Colorado Spring, CO
Software Engineer - Data Streaming Architect
Targeted Years of Experience: 7-10 years
Create and manage a company's Big Data infrastructure and tools.
Get results from vast amounts of data quickly.
Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.).
Work with other staff to integrate innovations and algorithms into production systems.
Process unstructured data into a form suitable for analysis.
MUST HAVE SKILLS (Most Important):
• Candidates must possess 7+ years of overall software development experience.
• Must have 7+ years of relevant work experience in data ingestion on streaming / real-time platforms.
• Must have 7+ years of recent experience as a developer using Java/C++/Python
• Must have 7+ years of recent experience working with data stream processing technologies like IBM Streams, Apache Flink, or Spark Streaming
• Must have 7+ years experience operationalizing AI/ML pipelines in a real-time environment
• Must have 7+ years experience building input streaming data pipelines for AI/ML use cases
• Must have 7+ years of recent experience working with streaming data stores like Cassandra or Redis
• Must have experience of handling Large volume of data in terabytes as well low volume but high frequency data
• Must have 7+ years of recent experience working with pub/sub technologies like Kafka and Rabbit MQ
• Must have 7+ years experience on Unix based OS
• Experience in implementing multiple connectors (IBM MQ, Jdbc,Apache Pulsar, S3 etc)
• Exposure to NoSQL-based, SQL-like technologies (e. g., Hive, Pig, Spark SQL/Shark, Impala, BigQuery, Cassandra).
• Exposure to Google Cloud Platform (GCP) and integrating it with real-time data streaming applications
• Ability to work independently, and multi-task under short deadlines, based upon general direction
• Telecom domain knowledge is preferred
• Able to mentor, guide and lead a team of streaming data engineers
• Demonstrated effective verbal and written communication skills.
• Good Knowledge in Hadoop platform including Big Data tools
• Some experience with data warehousing and data lakes
• Experience in IBM streams preferred
• Primary responsibility is to design, develop, and implement data service architecture that ingests real time and batch data using IBM streams
• Another key responsibility will be to evaluate Apache Flink along with other new data streaming technologies to determine their viability as replacements for IBM Streams
• Develop IBM Streams application to implement new use cases from business requirements
• Integration of new sources into IBM Streaming platform
• Developing streams application to deploy model in streams platform
• Doing proof of concepts to expand current platform capabilities
• Closely collaborate with cross-functional teams
• The successful candidate will have an understanding of coding and a strong sense of architecture, deployment and operational skill.
• Bachelors degree in computer science or related field - required
• Masters degree in computer science or related field - preferred
Shift: Standard 9-5. On-call rotation required (which will involve some weekend work at times)
Multiple Locations are available but preferred location is Alpharetta, GA.
This job and many more are available through The Judge Group. Find us on the web at