Kforce has a client in search of a Software Engineer (Data Engineering/Big Data) in San Jose, CA.
The Software Engineer will be responsible for the Data Analytics Platform - Design, implement and operation a Global Big Data infrastructure. They are not doing the processing (analytics). They own the platform, NoSQL, Hadoop, and Kafka. This team manages and maintains the data platforms and pipelines.
* Data Engineer will collect and process raw data at scale for a variety of projects and initiatives
* Design and develop data applications using selected tools and frameworks as required and requested for a variety of teams and projects
* As a Data Engineer you will lead, extract, transform, stage and load data to selected tools and frameworks as required and requested
* Perform tasks such as writing scripts, web scraping, calling APIs, write SQL queries, etc.
* Work closely with the engineering team to integrate your work into our production systems
* Process unstructured data into a form suitable for analysis
* Analyze processed data
* Data Engineer will also support business decisions with ad hoc analysis as needed
* Monitoring data performance and modifying infrastructure as needed
* Define data retention policies
* Automate deployment, monitor pipeline
* Monthly updates to software, ongoing task, OS patches, Applications patches
* Monitor capacity planning
* Pipeline monitoring
* Platform Monitoring
* Linux and Big Data technologies
* Hadoop and Kafka, NoSQL
* SQL Data Bases
* Data engineer who is good in SQL Data, (NoSQL, Relational Data bases)
* Automate the deployment - Nice to have, not a must
* Apache Spark - Nice to have
Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status. - provided by Dice