Job Details

Technical Lead, Big Data Workflow Platform

San Francisco, California, United States
About Pinterest: Millions of people across the world come to Pinterest to find new ideas every day. It's where they get inspiration, dream about new possibilities and plan for what matters most. Our mission is to help those people find their inspiration and create a life they love. As a Pinterest employee, you'll be challenged to take on work that upholds this mission and pushes Pinterest forward. You'll grow as a person and leader in your field, all the while helping users make their lives better in the positive corner of the internet. We're looking for a technical lead on our big data workflow / orchestration platform to help us build and lead the next generation of ETL orchestration platform at Pinterest. You'll be working on some of the most exciting big data open source technologies (Airflow, Hadoop, Hive, Spark, Presto, etc.), at the scale of hundreds of petabytes of data to help Pinners discover and do what they love. What you'll do: Improve and customize the internals of open source big data technologies to meet our challenges at scale Build and scale workflow orchestration platforms using the latest open-source technologies to process petabytes-scale datasets Build and scale systems that orchestrate and execute complex workflows in big-data pipelines Provide reliable, performant workflow engines as services to enable ETL, big data analytics and actionable insights Contribute to the team's technical vision and long-term roadmap while helping mentor engineers and grow them in their careers. What we're looking for: Extensive experience with big data workflow orchestration engines for ETL jobs, such as Airflow, Oozie, Azkaban, etc. Experience with large scale distributed framework and one or more open-source Big Data technologies ( Hadoop, YARN, Spark, Hive, Presto, SQL, Kafka, Impala, Parquet, HDFS, etc.) Knowledge of container and orchestration frameworks such as Docker and Kubernetes and proficiency in one or more programming languages (Python, Java, Scala) Minimum of four years of work experience Pluses: AWS experiences, internal knowledge or expertise of open source or related big data technologies #LI-KL1

Send application

Mail this job to me so I can apply later

Apply With CV

You are not logged in. If you have an account, log in to your account. If you do not have an account, why not sign up? It only takes a minute!

latest videos

Upcoming Events