Data Engineer Team Lead
Pagaya is a global technology company making life-changing financial products and services available to more people nationwide, as it reshapes the financial services ecosystem. By using machine learning, a vast data network and a sophisticated AI-driven approach, Pagaya provides comprehensive consumer credit and residential real estate solutions for its partners, their customers, and investors. Its proprietary API and capital solutions integrate into its network of partners to deliver seamless user experiences and greater access to the mainstream economy. Pagaya has offices in New York and Tel Aviv. For more information, visit pagaya.com.
About the Role:
We are on the lookout for a dynamic Data Engineering Team Lead who will steer the direction of our budding data pipeline. Constructed using the potent combination of Apache Spark, Iceberg tables, and the Nessie framework, this pipeline lies at the nexus of our data-driven decisions. As a leader, you will not only oversee the technicalities but will also inspire, mentor, and coach engineers. This role requires interfacing closely with other team leads, the group lead, the Engineering director, and other departments, especially Research. In essence, you are the fulcrum balancing our legacy tech stack of DBT, Airflow, and Snowflake with the new innovations we're implementing.
By joining our team, you will be pivotal in the evolution and scaling of our data infrastructure, ensuring its resilience, performance, and utility across departments. If you are driven by challenges, innovation, and leadership, we would be delighted to welcome you aboard.
- Spearhead complex data engineering projects from conception to deployment.
- Partner effectively with product managers to maintain and execute against the product roadmap.
- Drive the development and enhancement of our Data Lakehouse while ensuring its seamless integration with our data warehouse.
- Frame effective monitoring solutions to ensure the uninterrupted functioning of the data infrastructure.
- Contribute across the data team spectrum, tackling performance, scalability, and stability challenges.
- Mentor and oversee a team of data engineers, imparting best practices and fostering professional growth.
- Collaborate with fellow team leads, technical leads, principal engineers, and higher-ups to align the company's technical direction with its strategic goals.
- Take pride in steering a driven team of engineers within a culture of mutual respect, knowledge-sharing, and empowerment.
- Solve challenging data problems and utilize cutting-edge data technologies to gain business advantages.
- Relish the autonomy in tool selection - you pick what's best suited for the job.
- Engage with open-source projects and contemporary frameworks.
- Collaborate with a talented, motivated, and fun team, always eager to innovate and solve challenges.
- Embed your contributions within an environment that prizes correct, maintainable, and elegant code.
- 3+ years of engineering management experience
- 6+ years of experience in a fast-paced software development environment at scale
- Strong track record of delivering high-quality software while leading teams of 3 to 10 engineers
- Hands-on experience in SQL and Python and a comprehensive grasp of data engineering concepts of ETL and data modelling.
- Experience working with cloud environments (AWS preferred), big data technologies (EMR, EC2, S3, Snowflake, Spark-Streaming, Hive, DBT), and a broader Data Engineering ecosystem.
- Proven track record of designing, deploying, and maintaining large-scale data infrastructure.
- Exceptional problem-solving skills, especially in debugging large-scale systems.
- Proficiency in English, both written and spoken.
- Bachelor's degree in Computer Science, Computer Engineering, or related disciplines.
- Demonstrated expertise with Snowflake Data Warehouse and familiarity with other data warehousing technologies.
- Vast experience with Big Data tools: Spark (preferred), Kafka, Elasticsearch
- Familiarity with UNIX, Linux, and Mac OS systems.
Pagaya was founded in 2016 by seasoned research, finance, and technology entrepreneurs, and we are now 500+ strong in New York, Los Angeles, and Tel Aviv.
We move fast and smart, identifying new opportunities and building end-to-end solutions from AI models and unique data sources. Every Pagaya team member is solving new and exciting challenges every day in a culture based on partnership, collaboration, and community.
Join a team of builders who are working every day to enable better outcomes for our partners and their customers.
Our values are at the heart of everything we do. We believe great solutions are built through a great community.
- Advance Inclusion- We create a world where everyone can win, designing systems that better represent people and generate sustainable value for our employees, partners and investors.
- Be Accountable Together- We proudly own our actions and our results, taking initiative to ensure our work gets over the finish line as a team.
- Continuously Learn- We challenge ourselves for the sake of getting better as individuals, as teams, and as an organization to deliver for our partners.
- Debate and Commit- We respectfully and openly debate to strengthen our ideas and build shared conviction - once we decide, we go all in, together.
- Dream Big and Act- We boldly tackle complex problems, pressure-test solutions in real-time, and adapt with speed and energy.
More than just a job
We believe health, happiness, and productivity go hand-in-hand. That's why we're continually looking to enhance the ways we support you with benefits programs and perks that allow every Pagayan to do the best work of their life.