VIOLA

CAREERS

Looking for Your Next Dream Job?

Join Viola's Talent Network and get access to opportunities in Israel's top tech companies

Data Infrastructure Engineer

4M Analytics

4M Analytics

Other Engineering
Tel Aviv-Yafo, Israel
Posted on Monday, January 1, 2024

Software Engineer - Data Core

  • R&D
  • Tel Aviv, Israel
  • Senior
  • Full-time
  • ID: 0037

Description

Who We Are

The 4M story is likely one you haven’t heard before: we are on a mission to create a global map of the subsurface and above surface infrastructure utilities, to do what Google Maps did for the transportation world. Achieving this goal requires amazing products and technology, which both come from having strong teams built of great people who love to collaborate, innovate, and solve complex challenges.

We seek an experienced Software Engineer with a strong background to become an integral member of our Data-Core team, tasked with the mission of processing, digitizing, and analyzing hundreds of millions of data sources. Your role will be pivotal in creating a unified, up-to-date, and accurate utilities map, services, and tools for accelerating our mapping operations. Your contributions will directly impact our core product's success.

Your Mission

  • Collaborate with cross-functional teams to design, build, and maintain data processing pipelines, ensuring efficient data integration, transformation, and loading.
  • Implement geospatial data processing techniques and contribute to the visualization of data on our unified map, enhancing the product's geospatial features.
  • Drive the scalability and performance optimization of data systems, addressing infrastructure challenges as data volume and complexity grow.
  • Develop and optimize complex Python-based code for data cleansing, validation, and standardization, enhancing the quality and accuracy of datasets.
  • Create and manage data infrastructure components, including ETL workflows, data warehouses, and databases, supporting seamless data flow and accessibility.
  • Collaborate in designing and implementing data architecture, ensuring effective data storage, retrieval, and security.
  • Design and implement CI/CD processes for data processing, model training, releasing, and monitoring, ensuring robustness and consistency.

Requirements

  • 7+ years of proven experience as a backend/software engineer.
  • 2+ years of proven experience as a Data Infrastructure Engineer or in a similar role in managing and processing large-scale datasets.
  • A proven experience in building scalable online services.
  • Experience with frameworks like Kafka, DataLake, Airflow, Docker, and K8S to build data processing and exploration pipelines along with ML infrastructure to power our intelligence.
  • Experience in AWS/Google cloud environments, writing in Python/Java/Scala/Go.
  • Experience in deploying a diverse range of cloud-based technologies to support mission-critical projects, including expertise in writing, testing, and deploying code within a Kubernetes environment.
  • Experience working with both SQL and NoSQL databases such as Postgres, MySQL, Redis, or DynamoDB.

Diverse Perspectives

We know that innovation thrives on teams where diverse points of view come together to solve hard problems. As such, we explicitly seek people that bring diverse life experiences, diverse educational backgrounds, diverse cultures, and diverse work experiences. Please be prepared to share with us how your perspective will bring something unique and valuable to our team.