Senior Software Development Engineer, Big Data
The Zillow Group’s Valuation & Housing Trends Data Engineering team brings innovative analysis, creativity, and excellence to the U.S. housing market. We create, productionalize, analyze, and share a full collection of housing metrics and analysis for Zillow’s internal and external customers. Our work contextualizes Zillow’s unparalleled living database of all homes and hundreds of millions of customers to provide insight to everyday consumers and open data to hundreds of researchers and policymakers across the country. You’ll work in a fun, collaborative atmosphere with a high-performance team with diverse data engineering and analytical skills.
Your work will be highly impactful. The data we create ends up in the nation’s most reputable publications, used by government institutions in policy-making and official reports, in public-facing tools used by millions of Zillow consumers, and in the work of hundreds of academic researchers.
Small team = big impact. Engineering teams are highly decentralized in order to create the small team speed and autonomy of a start-up environment but are backed by big company resources.About the role
We process hundreds of millions of data points to produce macroeconomic housing market indicators, forecasts, and insights. We publish those so businesses, policymakers, and individuals can make informed decisions about homes. We are also an integral part of housing market intelligence for our internal businesses, and an important input into tactical and strategic business decisions with customers across the company (from marketing, finance, business unit ML teams, to executives).
We are looking for a strong engineer who will be working closely with the data science and forecasting teams, and our customers to help us re-shape our rapidly-evolving data products and the end-to-end data processing and model development framework. You will have great opportunities to learn cutting-edge ecosystems, and cloud technologies, and work with a group of versatile software developers in a collaborative and supportive team environment.
You will:
- Build end-to-end data processing systems to ingest raw data and concrete Housing Trends insights.
- Build and extend our internal forecasting model development framework
- Architect, build and maintain scalable data products and deliver pipelines built for speed, accuracy, and consistency that will scale as our needs and those of our users grow.
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability and performance, etc.
- Design and implement LLM based Chatbot tools to help our customers better understand our data and improve their productivity.
- Be a trusted partner for Economists, Applied Scientists, and Data Analysts.
- Work with multiple teams – including our Data Science, Forecasting, Engineering (think Zestimates), and Property Data teams – to efficiently source Zillow’s proprietary datasets in the creation of industry-leading housing metrics and forecasts.
- Translate requirements to development plans and effectively execute on goals.
This role has been categorized as a Remote position. “Remote” employees do not have a permanent corporate office workplace and, instead, work from a physical location of their choice which must be identified to the Company. Employees may live in any of the 50 US States, with limited exceptions. In certain cases, an employee in a remote-designated job may need to live in a specific region or time zone to support customers or clients as part of their role.In California, Colorado, Connecticut, Nevada, New York and Washington the standard base pay range for this role is $159,400.00 - $254,600.00 Annually. This base pay range is specific to California, Colorado, Connecticut, Nevada, New York and Washington and may not be applicable to other locations.In addition to a competitive base salary this position is also eligible for equity awards based on factors such as experience, performance and location. Actual amounts will vary depending on experience, performance and location.Who you are
- Proficiency in Python, Spark, SQL, and Hive programming for data analytics and ETL pipeline development.
- Experience building systems on AWS cloud platforms is required.
- In-depth knowledge of big data ecosystems and platforms, including Spark, EMR, EKS, Hive, Hadoop, Presto, Kubernetes and Airflow.
- Hands-on experience with Lakehouse technologies such as Databricks, Delta Lake, HUDI, Iceberg is a significant advantage.
- Familiarity with Agile/DevOps software development processes and tools such as GitLab and CI/CD.
- Demonstrated experience in building and orchestrating batch and real-time ETL pipelines, handling vast amounts of data from diverse sources.
- A good understanding of data science concepts, fundamental machine learning algorithms/libraries, and development lifecycle is a must-have.
- Collaboration with data scientists to conduct feature engineering and deploy machine learning models in production environments, and identify growth opportunities
- Strong data analysis skills, enabling thorough investigation of data issues and providing comprehensive responses to customer queries.
- Strong ability in performing root cause analysis to address operational issues and identify opportunities for improvement.
- Good understanding of measuring and ensuring data quality at scale, along with expertise in the necessary tooling for monitoring and optimizing the performance of all data pipelines.
- Familiarity with LLM technologies (OpenAI API etc.) is a nice-to-have.
- 5+ years of experience in Big Data Engineering, Machine Learning Engineering or Data Analytics roles.
Get to know us
Zillow is reimagining real estate to make home a reality for more and more people.
As the most-visited real estate website in the United States, Zillow® and its affiliates help movers find and win their home through digital solutions, first class partners, and easier buying, selling, financing and renting experiences. Millions of people visit Zillow Group sites every month to start their home search, and now they can rely on Zillow to help make it easier to move. The work we do helps people get home and no matter what job you're in, you will play a critical role in making home a reality for more and more people.
Our efforts to streamline the real estate transaction are supported by a deep-rooted culture of innovation, our passion to redefine the employee experience, a fundamental commitment to Equity and Belonging, and world-class benefits. These benefits include comprehensive medical, dental, vision, life, and disability coverages as well as parental leave, family benefits, retirement contributions, and paid time off. We’re also setting the standard for work experiences of the future, where our employees are supported in doing their best work and living a flexible, well-balanced life. But don’t just take our word for it. Read recent reviews on Glassdoor and recent recognition from multiple organizations, including: the 100 Best Companies to Work For, Glassdoor Employees’ Choice Award, Bloomberg Gender-Equality Index, Human Rights Campaign (HRC) Corporate Equity Index, and TIME 100 Most Influential Companies list.
Zillow Group is an equal opportunity employer committed to fostering an inclusive, innovative environment with the best employees. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. If you have a disability or special need that requires accommodation, please contact us at RecruitingAccessibility@zillowgroup.com.
Applicants who receive job offers from Zillow Group will be asked to sign a Proprietary Rights Agreement which includes confidentiality, intellectual property assignment, customer and employee non-solicitation, and non-competition provisions. If you are contacted for a role at Zillow Group and wish to review a copy of the Proprietary Rights Agreement prior to receiving an offer, you may request a copy from your Recruiter.
{{notification.msg}}