Design Zillow

You're asked to design a real estate marketplace like Zillow or Redfin. Users search for homes on a map with filters for price, bedrooms, and square footage; every property has a detail page with photos, history, and an estimated value; and the underlying listing data arrives continuously from hundreds of independent sources that disagree with each other.

The search path is familiar. The difficult work happens upstream, where three brokerages may describe the same house with different addresses and prices. Let's focus on normalizing those feeds and resolving them to one canonical property.

Clarify the requirements

• Where does listing data come from? Assume hundreds of regional MLS feeds and direct broker submissions, each with its own schema and update cadence.
• How fresh does a listing need to be? Minutes matters, because a listing that stays visible after it sells is the complaint users report most.
• What does search have to support? A map viewport plus filters, sorted by relevance or price. Geographic search with attribute filters, same shape as Design Yelp.
• Is the value estimate in scope? Confirm it. An estimate for every home in the country is a large recurring batch job, so include it only if the prompt calls for it.
• Do we handle photos? They dominate storage. Put them in object storage behind a CDN, then keep the rest of the discussion on listing data.

Assume: many heterogeneous feeds, minute-level freshness, map search with filters, and value estimates for all properties.

Back-of-envelope numbers

• Properties: $150\text{M homes}$ in the database, of which roughly $1.5\text{M}$ are actively listed at any time
• Feed updates: $10\text{M updates/day} \approx 120/\text{sec}$, arriving in bursty per-feed batches rather than smoothly
• Searches: $50\text{M/day} \approx 600/\text{sec}$, peaking several times higher on weekend evenings
• Photos: $1.5\text{M listings} \times 30 \text{ photos} \times 2\text{ MB} \approx 90\text{ TB}$
• Valuation: recomputing $150\text{M}$ estimates on a daily cycle is $\approx 1{,}700/\text{sec}$ of sustained model throughput

The listing count is manageable. Spend your time on reconciling contradictory inputs and recomputing 150 million estimates on a schedule.

High-level architecture

Components

1. Feed ingestion. One adapter per source, handling that feed's format, schedule, and quirks.
2. Raw feed store. Every record exactly as received, kept so the pipeline can be re-run after a logic change.
3. Normalize and resolve. Map each source's schema onto a common one, then decide which existing property a record refers to.
4. Property store. The canonical record per physical home, with its listing history.
5. Search index. Denormalized documents optimized for map-plus-filter queries.
6. Valuation job. Recomputes estimated values on a schedule.
7. Search and detail API. Serves the two user-facing query shapes.
8. Cache. Property detail pages, which are read far more often than they change.

Deep dive 1: normalizing hundreds of feeds

Every MLS has its own schema, its own field names, its own enumerations, and its own idea of what "pending" means. Writing one parser that handles all of them produces a file nobody can safely change.

The structure that holds up is an adapter per feed, a common schema in the middle. Each adapter's only job is to turn its source's format into the canonical representation, and everything downstream sees one schema. Adding a new MLS is then a new adapter rather than an edit to shared logic, and a feed that changes its format breaks one adapter loudly instead of corrupting records silently.

Three things belong in the pipeline around that:

• Keep the raw records. Store what arrived, unmodified, before any transformation. When you discover in three months that one feed's square-footage field was being parsed wrong, you re-run from raw rather than asking the source to resend a quarter of history.
• Validate at the boundary. Schema checks, plausibility checks such as a house not being eleven square feet or a price not being negative, and volume checks. A feed that normally sends eight thousand updates and today sent forty is broken, and detecting that before ingesting is much cheaper than detecting it after.
• Quarantine rather than fail. Records that fail validation go to a dead-letter path with the reason. One bad record should not stop a feed, and one bad feed should not stop the pipeline.

Deep dive 2: entity resolution

The same house may arrive as "123 N Main St Apt 4B", "123 North Main Street #4B", and "123 Main St Unit 4B" from three sources. Resolve them to one property so search does not show three listings for the same home.

Resolve entities in stages, starting with the cheapest checks:

• Address normalization. Run every address through a standardization service that resolves abbreviations, directionals, and unit formats into a canonical form. This alone resolves the large majority of duplicates and costs almost nothing.
• Geocode and match on coordinates. Two records within a few meters of each other are strong candidates for the same property, which catches cases where the address strings differ in ways normalization missed.
• Match on stable identifiers. The county parcel number is authoritative where it's present, and it should override the address-based match rather than merely voting alongside it.
• Score the remainder. For records that still haven't resolved, compare bedroom count, square footage, and lot size, and merge above a confidence threshold. Below it, keep them separate and flag for review, because a wrong merge is harder to detect and undo than a duplicate.

Once resolved, the property store holds one canonical record per physical home with the source records attached, so a merge can be reversed and a field's provenance is answerable. When two sources disagree on the price, the resolution is a source-priority rule where the listing brokerage's own feed beats a syndicated copy, rather than last-write-wins, which would make the displayed price flap between values as feeds arrive in different orders.

Deep dive 3: recomputing valuations at scale

An estimated value for every home in the country is a batch job with a hundred and fifty million rows, and treating it as one monolithic run is what makes it unmanageable.

Partition by geography. Home values are local, since the features that matter are recent nearby sales, so partitioning by county or metro area keeps each unit of work self-contained and makes the whole job embarrassingly parallel. It also means a failure affects one region rather than the country.

Recompute on change, not on a fixed schedule for everything. A full nightly recomputation of every property spends most of its compute on homes whose inputs didn't change. Recomputing a region when a nearby sale closes, and sweeping the rest on a slower cycle, gets most of the freshness for a fraction of the cost.

Write to a new partition and swap. Overwriting estimates in place means a failed run leaves the country half-updated with no clean state to serve. Writing the new values to a separate partition and atomically switching once the run validates gives you a clean rollback and an all-or-nothing update, which is the standard shape for any recurring derived-data job.

Validate the output before publishing. Compare the distribution of new estimates against the previous run and halt if it shifted implausibly. A model or feature bug that moves every estimate in a metro by fifteen percent is highly visible to users and to the press, and a distribution check catches it before publication.

Deep dive 4: keeping search fast and current

Search reads a denormalized index rather than the property store, for the same reason it does in any read-heavy system: the query shape is completely different from the write shape.

Each indexed document carries what a result card needs: location, price, beds, baths, square footage, status, and a thumbnail reference. Rendering a page of results requires no joins. The map query is a bounding box plus filters, which is the geographic search problem covered in Design Yelp.

The freshness requirement drives the update path. Rebuilding the index in a nightly batch would leave sold homes visible all day, which is the most-reported complaint about products in this category. Instead, changes flow from the property store to the index continuously, with the batch rebuild kept as a periodic correctness backstop rather than the primary mechanism.

Status changes deserve their own priority. A price change can propagate in minutes without anyone noticing, but a home going from active to sold should leave the index as fast as possible. Treating status transitions as high-priority updates and everything else as normal-priority is a small distinction that maps directly onto what users complain about.

Common pitfalls

• One parser for all feeds. It becomes unmaintainable, and one source's format change corrupts records from every source.
• No entity resolution. The same home appears three times in one result set.
• Last-write-wins on conflicting prices. The displayed price flaps as feeds arrive in different orders.
• Recomputing every valuation nightly in place. Most of the compute is wasted, and a failed run leaves the data half-updated.
• Rebuilding the search index in a nightly batch. Sold homes stay visible for a day, which is the complaint users actually file.

Leveling signals

Related lessons

Design YelpMedium

Search local businesses by proximity, with filters and aggregate ratings.

Design a Web CrawlerHard

Crawl and download a large set of web pages while respecting per-site rate limits.