Frontend System Design18 min readPremium

Design Google Photos – Frontend System Design

70. Design Google Photos

We’ll walk through the frontend system design for a Google Photos-style library: tens of thousands of photos in a date-grouped grid that scrolls at 60fps, uploads that survive a crash on a train, search, albums, and a People tab built on face clusters you did not compute in the browser. You’ll see why the grid is justified rather than masonry, how a slim layout index plus a virtualizer keeps the DOM small, why decoded RGBA is the scarce resource, and how an IndexedDB job queue makes the network optional.

This is not Pinterest (masonry discovery) and not Instagram (square feed). It is also not Photoshop on the web. You are not editing pixels. You are fetching the right variant of the right photo at the right time, and not decoding the other 49,999.

Problem Statement

Design the frontend for a personal photo library like Google Photos.

Requirements:

  • An infinite, date-grouped photo grid that stays smooth with 50,000+ items
  • Thumbnails that keep their aspect ratio and fill each row
  • A full-screen viewer with next/prev and pinch-zoom
  • Uploads from the file picker or drag-and-drop that work offline and resume
  • Albums, text search, and a People view of face clusters
  • A memory budget that holds on mid-range phones

Describe the client architecture, the grid, the image pipeline, and the upload queue. Face detection and embedding are server-side; the client is a UI over cluster IDs.

Interviewers are checking three things. Can you virtualize a grid whose cells are not the same size? Do you treat image bytes on the wire and image pixels in RAM as different problems? And do you persist uploads before you touch the network?


Clarifying Questions

# Question Interviewer’s Answer
1 Web, native, or both? A web app. Desktop and mobile browsers.
2 How large is a big library? 50,000 photos is the design point. Some users have 200,000.
3 Videos? Yes, but they are photos with a duration. Playback is the native <video> element.
4 Do we need an editor? No. Crop, rotate, and filters are out of scope. This is the library.
5 Offline? Browse recently viewed thumbs. Queue uploads. A full offline library is not required.
6 Who runs face grouping and object labels? The server. The client renders clusters, names, merges, and splits.
7 Sharing and multiplayer? Shared albums with members. Live cursors and collab editing are out of scope.

Question 2 decides virtualization and the slim index. Question 4 keeps you out of a Wasm editor. Question 6 is the trap: do not spend the interview on FaceNet. Question 5 tells you the upload queue is a local database, not an in-memory array.


Requirements

Functional

  • A chronological library grouped by day, with sticky date headers and a pinch or slider that changes thumbnail size.
  • Photos keep their aspect ratio. Rows share one height and fill the viewport width (a justified grid).
  • Open a photo in a viewer. Arrow keys and swipe move to neighbors. Pinch-zoom the detail variant.
  • Upload from <input type="file" multiple>, drag-and-drop, or the camera on a phone. Show each item in the grid immediately.
  • Create albums, add and remove photos, set a cover, share a link with members.
  • Search with typeahead over people, albums, dates, places, and server-side labels.
  • A People tab of face clusters. Name, merge, and split clusters. Opening a cluster is the same grid, filtered.
  • Multi-select, favorite, trash, and restore.

Non-Functional

  • 60fps while scrolling and pinch-zooming the grid. Input to visible movement within one frame.
  • First useful grid in under 1.5 seconds on a mid-range phone on 4G: chrome, date headers, and the first screen of placeholders.
  • Bounded memory: a byte budget for decoded bitmaps, a cap on in-flight fetches and decodes.
  • No lost uploads: a crash or a refresh mid-flight must not drop photos the user already picked.
  • Network politeness: abort thumbs that have scrolled away, honor saveData, and do not upload originals on a metered connection without asking.
  • The grid, the viewer, albums, and People are usable with a keyboard and a screen reader.

“First useful grid” does not mean the first screen of sharp thumbnails. It means the user can scroll a correctly sized layout. Dominant colors or empty frames are allowed. Waiting to measure every image before layout is not.

Napkin Math

Metric Estimate Calculation So what?
Slim index for 50,000 photos ~1.4 MB packed 50,000 × (16 + 8 + 4 + 4) bytes You can keep id, takenAt, and size for the whole library. Not the EXIF blob.
Same index as JSON ~6–8 MB Verbose keys and decimal numbers Prefer a compact page or a binary column. Don’t dump 50k rich objects into React state.
Rows in the library ~6,000 50,000 / ~8 photos per row, plus day headers Layout is O(n) once per width. Scroll lookup is a binary search.
Mounted cells ~80 8 columns × 6 visible rows + 2 overscan rows The React tree is tens of nodes, not tens of thousands.
Decoded thumb, 400×400 640 KB 400 × 400 × 4 bytes 64 of them ≈ 40 MB. Comfortable.
Decoded 12 MP original 48 MB 4000 × 3000 × 4 bytes 64 of them ≈ 3 GB. The tab dies. The grid must never decode originals.
Thumb on the wire ~20–40 KB WebP or AVIF at ~400 px One screen is a couple of megabytes if nothing is cached.
Upload of 40 photos at 4 MB 160 MB Must land in IndexedDB first Quota and navigator.storage.persist() are part of the design.

The original-vs-thumb row is the one interviewers like. A candidate who says “we use <img src={original}> and the browser scales it” has already lost the memory budget. Decoding happens at the source resolution. CSS width does not shrink the bitmap.


Architecture

Google Photos web client architecture: React UI shell, grid engine, decode workers, upload queue, and backend services

The client has four places work can live, and the important decision is which of them owns the library index.

Layer Owns Runs on Talks to
UI Shell Tabs, search box, album chrome, viewer chrome React, main thread Reads derived windows, sends intents
Grid Engine Slim index, justified rows, virtualizer, image LRU Main thread, no React state on scroll Decode workers, CDN, library API
Workers Image decode, SHA-256, EXIF Dedicated workers Transfers ImageBitmaps back
Upload Queue Job rows, file bytes, resume offsets IndexedDB + uploader Upload API, then the library index

Scroll position changes every frame during a fling. If that number lives in React state, every cell re-renders. The virtualizer reads scrollTop from the scroller, computes a window, and mutates a small pool of cell views. React learns about the window at a low frequency, or not at all: the cells can be plain DOM nodes the engine reuses.

“React renders the tabs and the viewer chrome. The library index and the scroll window live outside React, because a fling is a pointer event, not a state update.”

React can still render each visible cell if it is memoized on photoId. The senior boundary is the same one as in the Maps client: the thing that moves every frame does not go through the framework.

Why not masonry, and why not squares?

Layout Good at Wrong for Photos
Uniform squares Trivial getItemLayout, Instagram-simple Crops portraits and panoramas. Users notice.
Masonry (equal columns) Pinterest discovery, variable height Dates do not form rows. A day header cannot span cleanly. Scroll height is annoying to compute.
Justified rows Keeps aspect ratio, fills the width, date groups are rows You must pack rows and store prefix sums. Worth it.

Google Photos on the web is a justified grid. That is the layout you should name, and you should be able to pack a row on a whiteboard.


Data Model

Google Photos client data model: Photo, SlimIndexEntry, DateGroup, Album, FaceCluster, UploadJob, and ImageVariant

Split the library into a slim index you can layout from, and fat records you hydrate on demand. Mixing them is how people end up with 50,000 objects in a Redux store and a 200 ms scroll handler.


type PhotoId = string;
type ClusterId = string;

interface SlimIndexEntry {
  id: PhotoId;
  takenAt: number;       // epoch ms, sort key
  w: number;
  h: number;
  dominant: number;      // 0xRRGGBB
}

interface Photo {
  id: PhotoId;
  takenAt: string;
  width: number;
  height: number;
  durationMs?: number;
  albumIds: string[];
  clusterIds: ClusterId[];
  variants: { thumb: string; detail: string; original: string; poster?: string };
}

interface UploadJob {
  localId: string;
  hash?: string;
  bytesKey: string;      // IndexedDB key for the Blob
  state: "queued" | "hashing" | "uploading" | "processing" | "ready" | "failed";
  offset: number;
  error?: string;
}

The slim index is enough to pack every row and to paint a placeholder. Hydrate a Photo when a cell is visible or the viewer opens. Albums, search hits, and face-cluster members are arrays of ids into the same maps. A cluster itself is an id, optional name, face-crop URL, and count — never embeddings. An upload job becomes a Photo when the server returns a photoId; until then the cell keys on localId and a blob: URL.


The Justified, Virtualized Grid

A date-grouped justified photo strip with a viewport window and overscan, plus the virtualizer range and prefix-sum lookup

Two problems, stacked. First, turn a list of aspects into rows that fill the width. Second, mount only the rows that intersect the viewport.

Packing a row

Walk the slim index in takenAt order, one date group at a time. Accumulate photos until the row would overflow the viewport, then scale that row so it fits exactly.

The row packer is a pure function from one date group’s slim entries, the viewport width, a target row height, and the gap to a list of rows. It keeps a running sum of aspect ratios (width over height). After adding each photo, it computes the row’s width at the target height, including gaps. Once that width reaches the viewport, the row is closed and its height is solved so it fits exactly: available width minus gaps, divided by the aspect sum. The leftover photos at the end of a group become a final row at the target height, unstretched, so a single sunset never balloons to 1,200px wide.

Date groups are packed separately so June 12 and June 11 never share a row. Each group also emits a header row with a fixed height. The full library is a flat Row[]: headers and photo rows interleaved.

Pinch-zoom or the density slider only changes targetH (and therefore how many photos fit in a row). Relayout from the same index. Do not refetch.

Windowing

After packing, build a prefix-sum array of row heights. scrollHeight is the last prefix. Given scrollTop and the viewport height, a binary search returns the first and last visible row. Expand by an overscan of one or two rows so scrolling into a cell does not pop.

The visible range is two binary searches over the prefix sums: the last row starting at or above scrollTop, and the last row starting above scrollTop + viewHeight. Widen both by the overscan and clamp them to the row list. The result is a first and last row index, and only those rows get mounted.

A cell is a reused view: set photoId, set the placeholder color from dominant, and ask the image loader for the thumb variant. When the virtualizer recycles the node onto another id, abort the old request and reset the image.

content-visibility: auto skips paint for off-screen mounted nodes. It does not stop you from creating 50,000 of them, so it is an extra, not a virtualizer.

Scroll anchoring

When the viewport width changes, or the user pinches, every row height changes and scrollTop now points at a different photo. Remember the photoId nearest the top of the viewport, relayout, then set scrollTop so that photo is back at the same offset. Without an anchor, rotating a phone jumps the user three months.

The same trick applies when today’s uploads prepend at the top: shift scrollTop by the added height so the user does not lose their place.

The slim index, not infinite “just fetch more”

A social feed can be a cursor of pages because the user never jumps to 2019. A photo library has a date rail and a scrollbar for the whole archive. An honest scrollbar needs the total height, and packing rows needs every aspect ratio. Load a compact slim index (shard by year if a 200,000-photo user makes even that too big). Fat Photo records stay paginated. Implement the 50,000-photo case fully; mention the year-shard fallback.


Progressive Images

Progressive image pipeline from placeholder to thumbnail to viewer detail, with a byte-budget LRU that evicts ImageBitmaps

A cell is not one URL. It is a ladder of representations, and you climb only as far as the surface needs. The image optimizations guide covers formats and srcset; here the interesting part is which rung the grid is allowed to stand on.

Rung Size When Decoded cost
Dominant color 3 bytes in the slim index Every cell, immediately None
LQIP / blurhash ~20–200 bytes Optional, first screen Tiny
Thumbnail ~400 px, WebP/AVIF Visible + overscan cells ~0.6 MB
Detail ~1600 px Open viewer, prefetch ±1 ~6–10 MB
Original 12 MP JPEG / HEIC Download, or an editor you are not building ~48 MB

Pick the thumb URL from CSS width times devicePixelRatio, not from a single global size. A 128 px cell on a DPR-2 phone wants ~256 px, not the 1600 px detail. Serve variants from a stable CDN shape, /media/:id/w:400.webp, with long Cache-Control and a content hash in the path so a reprocess is a new URL.

Fetch with an AbortController per cell. When the virtualizer recycles the cell, abort. Decode with createImageBitmap in a worker and transfer the bitmap back so the main thread does not spend 20 ms in the JPEG decoder during a fling. Cap concurrent decodes at a small number and order the queue by distance to the viewport center.

Use Cache Storage for thumbs you have already shown. The memory LRU is for decoded bitmaps; the HTTP cache is for compressed bytes. Evicting a bitmap means you decode again from Cache Storage, not that you refetch the CDN. HEIC from recent iPhones will not decode in every browser: transcode on the server, and on Safari show a local blob: URL until the CDN WebP exists.


Offline Upload Queue

Sequence diagram of an offline photo upload: persist to IndexedDB, optimistic grid insert, resumable PUT, processing, and ready swap

The user considers a photo uploaded the moment they picked it. The network is a later optimization. That one sentence decides the queue.

  1. Accept File objects from the picker, drop, or showOpenFilePicker().
  2. Write an UploadJob and the file bytes (or a FileSystemFileHandle) into IndexedDB. Await the commit.
  3. Insert a slim entry at the top of today’s group with a localId and a blob: URL. The grid paints immediately.
  4. In a worker, compute SHA-256 and read EXIF (takenAt, orientation, width, height).
  5. When online, POST /uploads/init with the hash and size. If the server already has that hash, skip the bytes and jump to ready.
  6. PUT chunks with Content-Range (or a tus-style offset). Persist offset after each acknowledged chunk.
  7. On complete, the server transcodes variants. The job becomes a real Photo. Swap the cell from the blob URL to the CDN thumb and revokeObjectURL. Delete the IDB bytes.

Step 2 before step 5 is the part people skip. A File in React state dies with the tab. After a crash, reload reads the queue store and rebuilds the optimistic cells. Ask for persistent storage after the first successful enqueue:

The upload queue owns a quota check before each enqueue. Request persistence once with navigator.storage.persist(), then compare free space from navigator.storage.estimate() (quota minus usage) with the incoming bytes plus a safety margin. If it doesn’t fit, fail the enqueue with a typed quota error before writing anything, and tell the user to free space or wait for uploads to drain. A half-written job is worse than a refused one.

Resume on online, on visibilitychange back to visible, and, where it exists, on a Service Worker sync event. Background Sync is Chromium-only and not a substitute for IDB. Treat it as a wake-up, not as storage.

Honor navigator.connection.saveData and a user-facing “upload on Wi-Fi only” flag. Large original PUTs on a metered radio are how you get one-star reviews, not how you get a faster library. Hashing and EXIF can still run offline so the job is ready to go the moment Wi-Fi appears.

Failures are per job, not per batch. One corrupt HEIC must not freeze the other 39. Cap retries with exponential backoff, surface a retry button, and keep the local thumb visible in a failed state. When the server returns “already exists” for a hash, that is success: point the cell at the existing photoId.


Albums, Search, and Face Clusters

All three reuse the grid. They are different indexes into the same photos, not three new renderers.

Albums

An album is a title, a cover id, a count, and a list of photo ids (or a cursor of ids if a shared album is huge). The album page feeds those ids to the same justified virtualizer. Do not store a second copy of the image records. Shared albums have members and a role; the client only needs enough to render an invite dialog and to hide mutating actions for viewers.

Typeahead is a debounced, abortable GET. 150–200 ms, one in-flight request, AbortController on the next keystroke. Bias toward people and albums the user already has, then server labels (“beach”, “receipt”) and dates. Selecting a suggestion opens either a cluster, an album, a date jump, or a result grid of ids.

The result grid is the same virtualizer with a different index. If the server returns 4,000 ids, you still slim-layout them. Persist recent queries in localStorage.

Face clusters as a UI

This is where candidates wander into machine learning. Stop them. The server already clustered faces. Your job is a People tab.

  • A grid of cards: circular face crop, optional name, photo count.
  • Opening a card requests member ids and renders the library grid filtered to that cluster.
  • Name a cluster: PATCH /faces/clusters/:id { name }.
  • Merge: the user picks two cards, you POST /faces/clusters/merge { a, b }, then refetch the list. One card remains.
  • Split: from a photo, “this is not them” sends POST /faces/clusters/:id/remove { photoId }. The server may attach that face to another cluster or to an unnamed one.

Unnamed clusters are first-class (“Person 3” plus the crop). Hiding them until they are named makes the feature look broken. Do not run a detector on the client. You do not have the archive decoded, you will drain the battery, and you will disagree with the server. Cropping a card with a server-provided bounding box is layout, not ML.


Memory Pressure

Compressed bytes in Cache Storage are cheap. Decoded bitmaps are not. Design the budget around RGBA, then everything else fits.

The decoded budget is a small function of device signals. Use navigator.deviceMemory where it exists (Chromium) and assume 4 GB where it doesn’t (iOS). With Save-Data on, or 2 GB or less, budget about 48 MB of RGBA. Up to 4 GB gets about 96 MB, and anything larger gets about 192 MB.


interface DecodedSlot {
  key: `${PhotoId}:${"thumb" | "detail"}`;
  bitmap: ImageBitmap;
  bytes: number;          // bitmap.width * bitmap.height * 4
  lastUsed: number;
}

On insert, if sum(bytes) > budget, evict the least recently used off-screen slot and call bitmap.close(). iOS does not expose deviceMemory, so start conservative. The viewer holds one detail bitmap and may prefetch ±1. When it closes, those slots rejoin the LRU. The grid never decodes detail on hover. Revoke blob: URLs and delete IDB bytes when a job is ready. There is no standard memory-pressure event; your eviction policy is the handler. Pause the uploader on pagehide.


State and Data Flow

One store, several indexes, a scroller that is not in the store.

  1. Bootstrap. Fetch the slim index (or the current-year shard) and album / cluster summaries. Paint chrome and placeholders.
  2. Scroll. The scroller updates the virtualizer. Visible ids go to the image loader. React is not on this path.
  3. Hydrate. Fat Photo records for the window arrive in a batch. Cells that wanted a caption or an album badge update.
  4. Open. The viewer route is /p/:photoId. Adjacent ids come from the current index (library, album, search, or cluster).
  5. Upload. IDB commit, optimistic slim insert, background PUT, swap to photoId.
  6. Mutate. Favorite, trash, album add, cluster merge: optimistic index edit, then confirm. Revert on 409/404.

The URL holds the surface and the open photo, not the scroll offset. A shareable link to a photo is enough; restoring a library to the exact row of June 2018 is what the date rail is for.


API

Library reads are cacheable pages. Uploads are a resumable protocol. Faces are boring REST on purpose.


// Slim index and fat records
GET /api/library/index?shard=2024          // SlimIndexEntry[] , Cache-Control: private, max-age=60
GET /api/photos?ids=p1,p2,p3               // Photo[] batched, cap the id list
GET /api/library?cursor=:c&limit=200       // fat page if you did not ship a shard

// Albums and search
GET  /api/albums
GET  /api/albums/:id/photos?cursor=:c
GET  /api/search/suggest?q=bea             // people, albums, labels, dates
GET  /api/search?q=beach&cursor=:c         // { ids, next }

// Faces (server clustered)
GET   /api/faces/clusters
GET   /api/faces/clusters/:id/photos?cursor=:c
PATCH /api/faces/clusters/:id              // { name }
POST  /api/faces/clusters/merge            // { a, b } -> { id }
POST  /api/faces/clusters/:id/remove       // { photoId }

// Resumable upload
POST /api/uploads/init                     // { hash, size, contentType, width, height, takenAt }
//   -> { uploadId, offset, photoId? }     // photoId set if hash already stored
PUT  /api/uploads/:id                      // Content-Range: bytes 0-262143/4000000
POST /api/uploads/:id/complete             // -> { photoId, variants }

// Media
GET  /media/:id/w:400.webp                 // Cache-Control: public, max-age=31536000, immutable

Batch the fat-photo GET. A window of 80 cells must not become 80 HTTP requests for metadata. Thumbs are a different origin (the CDN) and multiplex on HTTP/2. Sign private media URLs if libraries are not public; short-lived query tokens beat a cookie on a CDN.


Optimizations

Problem Technique Effect
50,000 DOM nodes Virtualizer + cell pool ~80 mounted cells
Scroll jank from React Window outside React state Fling stays on the scroller
Wrong photo after rotate Anchor to photoId, then relayout Library does not jump
Blank cells while scrolling Dominant color + overscan + Cache Storage Something is always there
Decode jank Worker createImageBitmap, concurrency cap Main thread only paints
3 GB of RGBA Variant by surface, byte-budget LRU, close() Grid stays near 40 MB
Lost uploads IDB before network, chunk resume Crash-safe queue
Slow first paint Inline first-screen slim rows, preload thumbs Layout before the full index

For the first load, the server can inline the first 40 slim entries and the first screen of dominant colors in the HTML. The engine packs one viewport, paints placeholders, and only then fetches the rest of the year shard. That is the photo-library version of Maps inlining center tiles.


Handling Edge Cases

  • EXIF orientation. A phone JPEG may be 4032×3024 with orientation 6. Layout must use the oriented width and height or every portrait becomes a landscape hole. Apply orientation once, in the worker, and store the upright size in the slim entry.
  • Zero or missing dimensions. Some uploads have no EXIF. Use a square fallback for layout, then patch the slim entry when the decoder reports the real size, and relayout that group with scroll anchoring.
  • Corrupt or unsupported files. Mark that job failed and keep the queue moving.
  • Quota exceeded. Stop enqueueing and explain that the device is full. Delete leftover IDB bytes on ready jobs before blaming the user.
  • Timezones. Group by the photo’s local taken date, not the server’s UTC day.
  • Duplicates. Same hash in the same library reuses the existing photoId. Same scene, different hash: two photos.
  • Hidden tabs. Pause prefetch and uploads on document.hidden.

Accessibility and Security

Accessibility

A grid of images is a list of items, not a canvas. Use that.

  • The library is a virtualized list. Each visible photo is a button or link whose alt comes from the date, album, or a server caption.
  • Arrow keys move the selection, Enter opens the viewer, Space toggles select, Escape closes. Date headers are headings.
  • Announce the date under the top of the viewport in aria-live="polite". Do not announce every row of a fling. The viewer is a dialog: “Photo 12 of 128”.
  • Respect prefers-reduced-motion. Expose upload progress as text, not only a ring.

Security and privacy

  • Library media is private. Sign CDN URLs or make them unguessable, and expire them.
  • Face names and crops stay out of analytics. Render titles and queries as text, never as HTML.
  • Upload endpoints need auth, a quota, and a size cap. Hash dedup must not tell you whether another user already uploaded a file.
  • Shared album links are capability URLs. Rotating one invalidates the old token. Viewers get signed read URLs, not the original.

What a Strong Answer Sounds Like

Topic Junior answer Senior answer
Grid A flex wrap of <img> tags Justified rows, prefix sums, ~80 recycled cells
Images One URL, let CSS scale it Variant per surface, worker decode, abort on recycle
Memory “Browsers handle it” Byte-budget LRU of ImageBitmaps, originals never in the grid
Uploads POST the File from React state IDB first, chunk resume, hash dedup, Wi-Fi-only option
Faces Run a model in a worker Cluster IDs, crops, merge/split RPCs, same grid
Search Filter the array in memory Debounced suggest, abort, result index of ids

Conclusion: How to Design Google Photos

Start with the index. A slim record of id, timestamp, and aspect ratio lets you pack a justified, date-grouped grid for the whole library and put an honest scrollbar on it. Virtualize that grid so a fling mounts a window, not a career. Climb an image ladder that stops at the thumbnail until the viewer opens, and treat decoded RGBA as a budgeted cache you close by hand.

Then make the network optional for the one action users cannot stand to lose: upload. Persist the job, paint a local thumb, resume chunks, swap to a CDN variant. Albums, search, and People are other indexes into the same engine. If you can explain why 64 originals are 3 GB and why face clusters are REST, you have shown the interviewer you understand the product, not just an infinite scroll demo.