Design Pastebin

Design a service where anyone can paste a block of text (a stack trace, a config file, a code snippet), get back a short link such as paste.io/k3Fq9xA2, and share it. Readers open the link and see the text, optionally with syntax highlighting, plus a view counter.

Authors can choose an expiry (10 minutes, a day, a month or never) and can mark a paste as burn after reading, so it disappears after the first view. Pastes range from a few bytes to several megabytes, and a paste linked from a popular forum thread can collect tens of thousands of views in an hour.

In 45 minutes, settle the requirements, size the system, define the API and storage layout, draw the architecture, then go deep on where the text lives, how expiry actually deletes data, and how you count views without hurting reads.

Sketch on the HLD board, write your answer in the Design doc, and size it with the Estimator. Start the mock interview from the timer for a 45-minute run.

Design Path · Core Building Blocks · problem 1 of 18

Ask before you design. Try answering each yourself, then compare.

1. What is the maximum paste size?
   Typical answer

   10 MB hard limit; the median paste is around 5 KB and the mean about 10 KB.

2. Do users need accounts?
   Typical answer

   Anonymous pasting is the main flow. Signed-in users can list and delete their own pastes, but auth itself is out of scope.

3. Can a paste be edited after it is created?
   Typical answer

   No. Pastes are immutable; an edit creates a new paste. That makes caching much easier.

4. How accurate must view counts be?
   Typical answer

   Approximate and a minute or so behind is fine, except for burn-after-read, which must allow exactly one view.

5. What happens when a paste expires?
   Typical answer

   It must stop being readable at the expiry time. Physically deleting the bytes can lag by hours.

6. Should links be hard to guess?
   Typical answer

   Yes. Unlisted pastes often hold secrets, so keys must not be sequential or enumerable.

7. What traffic should we plan for?
   Typical answer

   About 1 million new pastes a day and roughly 10 reads per paste.

Requirements

Functional

• Create a paste with text, optional title, syntax language, expiry and burn-after-read flag; return a short URL.
• Read a paste by key, rendered or raw.
• Show an approximate view count per paste.
• Expired and burned pastes are no longer readable.

Non-functional

• Read p99 under 100 ms for pastes under 100 KB; 99.9% availability.
• Writes are durable once acknowledged (no lost pastes).
• Keys are 8 random base62 characters and cannot be enumerated.
• Expired pastes become unreadable within 1 second of their expiry time.

Out of scope

• User sign-up, login and billing.
• Full-text search across public pastes.
• Malware and secret scanning (good follow-up).

Back-of-the-envelope estimates

• Writes: 1 M pastes a day is about 12 per second, roughly 35 per second at peak. Tiny.
• Reads: 10 reads per paste gives 10 M reads a day, about 115 per second, 350 at peak.
• Storage: 1 M × 10 KB = 10 GB of text per day, about 3.6 TB a year and 18 TB over five years before expiry removes anything. Object storage handles this cheaply with its own replication.
• Metadata: about 200 bytes per paste, so 1.8 B rows over five years is only ~360 GB.
• Bandwidth: 350 reads/s × 10 KB ≈ 3.5 MB/s at peak; a single viral 2 MB paste matters more than the average.
• Cache: the hottest 20% of a day's pastes is ~2 GB, which fits in memory on one node.

The numbers are small, so this is not a throughput problem. The design is driven by the size spread of pastes (bytes to megabytes) and by correct expiry, not by QPS.

API

POST   /v1/pastes   {content, title?, language?, expiresIn?, burnAfterRead?}
                    -> 201 {key, url, expiresAt}
GET    /v1/pastes/{key}        -> 200 {content, language, createdAt, views}
GET    /raw/{key}              -> 200 text/plain  (cacheable, served via CDN)
DELETE /v1/pastes/{key}        -> 204  (owner or holder of delete token)

Creation returns a separate secret delete token for anonymous authors, so deleting a paste never depends on knowing only its public key.

Data model

paste_meta(key PK, blob_ref, size, language, owner_id?, created_at, expires_at, burn_after_read, delete_token_hash) lives in a key-value store partitioned by key. Every lookup is by exact key, so a single-key store is enough and scales by adding partitions. A secondary index on owner_id serves "my pastes".

The text itself goes in an object store under pastes/{key}. Pastes under ~4 KB can be stored inline in the metadata row to save a second round trip for the common tiny paste.

paste_views(key, count) is updated in batches by the view worker, never on the read path.

High-level design

Write path: the client posts to the load balancer, which forwards to a stateless paste service. The service validates size, generates a random 8-character key, uploads the text to the object store (or inlines it if small), then does a conditional insert of the metadata row so that a key collision fails instead of overwriting. Only after the metadata write succeeds does it return the URL; an orphaned blob from a failed insert is harmless and gets swept later.

Read path: raw reads go to the CDN first; because pastes are immutable, the edge can cache them until their expiry time. On a miss, the request reaches the paste service, which checks the in-memory cache, then the metadata store. If the row is missing or expires_at has passed, it returns 404 immediately. Otherwise it fetches the body from the cache or the object store and returns it.

Each successful read emits a view event to a stream; a worker sums events per key per minute and adds them to the stored counts. A sweeper job deletes expired rows and their blobs in the background.

Open the HLD board and press Reference to see the architecture diagram.

Deep dives

Splitting metadata from content

Keeping multi-megabyte values in the same database as small metadata rows hurts that database: compaction, replication and cache efficiency all degrade because a few huge values dominate the bytes. Instead, put the body in object storage, which is cheap, replicated and good at large sequential reads, and keep a ~200 byte row in the key-value store. The trade is an extra round trip on reads, so inline bodies below a threshold (about 4 KB covers more than half of pastes) and cache the rest. Write order matters: upload the blob first, then insert the row. A failure between the two leaves an unreferenced blob, which a lifecycle rule can delete, never a row pointing at nothing.

Making expiry correct and cheap

Split expiry into two jobs. Hiding is exact and happens on every read: the service compares expires_at with the current time and returns 404 once it has passed, and CDN responses carry a max-age no longer than the remaining lifetime. Deleting is lazy: a sweeper scans a time-bucketed index such as expiry_by_hour(hour, key), deletes the metadata rows, then the blobs. Scanning by bucket avoids a full-table scan. Many key-value stores also support per-row TTL, which removes the sweeper for metadata, but blobs still need a lifecycle rule or the sweeper.

Burn after reading

This is the one place reads must be exact. Two readers racing must not both see the text. Use an atomic conditional operation in the metadata store, such as delete-if-exists that returns the old row, or a compare-and-set from state=live to state=burned. Only the caller whose operation succeeds gets the body; everyone else gets 404. These pastes must never be cached at the CDN or in shared caches, so mark them Cache-Control: no-store. Link previewers (chat apps fetching the URL to build a preview card) can burn a paste accidentally, so a confirm page that requires a click before revealing the content is a sensible product fix.

Counting views without slowing reads

Incrementing a counter row on every read turns a hot paste into a hot write partition. Instead, each read emits a small event to a stream, and a worker aggregates per key over a short window, writing one increment per key per minute. A paste with 20,000 views an hour then costs 60 writes, not 20,000. Reads served entirely from the CDN never reach the service, so either accept undercounting for them or pull counts from CDN logs in a batch job. Since counts are informational, losing a few events during a stream outage is acceptable.

Failure modes

• If the object store is slow or down, small inlined pastes keep working and larger ones fail fast with 503; writes are rejected rather than half-written.
• If the view stream is down, reads continue and view events are dropped or buffered locally; counts lag but nothing user-visible breaks.
• If the sweeper stalls, storage grows but correctness holds because reads already hide expired pastes; alert on the age of the oldest unswept bucket.
• If a metadata partition fails, its replicas take over; reads for that key range briefly return errors until a new leader is chosen.

Core algorithm in code

import secrets, time

ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"


def new_key(length=8):
    """Random base62 key: 62^8 is about 2.2e14, so blind guessing is hopeless."""
    return "".join(secrets.choice(ALPHABET) for _ in range(length))


class PasteStore:
    """Toy store: metadata dict plus a 'blob' dict, with lazy expiry and burn-after-read."""
    def __init__(self):
        self.meta, self.blobs = {}, {}

    def create(self, text, ttl_s=None, burn=False, now=None):
        now = now or time.time()
        while True:
            key = new_key()
            if key not in self.meta:          # conditional put in a real store
                break
        self.blobs[key] = text
        self.meta[key] = {"expires": now + ttl_s if ttl_s else None, "burn": burn, "views": 0}
        return key

    def read(self, key, now=None):
        now = now or time.time()
        m = self.meta.get(key)
        if m is None or (m["expires"] and now >= m["expires"]):
            return None                        # expired rows are hidden before the sweeper deletes them
        if m["burn"]:
            del self.meta[key]                 # atomic delete-and-return in a real store
            return self.blobs.pop(key)
        m["views"] += 1                        # really: an event, aggregated later
        return self.blobs[key]


s = PasteStore()
t0 = 1_000_000.0
a = s.create("print('hello')", ttl_s=3600, now=t0)
b = s.create("db password rotation notes", burn=True, now=t0)
print("read a:", s.read(a, now=t0 + 10), "| views:", s.meta[a]["views"])
print("read a after 2h:", s.read(a, now=t0 + 7200))
print("burn b first:", s.read(b, now=t0 + 5))
print("burn b again:", s.read(b, now=t0 + 6))

Trade-offs

• Object storage for bodies gives cheap, durable bytes but costs an extra hop on reads; inlining small pastes wins that hop back.
• Random keys are unguessable but need a collision check on insert; counter-based keys never collide but are enumerable.
• CDN caching makes viral pastes nearly free but undercounts views and complicates deletion, so cap max-age at the remaining lifetime.
• Lazy deletion is cheap and simple but means expired bytes linger on disk for hours, which matters if users expect real erasure.

What a strong answer shows

• Separates large bodies from small metadata and explains why.
• Treats immutability as the key to aggressive caching and CDN use.
• Distinguishes hiding an expired paste (exact) from deleting it (lazy).
• Handles burn-after-read with an atomic operation and keeps it out of caches.
• Generates unguessable keys and handles collisions safely.

Mistakes to avoid

• Storing multi-megabyte text blobs in a relational table next to the metadata.
• Relying on a cron job alone for expiry, so expired pastes stay readable until it runs.
• Incrementing a view counter synchronously on every read.
• Caching burn-after-read pastes at the CDN, which lets them be read many times.
• Using auto-increment IDs as public keys, making every paste enumerable.

Follow-ups to expect

• How would you add full-text search over public pastes? (Index public pastes asynchronously into a search cluster; never index unlisted ones.)
• How do you stop people using the service to host malware or leaked credentials? (Async scanning after write, takedown flag on the metadata row, per-IP rate limits.)
• How would you support editing? (Versioned pastes: each edit writes a new blob and bumps a version pointer; caches key on version.)
• How would you serve users on another continent faster? (CDN for bodies plus read replicas of metadata in a second region.)

Practice next