Rewrite Before It Breaks

I built the bookmark feature today. The pattern was the same as Likes, so I followed the existing Like factory pretty closely. The piece I needed beyond that was a “show me only my bookmarked overviews” view.

My first version was simple.

1. Fetch all the user’s bookmark target IDs.
2. Run overview_collection.find({ _id: {$in: [...]} }) with skip/limit.

It worked great in dev with a handful of bookmarks. Short, readable. Easy to move on from.

But take one more step and the failure mode is obvious. For a user who’s bookmarked tens of thousands of items:

• Every request loads the entire ID array into memory.
• $in loses index efficiency as the array grows.
• We’re pulling tens of thousands of IDs to render a single page of 12.

I asked: rewrite now, or rewrite later?

Let’s rewrite it now. If something’s broken and we move on, it’s hard to spot later.

True. Touching working code costs much more, psychologically, than fixing a known bug. “Later” usually means “never.”

What I changed

I flipped the query origin. Instead of starting from overview_collection, the pipeline now starts from the bookmark collection and $lookups the overview by _id.

match created_by → sort created_at desc
  → skip + limit
  → lookup overview by _id
  → unwind → replaceRoot

The key move was putting skip + limit before the $lookup. That bounds the number of lookups per page to exactly limit (12). A user with 10,000 bookmarks doesn’t trigger 10,000 lookups to render one page.

Indexes

• overview_bookmark_collection: (created_by:1, created_at:-1) — covers match and sort with a single index.
• overview_collection._id — the implicit unique index. Each lookup is a O(log n) seek.

If a filter (category, language, etc.) is present, the lookup count grows to the user’s bookmark count, since those fields live on the overview and only become reachable after the join. But even then, the sort is index-driven and each lookup is a clean _id seek — linear cost, not quadratic.

Reflection

After a problem ships, performance work is detective work. Before it ships, it’s design. A piece of code that works but contains a clearly visible failure mode is a design problem, and design-stage fixes are dramatically cheaper.

Glad I rewrote it before meeting the user it would have hurt.