You Don't Need Redis, Elasticsearch and Kafka Yet — Postgres Does All Three

There's a diagram I keep seeing. A team with maybe 2,000 users sketches their architecture, and it has six datastores in it. Postgres for the real data. Redis for caching. Elasticsearch for search. Kafka for events. Mongo for the "flexible" documents. Something time-series for the dashboard.

It looks impressive. It looks like the architecture diagrams in the conference talks. And it is, for a team that size, mostly a liability.

Here's the thing nobody says out loud when that diagram gets drawn: every box on it is a deployment. It's also a backup policy, a restore procedure you have hopefully tested, a monitoring dashboard, a set of alert thresholds, a credential to rotate, an upgrade path, a failure mode at 3am, and — the expensive one — a thing that can silently disagree with the other five boxes.

I want to make the case that Postgres already does a credible version of most of that. Not a better version. A credible one, which for a small system is the correct trade.

Caching: an UNLOGGED TABLE is genuinely fast

Most applications that say "we need caching" mean: there are a few thousand rows that get read constantly and change rarely, and recomputing them is annoying.

Postgres has a purpose-built thing for exactly this. An unlogged table skips the write-ahead log entirely. That makes writes dramatically cheaper, at the cost of the table being truncated if the server crashes — which for a cache is not a bug, it's the definition.

CREATE UNLOGGED TABLE cache_entries (
    key         text PRIMARY KEY,
    value       jsonb NOT NULL,
    expires_at  timestamptz NOT NULL
);

CREATE INDEX ON cache_entries (expires_at);

-- write / overwrite
INSERT INTO cache_entries (key, value, expires_at)
VALUES ('dashboard:org:42', '{"revenue": 91200}'::jsonb, now() + interval '5 minutes')
ON CONFLICT (key) DO UPDATE
    SET value = EXCLUDED.value,
        expires_at = EXCLUDED.expires_at;

-- read
SELECT value
FROM cache_entries
WHERE key = 'dashboard:org:42'
  AND expires_at > now();

That's a primary-key lookup on a table that's almost certainly fully in memory. It is fast in a way that will surprise you if you've only ever thought of Postgres as "the slow durable thing".

Being fair to Redis: it wins decisively on raw operations per second, on the data structures Postgres doesn't have (sorted sets, HyperLogLog), on pub/sub, and on atomic counters under heavy contention. Those are real advantages. They are just not what most teams mean when they say the word "caching".

Full-text search: tsvector plus a GIN index

This is the one people underestimate most. Postgres full-text search is not a toy. You get stemming, stop words, ranking, prefix matching and multi-column weighting, all in the query planner you already understand.

ALTER TABLE articles
ADD COLUMN search_vector tsvector
GENERATED ALWAYS AS (
    setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
    setweight(to_tsvector('english', coalesce(body,  '')), 'B')
) STORED;

CREATE INDEX articles_search_idx ON articles USING GIN (search_vector);

The generated column keeps itself in sync on every insert and update — no trigger to write, no drift, no reindex job. Then the query:

SELECT id,
       title,
       ts_rank(search_vector, query) AS rank
FROM articles, websearch_to_tsquery('english', 'postgres queue locking') AS query
WHERE search_vector @@ query
ORDER BY rank DESC
LIMIT 20;

websearch_to_tsquery is the one to reach for, because it parses the syntax users actually type — quoted phrases, or, a leading minus to exclude. And ts_rank gives you real relevance ordering, weighted by the setweight calls above so a title match beats a body match.

My honest read on the ceiling: this holds up well into the millions of rows on ordinary hardware. I'm deliberately saying "well into the millions" rather than quoting a number, because the real limit depends on your document sizes, your update rate and how many filters you stack on top. Test it against your own data rather than trusting anyone's headline figure, mine included.

Elasticsearch earns its keep when relevance tuning becomes an ongoing product activity — synonyms, custom analyzers, per-language stemming, faceted navigation across large result sets, typo tolerance you actively tune. If someone on your team is going to spend real time on search quality, you want the tool built for that. If search is a box in the header that needs to find the right thing, Postgres is fine.

Queues: FOR UPDATE SKIP LOCKED is a real pattern

SKIP LOCKED arrived in Postgres 9.5 and it quietly solved the "you can't build a queue on a database" objection. It lets a worker claim rows while stepping over rows another worker has already locked, with no contention and no lost jobs.

CREATE TABLE jobs (
    id           bigserial PRIMARY KEY,
    payload      jsonb NOT NULL,
    status       text NOT NULL DEFAULT 'pending',
    run_after    timestamptz NOT NULL DEFAULT now(),
    attempts     int NOT NULL DEFAULT 0,
    locked_at    timestamptz
);

CREATE INDEX jobs_claim_idx ON jobs (run_after)
WHERE status = 'pending';

And the claim query — this is the whole trick:

UPDATE jobs
SET status = 'running',
    locked_at = now(),
    attempts = attempts + 1
WHERE id IN (
    SELECT id
    FROM jobs
    WHERE status = 'pending'
      AND run_after <= now()
    ORDER BY run_after
    FOR UPDATE SKIP LOCKED
    LIMIT 10
)
RETURNING id, payload;

Run twenty workers against that and each one gets a disjoint batch. No double processing, no polling storm, no separate broker. And because it's the same transaction as your business data, you get something an external broker cannot give you for free: enqueue a job and write the row it depends on atomically. Either both happen or neither does.

That property is worth more than most teams realise. A huge share of "the job ran before the record was committed" bugs simply cannot occur in this design.

JSON documents and analytics, briefly

Two more that follow the same shape.

jsonb covers most "we need Mongo" cases. You get schemaless columns, containment operators, path queries and GIN indexing on the whole document:

CREATE INDEX events_props_idx ON events USING GIN (properties jsonb_path_ops);

SELECT * FROM events WHERE properties @> '{"plan": "pro", "source": "referral"}';

The thing you keep, that a separate document store takes away, is joining that document to your relational data in one query.

Time series and analytics are further than people expect too. Window functions handle running totals and rankings. generate_series fills gaps so your chart doesn't skip empty days. Materialized views precompute the expensive rollups on a schedule. That combination carries a dashboard a very long way before a specialist columnar store starts paying for itself.

The real cost isn't the license

Every one of these tools is free to download. That's not where the money goes.

The cost is operational surface. Concretely, per additional datastore:

That last row is the killer. With one store, your data is either right or wrong. With two, it can be right in one and stale in the other. With six, the number of pairs that can disagree grows quadratically — six stores means fifteen pairs. Every one of those pairs is a class of bug that only shows up in production, usually as "the search results show a product we deleted."

You don't debug those quickly. You debug them by reasoning about two systems at once, at night.

When each tool genuinely earns its place

I don't want to be the person telling you Postgres does everything. It doesn't. Here's my honest read on when to actually split, with a concrete signal for each.

Redis earns it when you have very high-throughput ephemeral state — rate limiting, session counters, leaderboards, presence, pub/sub fan-out to many subscribers. You'll know because your cache writes have become a meaningful share of your Postgres write load, or because you're doing tens of thousands of counter increments a second and seeing lock contention on a single hot row. That's the signal. Not "we should probably cache things."

Elasticsearch earns it when search relevance is the product. Marketplaces, job boards, documentation portals, anything where a bad ranking loses a sale. You'll know because someone is filing tickets about result quality specifically, you want synonyms and typo tolerance and faceted counts across millions of documents, or you're supporting several languages with different analyzers. When search becomes a thing you tune weekly rather than a feature you shipped, move.

Kafka earns it when you need a durable, replayable event log consumed by several independent services at their own pace. You'll know because you want to add a new consumer and have it read the last thirty days of history from the beginning — that single requirement is the honest test.

And that Kafka one deserves emphasis, because it's the most commonly mis-picked tool of the three. Kafka is a log, not a queue. Many teams reach for it wanting "background jobs with retries" and end up hand-building consumer groups, offset management and dead-letter handling — reimplementing a job queue on top of something that was never a job queue. If what you want is "run this task later, retry on failure", SKIP LOCKED or a proper broker is the closer fit.

Starting on Postgres does not trap you

This is the argument I find most convincing, and it's the one that gets skipped.

The fear is that starting simple means a painful rewrite later. In practice it's the opposite. Moving one workload out of Postgres is a bounded project with a clear definition of done. Move search to Elasticsearch: you already know your query shapes, your document structure and your relevance requirements, because you've been running them in production. You dual-write, you compare results, you cut over, you delete the old code. Two weeks, maybe four, and it ends.

Running six systems from day one is not a project. It's an ongoing tax, paid every sprint, by everyone, forever — and paid at your least experienced moment, when you have the least idea which of those systems you'll actually need.

You're also making the extraction easier by waiting. The team that migrates search after two years of running it in Postgres knows exactly what good relevance looks like for their data. The team that chose Elasticsearch on day one is guessing at analyzer configuration for a product that doesn't exist yet.

My personal take

I think the six-box diagram is mostly a status object. It signals seriousness. It looks like what real engineering is supposed to look like, and there's a quiet fear that a diagram with two boxes means you're not building anything real.

But the actual engineering skill on display in a two-box architecture is usually higher. Knowing that SKIP LOCKED exists, that unlogged tables are a real cache, that a generated tsvector column keeps itself in sync — that's more specific knowledge than knowing Redis is fast.

We're a small studio. The one product we've shipped end to end, an Ayurvedic e-commerce platform on Angular and Laravel, runs on a single relational database, and every time I've been tempted to add a second store the honest answer was that I wanted the diagram, not the capability.

So my rule is simple. Add the specialist store when Postgres has actually stopped being good enough at the thing, with a metric you can point to. Not when you imagine it might. The imagined version of that moment arrives about two years before the real one, and everything you build in between is paid for twice.

The short version

• Six datastores means six deployments, six backup policies, six runbooks — and fifteen pairs that can disagree.
• UNLOGGED TABLE is a genuinely fast cache; most apps' cache is a few thousand rows.
• tsvector + a GIN index gives real ranking and stemming, and holds up well into the millions of rows.
• SELECT ... FOR UPDATE SKIP LOCKED is a correct, battle-tested queue — and it's transactional with your business data.
• jsonb with GIN covers most "we need Mongo"; window functions and materialized views cover most early analytics.
• Redis wins on throughput and pub/sub. Elasticsearch wins when relevance is the product. Kafka wins when you need a replayable log — which is not a job queue.
• Extracting one workload later is a bounded project. Running six from day one is an unbounded tax.

Written by the TechKis team — an AI-first engineering studio. techkis.tech