Postcard helps people discover places worth visiting. To do that, the app needs a steady stream of recommendations from publishers, and every publisher writes about places differently. One puts the restaurant name in a heading. One buries it in a paragraph. One publishes structured JSON, one publishes a wall of HTML. The place names then have to become real, deduplicated records with coordinates before the app can use them.
That is the job Journo does. I designed and built it end to end, and the code is public at github.com/rndm-jpg/journo. This post is the walkthrough I wish someone had written for me before I started.
The shape of the problem
Three things make this harder than "scrape and save":
- Every source is its own structure. A single generic scraper produces garbage. Each source needs a crawler that knows where the content lives and a parser that knows what a place mention looks like there.
- A name is not a place. "Sunda Kelapa" is a harbor, a mosque, a district, and several restaurants. The pipeline has to turn a string into a specific location with coordinates, or admit it cannot.
- The work never ends. New articles arrive daily, and a good match today should not be rediscovered and re-uploaded tomorrow.
How it is built
Source registration. A source is an identifier tied to a crawler function and a parser module. Adding a publisher means adding those two pieces and registering them. Nothing else in the pipeline changes.
Crawl and parse. Crawlers collect raw content per source. Parsers pull candidate place mentions out of that source's HTML or JSON, along with whatever context sits around them.
Enrichment with spaCy. Before matching, spaCy adds location context to each candidate, so "the cafe on the corner in Kemang" carries "Kemang" into the search rather than losing it.
Matching with SerpApi and Who's On First. Each enriched candidate is searched, then cross-referenced against Who's On First gazetteer data so the coordinates and the administrative context agree.
Scoring and thresholds. A match is scored on names, descriptions, contacts, and websites. A configurable threshold drops weak matches. A unique place identifier stops the same place being processed twice across sources and runs.
Operator dashboard. A React and Next.js frontend, backed by Django APIs, shows each source's processing session, how many pages and places it has handled, and where it stopped. Nobody should have to read logs to know whether last night's run worked.
Parallelization. The amount of concurrent work is configurable, so the same pipeline runs on a laptop and on a proper server.
What I would tell a founder
The interesting engineering was not the scraping. It was deciding where a human needs to be able to look. Every stage writes its state where the dashboard can show it, because a pipeline that fails silently costs more than one that never ran.
The second lesson: keep the per-source code small and boring. The clever parts, enrichment, matching, scoring, are shared. The parts that break, the parsers, are isolated and cheap to fix.
If your product depends on turning messy public content into structured records, this is a workable pattern. Read the matching write-up in the repository if you want the detail, and the project story for the shorter version.
What I deliberately left out
There are no throughput numbers in this post, because the honest answer is "it depends on the source, the hardware, and the threshold you choose." A strict threshold gives you fewer, cleaner places. A loose one gives you volume and a review queue. That trade-off belongs to the product owner, which is why it is a configuration value and not a constant buried in the code. The same goes for parallelization: the pipeline should fit the machine you have, not demand the machine I had.
Isolate what breaks, share what is clever, and make every stage visible to a person.