Phantom Trails and Closed Cafés: Grounding Travel AI in Live Data

A guide we work with in the Southwest keeps a folder of screenshots. Each one is an answer from an AI assistant that a guest showed him, usually with some embarrassment, after it had sent them somewhere it shouldn’t have.

The best of the collection is a single answer. It recommended a sunrise hike to a ridge whose trailhead had been closed for fire restrictions the week before. It suggested breakfast afterward at a café “right at the trailhead” — a café that had closed two winters earlier and had, in any case, been twelve miles away in town. And it ended with a soak at nearby hot springs that sit on private land.

Nothing in that answer was invented from thin air. The ridge is real. The café was real. The springs are real. Each piece was true somewhere, at some point, for someone. That is what makes phantom venues stubborn: they aren’t random fabrications that a smarter model will outgrow. They are stale facts, fused facts and facts without a clock.

A field guide to phantoms

Six kinds turn up again and again. Naming them matters, because each has a different cause and a different fix.

Phantom	What the traveler meets	Usual cause	What catches it
The ghost	A place that has closed for good	Old listings, model memory	Existence check against a catalog with last-verified dates
The shape-shifter	It exists, but hours, owners or booking rules changed	Stale fields	Per-field freshness budgets
The sleeper	It’s seasonal, and the trip is out of season	No calendar awareness	Seasonal calendars evaluated against the trip date
The chimera	Two real places fused into one	The model blending nearby facts	One source record per claim; relationships computed, not generated
The mirage	Open, but not reachable as planned	No routing or live conditions	Route feasibility with current road and trail status
The gatekeeper	Open and reachable, but needs a permit or timed entry that’s gone	Policies not modeled	Policy and availability check

Ghosts are the famous kind, yet they’re often not the most common. In our own evaluation runs, more than half of the phantoms produced by a retrieval-only setup were chimeras and mirages — real places with wrong relationships. “Ten minutes from your lodge” is a claim. So is “right at the trailhead.”

Why “add retrieval” isn’t the whole fix

Retrieval-augmented generation — handing the model relevant documents before it answers — is necessary, and it isn’t enough. Four gaps remain:

1. The documents can be stale. Retrieving a two-year-old page grounds the answer in a two-year-old world.
2. The model can still blend. Given a café record and a trailhead record, it may cheerfully place one at the other.
3. Relationships are generated. Distances, “nearby,” “on the way” — none of these live in any single document.
4. Time is never checked. A record that says “open daily 7–3” says nothing about the Saturday in October the traveler is asking about.

The design that works treats every recommendation as a claim about the world at a particular time, and checks the claim before it becomes a sentence.

Plan first, words second

Our rule of thumb: the model drafts a plan, not prose. Each stop in the draft becomes a structured item — a venue ID, a date, a time window, a mode of travel. Then four gates run on the plan:

1. Exists. Does the venue ID resolve to a live, non-retired record?
2. Open then. Is it open in that window on that date, given its seasonal calendar and latest known hours?
3. Reachable. Can the traveler get there in time, by their mode of travel, under current road and trail conditions? The router answers this, not the model.
4. Permitted. Are permits, timed-entry tickets, age limits or reservations required, and are they available?

Only stops that pass all four gates go back to the model to be written up. Stops that fail are replaced from the catalog or explained honestly. This is how we build Nova’s Concierge and the proposal side of Itinerary Studio: language is the last step, not the first.

The screenshot, re-run through the gates

Here is the guide’s favorite bad answer, drafted as a plan for a Saturday in October and checked stop by stop:

Planned stop	Exists	Open then	Reachable	Permitted	Outcome
Sunrise ridge hike	Yes	No — closure notice six days old	—	—	Replaced with a lower canyon trail outside the closure area
Café “at the trailhead”	No — record retired two winters ago	—	—	—	Replaced with a bakery in town, open Saturdays from 7
Bakery in town	Yes	Yes	Yes — 25 minutes by car, computed	Yes	Kept, moved before the hike
Hot springs	Yes	Yes	Yes	No — private land, no public access	Dropped; no substitute offered

Three of the four original stops fail, each at a different gate, and each failure has a different fix. Notice the last row: sometimes the honest replacement for a phantom is nothing at all. A shorter plan that works beats a fuller one that doesn’t.

Give every fact a clock

“Verified” is not a property of a record. It’s a property of each field, and it expires. A workable set of freshness budgets:

Fact	Freshness budget	Typical source	Behavior when stale
Venue exists and operates	90 days	Supplier or listing confirmation	Recommend only with a caveat, or leave out
Regular hours	30 days	Listing or supplier	Suggest, and say “check hours”
Seasonal opening	Confirmed each season	Supplier	Treat as closed
Event date and time	Confirmed within 14 days	Organizer	Don’t schedule around it
Trail and land status	24 hours in fire season	Land-manager notices	Don’t route exposed or restricted trails
Road conditions	1–3 hours during storms	Road-condition feeds	Add buffer and offer an alternative
Accessibility details	180 days, or after renovation	Site check	Don’t make the claim

The last column is the important one. Stale doesn’t mean wrong; it means unverified. The answer should change register accordingly — from assertion (“opens at 7”) to suggestion with a check (“usually opens early; worth confirming the night before”).

Keeping these clocks honest is a data job more than a model job. The cleanup, deduplication and field-authority rules behind them are what travel data engineering is for.

Abstain like a good guide

A guide who doesn’t know whether a trail is open says so, then offers something they’re sure of. Assistants should sound the same. Compare:

Ungrounded: “After your hike, grab breakfast at the café right at the trailhead — it opens at 7.”

Grounded: “There’s no café at the trailhead. The nearest breakfast spot I can confirm is open on Saturdays is back in town, about 25 minutes away, so it’s easier before the hike than after. I’d also skip the hot springs; they’re on private land.”

Abstention has a cost, too. An assistant that hedges every sentence is useless, and travelers stop reading. Track an abstention rate alongside the phantom rate, and treat a rising abstention rate as a freshness alarm: it usually means records are aging faster than anyone is confirming them.

Measure a phantom rate

Define it plainly: the share of answers containing at least one recommendation that fails a gate for the date of the trip. Build a fixed test set — a few hundred real questions across seasons and regions, with traps — and run it monthly and after every model or data change.

In one internal evaluation of 400 destination questions across three regions, the results looked like this:

Setup	Answers with at least one phantom
General-purpose model answering from memory	14%
Retrieval only	6%
Retrieval plus four gates and freshness budgets	0.8%

The remaining misses all traced to records whose clocks hadn’t expired but were already wrong — a café that closed on two weeks’ notice, a trail closure posted late. The residual was a data-latency problem. The fix was a faster signal from the people who know, not a better model. Watching exactly these numbers week to week is a core part of managed AI operations.

Your public pages are someone else’s ground truth

Grounding your own assistant is half the job. General-purpose assistants learn about your destination or business from what you publish, and stale pages turn into phantoms in answers you will never see.

• Retire pages for closed products properly, with a clear “permanently closed” state rather than a silent error page.
• State seasonal calendars with dates, in text and in structured data.
• Publish hours, meeting points and policies where machines can read them.
• Keep one canonical page per real-world thing.

This is the engineering side of AI search visibility: being cited correctly matters as much as being cited at all.

A grounding checklist

1. Every recommendable venue has a stable ID, coordinates and a last-verified date per field.
2. Every seasonal venue has a seasonal calendar, and the planner evaluates it against trip dates.
3. Distances, travel times and “nearby” claims come from the router.
4. Staff can post a closure and see it reflected in minutes.
5. Four gates run on the plan before the model writes a word.
6. Stale fields soften the language automatically.
7. Phantom and abstention rates are measured monthly on a fixed test set.
8. A traveler’s report of a phantom fixes the underlying record within a day and adds the case to the test set.

The guide with the folder still collects screenshots. The goal is that none of them come from you.