How we built first-party attribution for our own funnel

We built this because we needed it, then discovered the version we needed was the product. Here is the engineering, including the parts that took longer than the estimate.

The data model

visitors(id, property_id, anonymous_id, person_id,
         ft_source, ft_medium, ft_campaign, ft_landing_url, ft_click_id, ft_occurred_at,
         lt_source, lt_medium, lt_campaign, lt_occurred_at,
         first_seen_at, last_seen_at)

people(id, property_id, email, identified_at, identified_via)

events(id, visitor_id, person_id, name, category, source,
       dedup_key, properties, occurred_at, discarded_at)

conversion_events(id, visitor_id, person_id, value_cents, external_ref,
                  ft_source, ft_medium, ft_campaign, occurred_at)

Two things there are deliberate and worth stealing.

First and last touch are denormalised onto the visitor. ft_* is written once, when the visitor is created with campaign parameters; lt_* is overwritten on every touch that carries them. Reporting then becomes a grouped aggregate over indexed columns instead of a correlated subquery per row against the fastest-growing table in the system.

Conversions carry a copy of the first touch. Denormalised again, at write time. A conversion is an immutable financial fact; it should not change because a later merge altered a visitor row.

The facade, and why it is the important part

Every write goes through one object:

Signals::Client.pageview(property:, anonymous_id:, source:, url:, referrer:)
Signals::Client.track(property:, anonymous_id:, source:, name:, category:, dedup_key:, properties:)
Signals::Client.identify(property:, anonymous_id:, email:, source:, via:)
Signals::Client.purchase(property:, anonymous_id:, email:, value_cents:, external_ref:, source:)

The public API controller calls it. Our own dogfooding calls it. Neither ever touches the internal services — Ingest, Identify, RecordConversion, RecordEvent — directly.

That rule is not architectural tidiness. It is what keeps the dogfooding meaningful. If our own instrumentation could reach into the internals, we would gradually build a private path that customers do not have, and our funnel would stop being a test of the product. Instead, when we break the public contract, our own numbers break first — in production, on data we care about, immediately.

The corresponding boundary in the codebase: anything flowsk-specific lives under Signals::Internal::* and may only call the facade. Anything that would serve any customer is product.

The five things that were harder than expected

1. Bots

The first version counted every crawler, uptime monitor, link previewer and prefetch as a visitor. The number was wrong by an embarrassing margin, and we found out by presenting it.

Now: user-agent filtering, Turbo-Frame header detection, and the family of prefetch headers (Sec-Purpose, X-Sec-Purpose, Purpose, X-Moz, Next-Router-Prefetch). Turbo 8’s prefetch in particular will silently double your pageview count if you do not handle it.

2. Identity merges

One person, two devices, two anonymous ids, one email — and events arriving late, out of order, and twice.

The merge has to be idempotent (running it again is a no-op) and order-independent (same final state regardless of arrival sequence). Getting that right took two weeks and produced subtle wrong-number bugs for a month after we thought it was done.

The guard that fixed most of it is one line: return early if the visitor is already bound to that person, before doing any backfill.

3. De-duplication

Client and server both report the purchase. We started with a time-window heuristic — “same visitor, same value, within five minutes” — and it was wrong in both directions: it merged genuinely distinct purchases and missed pairs that arrived far apart.

Explicit dedup_key, permanent, keyed on (property, dedup_key). Boring and correct. It also gave us webhook idempotency for free, which we had been about to build separately.

4. Ingestion off the request path

Recording a pageview inline adds latency to every page. Everything now goes through Active Job, with rate limits per visitor (20/minute) and globally, so a script cannot flood the queue.

The subtlety: events must carry their own occurred_at, captured at request time. Using the job’s execution time makes every timestamp wrong by the queue depth, which is invisible until you look at a journey and the ordering is nonsense.

5. Time

Late-arriving server events change yesterday’s totals after you already reported them. Timezones make “today” ambiguous. “Last 30 days” means three different things to three stakeholders.

No clever solution — just being explicit everywhere about which clock a number is on, and accepting that recent numbers move.

What we would do the same

• The facade. It is the reason the dogfooding is honest.
• Denormalised first touch. Reporting stayed fast for free.
• Explicit dedup keys. Every heuristic we tried was worse.
• Server-set cookie from day one. Everything downstream depends on it and retrofitting it would have been painful.

What we would do differently

• Filter bots before writing, not after. We cleaned up historical data twice.
• Record the identification cause from the start. We added identified_via later and could not backfill it.
• Build the journey view earlier. It is the debugging tool, and we built the reports first — then spent weeks debugging aggregates without being able to see the events underneath them.

That last one is the strongest recommendation in this post. If you build attribution, build the raw event inspector before you build a single chart.

Full honest scope, including when building is the right call, is in the build-vs-buy comparison. The Rails specifics are in the Rails guide.