The math

Rollups and series

A season of one-per-second telemetry is tens of millions of readings. Scrubbing a race through it has to feel instant — here is what makes it so, and what that costs.

The problem, in numbers

A boat with a normal instrument system produces a reading per second per channel. Thirty channels, six hours on the water, twenty days a season: that is around thirteen million readings a year, per boat. A two-hour coastal race on its own is a couple of hundred thousand.

Now consider what the debrief player asks for. Drag the scrubber across a three-hour day and the chart has to redraw at whatever zoom you happen to be at — and at three hours wide, a chart a thousand pixels across has room for about one point per eleven seconds. Fetching every reading to draw it would move ten times more data than the screen has pixels to put it in, and it would do it on every drag.

So the platform does not fetch every reading. It fetches summaries.

1 Summaries, folded as the data arrives

Alongside the recorded readings, the platform keeps a summary of each channel at two resolutions: one second and one minute. A summary — the platform calls it a rollup — is not an average. It is the ingredients an average is made of, which turns out to matter enormously:

per channel, per bucket: n how many readings landed in it sum what they add up to min the smallest max the largest

Read as a sentence: keep the count and the total, not the average — because counts and totals can be combined and averages cannot.

Buckets are aligned to the UTC second and the UTC minute, so the same instant always falls in the same bucket no matter who is asking or from where.

The fold

The summaries are built as data arrives, not by a nightly job that rebuilds them. When a batch of readings lands — from a file import or from a boat streaming live — each bucket it touches is updated in place:

n ← n + n(new) sum ← sum + sum(new) min ← smaller of ( min , min(new) ) max ← larger of ( max , max(new) )

Read as a sentence: counts and totals add; smallest and largest fold by taking the more extreme of the two.

Both resolutions come from the same pass, and the whole thing happens inside the transaction that stores the readings — so a reader can never see readings that have no summary, or a summary that has run ahead of its readings.

One detail carries more weight than its size suggests: the update is computed from the readings that were actually stored, not from the batch that was offered. Imports in Nordic Stars are idempotent — upload the same file twice and the second one adds nothing — and this is where that promise is kept for the summaries too. A redelivered batch, which happens routinely when a tablet drains an offline queue, cannot inflate a single count.

2 Angles cannot be averaged

Take two wind direction readings a second apart: 359° and 1°. The wind moved two degrees. Average them the ordinary way and you get:

( 359 + 1 ) / 2 = 180 — due south, from a wind blowing due north

This is not an edge case that shows up rarely. A northerly sits on that seam all day, and every summary of it would be wrong by 180 degrees.

What is stored instead

For an angular channel, each reading is treated as a direction: an arrow of length one pointing that way. What gets stored is the total of the arrows, kept as its two components:

sinSum = Σ sin(valueᵢ) cosSum = Σ cos(valueᵢ)

Read as a sentence: add the directions together as arrows, so the wrap at 360° has nothing to catch on.

And the average direction is the direction that total arrow points:

mean = atan2( sinSum , cosSum ) brought into [0°, 360°)

Read as a sentence: which way is the sum of all those arrows pointing? That is the average heading.

Sine first, cosine second — the classic thing to get backwards, and backwards gives you an angle mirrored about the 45° line, which looks plausible enough to survive a review. Run 359° and 1° through it and the answer is 0°, which is what the wind was doing.

These two sums are also what makes an angular calibration correction instant: rotating the summed arrow by the correction gives exactly the same answer as rotating every reading and re-summing.

Whether a channel is angular is a property of the channel in the platform’s registry, not a guess from its name. Angular channels also carry the ordinary total, because the storage is shared — but the platform never serves it, because a naive total across the 0°/360° seam is meaningless and a meaningless number that looks trustworthy is worse than no number at all.

3 What deletion costs

Counts and totals have a convenient property: they can be undone. Remove a reading and you could subtract one from the count and its value from the total, and the summary would be right again.

Smallest and largest have no such property.

If the coldest day of the year is removed from the record, the second coldest becomes the coldest — and nothing in the summary knows what that was. A minimum cannot be subtracted; it can only be found again.

So when data is deleted — a race removed from the debrief, an import withdrawn — the platform does not attempt to unwind the summaries. It does the only correct thing:

1. find the span of time the deleted readings covered 2. drop every summary bucket in that span 3. rebuild them from the readings that survived

Read as a sentence: the affected windows are recomputed from what is left, rather than patched.

Three consequences worth knowing, because they are what this choice buys:

  • The cost is the size of the deletion, not the size of the season. Delete one race and the rebuild touches that race’s minutes. A boat with five years of history pays the same as a boat with five days.
  • A bucket with nothing left simply does not come back. There is no empty summary sitting where data used to be, claiming a count of zero.
  • It happens inside the same transaction as the deletion. There is no window in which the readings are gone and the summaries still describe them.

4 What the player actually reads

Three resolutions are available to a chart — the recorded readings, the per-second summary and the per-minute summary — and the debrief player loads a race at the per-second level, which is finer than any screen can draw at race width. The channel decides how each bucket becomes a number — total over count for a speed, the arrow’s direction for a heading — and any calibration correction is applied on the way out, at every level.

The result is that zooming changes what is fetched but not what is true. The average boat speed of a minute is the same number whether you asked for it as a minute or worked it out from sixty seconds, because both are made of the same counts and totals.

A summary answers what a stretch of time looked like on average and at its extremes. It does not preserve the shape inside the bucket: a minute of half 6 knots and half 8 averages to the same 7 as a minute of steady 7. That is why the per-second resolution exists at all, and why the recorded readings stay available underneath it.