Spec gap: `reduce` and `scan` operator completion semantics are undefined #10

New issue

Open

opened 2026-05-27 05:25:20 +00:00 by hurui200320 · 0 comments

hurui200320 commented

2026-05-27 05:25:20 +00:00

Member

Problem

§5.3.2 describes reduce and scan as stateful aggregation operators:

scan — emits intermediate results
reduce — emits only the final result

However, neither operator defines what constitutes "intermediate" vs. "final", and neither defines what triggers the final emission. For reduce in particular, "only the final result" is meaningless without a definition of when the stream is "done."

Key unanswered questions:

What triggers a reduce emission? Only when the upstream stream signals completion? After a fixed number of messages? After a timeout? Cold streams may never signal completion in a long-running execution.
What is the type of the emitted value? The accumulator types (collect, concat, sum) produce a list, string, and number respectively — but does reduce emit the raw accumulated value or wrap it in a message?
What happens to reduce if only one message arrives and then the stream ends? Is that message the "final result"?
For scan — is every intermediate result emitted as a separate message downstream? If so, does the message content get replaced by the accumulated value at each step?

Impact on Library Implementation

reduce and scan cannot be correctly implemented without completion semantics. An implementation that emits on every message (treating each as "final") is inconsistent with reduce's stated purpose. An implementation that waits forever for a completion signal will hang on any finite stream.

Fix Needed

Specify:

The completion trigger for reduce (and whether it applies to scan)
The emission shape for both operators
Behavior when zero messages have been processed

References

§5.3.2 — scan / reduce operators
§5.3.6 — Accumulators
§5.3.1 — Stream Types (completion behavior may vary by cold/hot/replay)

## Problem §5.3.2 describes `reduce` and `scan` as stateful aggregation operators: - `scan` — emits intermediate results - `reduce` — emits only the final result However, neither operator defines what constitutes "intermediate" vs. "final", and neither defines what triggers the final emission. For `reduce` in particular, "only the final result" is meaningless without a definition of when the stream is "done." Key unanswered questions: 1. **What triggers a `reduce` emission?** Only when the upstream stream signals completion? After a fixed number of messages? After a timeout? Cold streams may never signal completion in a long-running execution. 2. **What is the type of the emitted value?** The accumulator types (`collect`, `concat`, `sum`) produce a list, string, and number respectively — but does `reduce` emit the raw accumulated value or wrap it in a message? 3. **What happens to `reduce` if only one message arrives and then the stream ends?** Is that message the "final result"? 4. **For `scan` — is every intermediate result emitted as a separate message downstream?** If so, does the message content get replaced by the accumulated value at each step? ## Impact on Library Implementation `reduce` and `scan` cannot be correctly implemented without completion semantics. An implementation that emits on every message (treating each as "final") is inconsistent with `reduce`'s stated purpose. An implementation that waits forever for a completion signal will hang on any finite stream. ## Fix Needed Specify: 1. The completion trigger for `reduce` (and whether it applies to `scan`) 2. The emission shape for both operators 3. Behavior when zero messages have been processed ## References - §5.3.2 — `scan` / `reduce` operators - §5.3.6 — Accumulators - §5.3.1 — Stream Types (completion behavior may vary by `cold`/`hot`/`replay`)

Rows
Columns

Spec gap: reduce and scan operator completion semantics are undefined #10

Problem

Impact on Library Implementation

Fix Needed

References

Spec gap: `reduce` and `scan` operator completion semantics are undefined #10