Skip to content

Conversation

@stefanmb
Copy link

Summary

  • When --start is omitted, synchronize now uses the intermediate table's
    partition range to compute the starting ID, instead of scanning the entire
    source table with an unbounded MIN(id).
  • Batch queries are also time-bounded to the partition range, preventing
    "no partition found" errors when source rows fall outside existing partitions.
  • Shared partition resolution helpers (transformIdValue,
    resolvePartitionContext, resolvePartitionTimeFilter) are consolidated
    in table.ts, eliminating duplication across synchronizer.ts and
    filler.ts.

Motivation

Fill has always resolved partition settings from the destination table and
applied a time filter to its batch queries. Synchronize was the odd one out —
it did a bare MIN(id) across the entire source table and had no time bounds
on batch fetches.

Callers no longer need to manually compute a starting ID based on partition
boundaries. The --start option still works for explicit overrides, but the
default behavior is now partition-aware.

What changed

  • table.ts: transformIdValue exported (with nullable/non-nullable
    overloads). New resolvePartitionContext returns settings, partitions, and
    time filter in one call. resolvePartitionTimeFilter is a convenience
    wrapper. partitions() signature widened to CommonQueryMethods.
  • synchronizer.ts: Uses shared helpers. init passes the time filter
    to sourceTable.minId() when --start is omitted. #fetchBatch applies
    the time filter to batch queries.
  • filler.ts: Replaced inline partition resolution and local
    transformIdValue with shared imports from table.ts.

Test plan

  • New test: verifies only in-range rows are synchronized when --start
    is omitted and partitions exist
  • Existing synchronize tests continue to pass
  • Integration tests with PGSLICE_URL

🤖 Generated with Claude Code

When --start is omitted, synchronize now resolves the partition time
range from the intermediate table and uses it to scope the initial
MIN(id) lookup and batch queries — matching how fill already works.

Shared partition resolution logic (transformIdValue, resolvePartitionContext,
resolvePartitionTimeFilter) is consolidated in table.ts, eliminating
duplication across synchronizer.ts and filler.ts.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@stefanmb stefanmb requested a review from mthadley February 12, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant