Skip to content

Port to TypeScript#15

Merged
mthadley merged 126 commits intomasterfrom
port-to-typescript
Feb 5, 2026
Merged

Port to TypeScript#15
mthadley merged 126 commits intomasterfrom
port-to-typescript

Conversation

@mthadley
Copy link
Collaborator

@mthadley mthadley commented Feb 4, 2026

Rationale

WorkOS is a TypeScript-centric engineering organization. Most of our stack runs on TypeScript with all engineers having strong proficiency with the language. Pgslice was originally written in Ruby (for which many of us have a strong affinity for, myself included), which presents several challenges:

  • Customizations to Pgslice (which we've already made) become harder to make and review, as Ruby expertise is not as prevalent.
  • Invoking pgslice means needing to have a Ruby interpreter available, complicating environments, and increasing our surface area for auditing/making (e.g. making sure we are running supported Ruby versions, gems, etc.)
  • Ruby pgslice only exposes a CLI tool, whereas in certain contexts we'd like the option to also invoke it as a TypeScript library, without needing to go through any kind of CLI wrapper, or parsing IO streams.

So porting pgslice to TypeScript makes contributions from our team easier, reduces maintenance overhead for our foundation team, and lets us integrate more directly for future automations.

Feasibility

Is it feasible or rationale to port a tool like pgslice? It's been done before, albeit in the form of an experimental Go package. But we think so.

pgslice is a fairly straightforward tool to port. The code is relatively small (on the order of 1k LOC), easy to read, and well-documented. With today's modern LLM tooling, we can lean heavily on them to do the grunt work of the porting, after we lay a strong foundation.

Finally, one of the great things about Ruby is its flexibility, allowing us to re-use the existing tests as a regression suite to help guide the porting process.

Porting Process

The full commit history is available below. As you'll notice Claude Opus 4.5 was the primary model uses, and I ensured that whenever it was used I committed its results in entirety, with attribution, and its own summary of the changes. Changes that I made "the old fashioned way" are delineated by having no attribution (Co-authored-by).

I broke the porting process down into phases:

Research and Planning

First, I documented the porting philosophy into a hand-written CLAUDE.md. You can see where I started and ended, iterating as I found recurring errors in the agent-assisted changes. Important notes were things like how to run the Ruby tests, or which features to not port (more on that later).

Key dependencies were chosen by my own research. slonik due to my own awareness and that it presented a nicer interface around the low-level pg package, which we already use. clipanion due to its superficial likeness to thor from the Ruby world, in hopes of making command-porting more straightforward.

Lay the Foundation

Before letting the AI loose, I focused on laying the foundation of the package itself. This went through several iterations, but you can see examples:

Agent-assisted Command Porting

Finally, I used Claude to port each command, one-by-one in separate sessions, cleaning up and abstracting after each go. Examples:

As more commands were ported (and I continued to refactor in between), the agent-assisted porting got both faster and better.

Results

By the end, it was possible to run USE_TYPESCRIPT_PORT=1 rake test and have the entire suite pass, minus a few tests that were intentionally skipped.

Regression Test Output
$ USE_TYPESCRIPT_PORT=1 bundle exec rake test TESTOPTS="-v"
/Users/michael.hadley/projects/workos/pgslice/test/pgslice_test.rb:401: warning: ambiguity between regexp and two divisions: wrap regexp in parentheses or add a space after `/' operator
/Users/michael.hadley/projects/workos/pgslice/test/pgslice_test.rb:427: warning: ambiguity between regexp and two divisions: wrap regexp in parentheses or add a space after `/' operator
/Users/michael.hadley/projects/workos/pgslice/test/pgslice_test.rb:435: warning: ambiguity between regexp and two divisions: wrap regexp in parentheses or add a space after `/' operator
/Users/michael.hadley/projects/workos/pgslice/test/pgslice_test.rb:463: warning: ambiguity between regexp and two divisions: wrap regexp in parentheses or add a space after `/' operator
Run options: -v --seed 8197

# Running:

PgSliceTest#test_ulid_no_partition = 2.87 s = .
PgSliceTest#test_add_partitions_negative_past = 0.25 s = .
PgSliceTest#test_prep_no_partition_trigger_based = 0.04 s = S
PgSliceTest#test_prep_invalid_period = 0.12 s = .
PgSliceTest#test_unprep_missing_table = 0.14 s = .
PgSliceTest#test_month = 1.43 s = .
PgSliceTest#test_v2 = 0.03 s = S
PgSliceTest#test_unswap_missing_table = 0.14 s = .
PgSliceTest#test_synchronize_missing_table = 0.14 s = .
PgSliceTest#test_analyze_missing_table = 0.15 s = .
PgSliceTest#test_disable_mirroring_missing_table = 0.14 s = .
PgSliceTest#test_synchronize = 0.60 s = .
PgSliceTest#test_enable_mirroring_missing_intermediate_table = 0.15 s = .
PgSliceTest#test_add_partitions_missing_tablespace = 0.28 s = .
PgSliceTest#test_enable_retired_mirroring_missing_table = 0.15 s = .
PgSliceTest#test_day = 1.51 s = .
PgSliceTest#test_disable_retired_mirroring_missing_table = 0.15 s = .
PgSliceTest#test_enable_retired_mirroring = 0.95 s = .
PgSliceTest#test_add_partitions_missing_table = 0.15 s = .
PgSliceTest#test_prep_no_partition_extra_arguments = 0.14 s = .
PgSliceTest#test_tablespace = 1.44 s = .
PgSliceTest#test_prep_missing_table = 0.16 s = .
PgSliceTest#test_year = 1.43 s = .
PgSliceTest#test_fill_missing_table = 0.15 s = .
PgSliceTest#test_enable_mirroring = 0.28 s = .
PgSliceTest#test_no_partition = 0.98 s = .
PgSliceTest#test_disable_retired_mirroring = 1.09 s = .
PgSliceTest#test_mirroring_triggers_work = 0.42 s = .
PgSliceTest#test_timestamptz = 1.43 s = .
PgSliceTest#test_trigger_based = 0.03 s = S
PgSliceTest#test_prep_missing_column = 0.16 s = .
PgSliceTest#test_tablespace_trigger_based = 0.03 s = S
PgSliceTest#test_ulid_fill_swapped = 3.50 s = .
PgSliceTest#test_prep_missing_arguments = 0.14 s = .
PgSliceTest#test_ulid_fill_with_start = 2.60 s = .
PgSliceTest#test_add_partitions_negative_future = 0.24 s = .
PgSliceTest#test_trigger_based_timestamptz = 0.04 s = S
PgSliceTest#test_disable_mirroring = 0.40 s = .
PgSliceTest#test_ulid_partitioned = 2.84 s = .
PgSliceTest#test_swap_missing_table = 0.15 s = .
PgSliceTest#test_swap_creates_retired_mirroring_trigger = 0.71 s = .
PgSliceTest#test_date = 1.45 s = .
PgSliceTest#test_enable_mirroring_missing_table = 0.14 s = .
PgSliceTest#test_add_partitions_non_partitioned_table = 0.15 s = .
PgSliceTest#test_ulid_numeric_still_works = 0.57 s = .

Fabulous run in 30.064246s, 1.4968 runs/s, 17.1965 assertions/s.

  1) Skipped:
PgSliceTest#test_prep_no_partition_trigger_based [test/pgslice_test.rb:101]:
TypeScript port doesn't support trigger-based partitioning

  2) Skipped:
PgSliceTest#test_v2 [test/pgslice_test.rb:67]:
TypeScript port doesn't support v2 partitioning

  3) Skipped:
PgSliceTest#test_trigger_based [test/pgslice_test.rb:57]:
TypeScript port doesn't support trigger-based partitioning

  4) Skipped:
PgSliceTest#test_tablespace_trigger_based [test/pgslice_test.rb:76]:
TypeScript port doesn't support trigger-based partitioning

  5) Skipped:
PgSliceTest#test_trigger_based_timestamptz [test/pgslice_test.rb:62]:
TypeScript port doesn't support trigger-based partitioning

45 runs, 517 assertions, 0 failures, 0 errors, 5 skips

Test coverage is generally more extensive and fine-grained than the original Ruby port.

Differences

As alluded to earlier, there are some differences between the Ruby and TypeScript versions, given the port is targeted at WorkOS needs.

  • The port does not support "trigger-based" partitioning. WorkOS uses modern versions of Postgres that support "declarative" partitioning.
  • The port does not support "v2" of pgslice, meaning it cannot be used against tables that were partitioned by older versions of pgslice.
  • The port does not support compound primary keys. The original Ruby version had some support, but not all commands handled them, and some seemed to even have bugs (only factoring in part of a compound key). WorkOS does not need compound key support, so it was removed.
  • Various command line flags were removed. These were untested (e.g. fill --where "some_condition") and we weren't planning on using anyways.

mthadley and others added 30 commits January 16, 2026 14:49
Seems like an easier way to set `PGSLICE_URL` when running tests
Sets up the foundation for the TypeScript port with:
- `package.json` with slonik, clipanion, and vitest dependencies
- `tsconfig.json` with strict mode and ESM output
- Basic CLI entrypoint with a placeholder `hello` command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When USE_TYPESCRIPT_PORT=1 is set, the run_command helper now invokes
the TypeScript CLI via subprocess instead of calling PgSlice::CLI
directly. This allows the Ruby test suite to serve as a regression test
for the TypeScript port.

---

Conversation summary:

User: Lets update the Ruby tests to check for the presence of an
Environment variable (maybe `USE_TYPESCRIPT_PORT), that when present,
causes the `run_command` helper to invoke the TypeScript CLI via a
subprocess, rather than using `PgSlice::CLI`. We haven't implemented any
of the commands yet so we should expect the tests to fail, but that's
okay. Once this shim is in place we can start the porting process.

Claude: Explored the Ruby test structure (test/pgslice_test.rb and
test/test_helper.rb) and the TypeScript CLI setup (bin/pgslice.ts,
dist/bin/pgslice.js). Created a plan to:

1. Add `require "open3"` and `use_typescript_port?` helper to
   test_helper.rb
2. Modify `run_command` in pgslice_test.rb to conditionally use
   Open3.capture3 to invoke the TypeScript CLI when the env var is set

Verified that:
- Without the env var: All 45 Ruby tests pass (730 assertions)
- With USE_TYPESCRIPT_PORT=1: Tests fail as expected since TypeScript
  commands aren't implemented yet

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
slonik v47 is CommonJS but depends on p-limit v6+ which is ESM-only,
causing "ERR_REQUIRE_ESM" errors on Node 20. slonik v46 doesn't have
this dependency issue.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Match the Ruby version's behavior where the database URL can be provided
via --url flag or PGSLICE_URL environment variable. This makes both
--url and --dry-run inherited global options for all commands.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Set up test infrastructure using vitest's test.extend() to provide
fixtures that wrap tests in transactions which automatically rollback,
ensuring test isolation.

- Add vitest.config.ts with .test.env loading
- Add src/testing/db.ts with pgslice and connection fixtures
- Add src/testing/index.ts for re-exports
- Update HelloCommand to run actual database statements
- Add hello.test.ts verifying table creation and rollback isolation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This implements the first real pgslice command in TypeScript. The prep
command creates an intermediate table for partitioning, which is the
first step in pgslice's workflow.

Key implementation details:
- Only version 3 (native Postgres partitioning) is supported
- Uses slonik for database operations with sql.identifier for safe DDL
- Dynamic DDL (indexes, FKs) executed via pg_temp.pgslice_execute_ddl()
  helper function to work around slonik's security restrictions
- Proper error handling with clean error messages (no stack traces)

New files:
- src/types.ts - Shared type definitions
- src/table.ts - Table utility functions
- src/commands/prep.ts - CLI command
- src/commands/prep.test.ts - 12 comprehensive tests

Test results:
- TypeScript tests: 13/13 passing
- Ruby regression: 1 expected failure (trigger-based not supported)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Convert loose functions in table.ts to a Table class that mirrors the Ruby
implementation. This provides a cleaner API where table operations are methods
on Table instances rather than standalone functions taking TableRef objects.

Changes:
- Add Table class with static parse() factory and instance methods
- Remove TableRef interface from types.ts
- Update pgslice.ts to use Table class instead of standalone functions
- Export Table class from index.ts for module consumers

The Table class includes:
- Derived tables: intermediate(), retired(), triggerName
- SQL generation: toSqlIdentifier(), toQuotedString(), toString()
- Async queries: exists(), columns(), columnCast(), indexDefs(), foreignKeys()

Standalone functions sqlIdent() and getServerVersionNum() remain as they don't
operate on tables.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Slonik recommends against using `sql.unsafe` in production code. This
replaces all usages in non-test files with `sql.type(z.object({}))`,
which works for DDL statements since the empty object schema only
validates returned rows (DDL doesn't return any).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove the `ensureDDLExecutor` and `executeDynamicDDL` functions that
created a temporary PL/pgSQL function to execute dynamic DDL. Instead,
use `sql.unsafe` directly with a properly constructed template strings
array (with a frozen `raw` property to satisfy slonik's validation).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
mthadley and others added 27 commits January 25, 2026 19:06
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- toSqlIdentifier() -> get sqlIdentifier
- toRegclassLiteral() -> get regclassLiteral
- toQuotedString() -> get quoted
- intermediate() -> get intermediate
- retired() -> get retired

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This command reverses the prep command by dropping the intermediate
table with CASCADE, removing any dependent objects like partitions.

Since TypeScript version only supports native Postgres partitioning
(not trigger-based), we skip the trigger function cleanup that the
Ruby version performs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…to TypeScript

- Refactor Mirroring class: rename `mode` to `targetType` for clarity
- Add optional `targetType` parameter to enableMirroring/disableMirroring
- Create EnableRetiredMirroringCommand and DisableRetiredMirroringCommand
- Add comprehensive tests for both new commands
- Update Ruby tests to check stdout instead of stderr for consistency

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
We have no historical usage of v2 so this won't be carried over.
Previously, identical JSON objects were triggering unnecessary UPDATE
operations because #rowsDiffer used strict equality (===) which fails
for objects with different reference identities. Now uses JSON.stringify
for deep comparison when either value is an object.

Also adds support for json/jsonb data types in valueToSql using slonik's
sql.json() and sql.jsonb() helpers.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There were some commands that played nice with compound primary keys,
like `prep` and `mirroring`, but others less so, like `fill` and
`synchronize`.

Since we don't use compound primary keys (all rows have an explicit
single column primary key, and more so, are all named `id`), I think we
should just remove this code and label it as unsupported.
Remove the Ruby gem release workflow and update the test workflow to
only test the TypeScript version against Postgres 13, 15, 17, and 18.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The TypeScript port is complete, so remove all Ruby-related files:

- Gemfile, Rakefile, pgslice.gemspec, Dockerfile
- lib/ (Ruby source code)
- exe/ (Ruby CLI entrypoint)
- test/ (Ruby test suite)
- docs/ (gem release documentation)

Also cleaned up .gitignore to remove Ruby-specific patterns.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove Ruby references and porting-related language now that the
TypeScript port is complete.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The tests require this environment variable to connect to the database.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Run build, format:check, and typecheck once in a separate lint job
instead of repeating them for each Postgres version. Tests now depend
on the lint job passing first.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@mthadley mthadley merged commit 1a9a9af into master Feb 5, 2026
7 checks passed
@mthadley mthadley deleted the port-to-typescript branch February 5, 2026 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants