fix: hydrate symlinks at runtime when postinstall is skipped#27
Open
adelin-b wants to merge 1 commit intoleinelissen:mainfrom
Open
fix: hydrate symlinks at runtime when postinstall is skipped#27adelin-b wants to merge 1 commit intoleinelissen:mainfrom
adelin-b wants to merge 1 commit intoleinelissen:mainfrom
Conversation
npm does not preserve symlinks when publishing packages. The platform packages include pg-symlinks.json and a postinstall script to restore them. However, tools like bunx, pnpm with --ignore-scripts, and yarn PnP skip lifecycle scripts, leaving the dylib/so symlinks missing. This causes postgres binaries to crash at startup with dyld/ld errors like "Library not loaded: libzstd.1.dylib". This commit adds runtime symlink hydration in getBinaries() — after importing the platform package, it checks pg-symlinks.json and creates any missing symlinks before returning the binary paths. The check is idempotent and silently skips already-existing symlinks or read-only filesystems. Fixes the same class of issue as leinelissen#21 (Linux) but for all platforms.
Owner
|
Hi @adelin-b. This looks promising! Thanks for your contribution. Could you perhaps add some tests that verify that the symlinks are correctly applied at runtime? I would also really appreciate it if we could also run the the tests for bun and Deno in CI to see if we run into any issues into the future. Let me know if that could work out for you :) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
npm does not preserve symlinks when publishing packages. The platform packages (e.g.
@embedded-postgres/darwin-arm64) ship apg-symlinks.jsonmanifest and apostinstallscript (hydrate-symlinks.js) to restore symlinks after installation.However, several common tools and configurations skip lifecycle scripts, leaving the dylib/so symlinks missing:
bunx— always skips postinstallpnpmwith--ignore-scriptsyarnPnP — doesn't run postinstall in certain configurationsnpmwith--ignore-scriptsWithout these symlinks, the postgres binaries crash at startup with errors like:
This is the same class of issue as #21 (Linux shared libraries), but affecting all platforms where symlinks exist in the native directory.
Solution
Add runtime symlink hydration in
getBinaries()(packages/embedded-postgres/src/binary.ts). After importing the platform package, it:pg-symlinks.jsonrelative to the binary pathlstathydrate-symlinks.jsbehavior)The check is:
lstatcalls that hit the fs cacheThe existing
postinstallscript is kept as-is for the fast path when lifecycle scripts do run.Test plan
bunx paperclipai@latest(which usesembedded-postgresinternally)initdbandpostgrescrash with dyld missing library errorspg-symlinks.jsonare hydrated and postgres starts successfully"true