-
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Description
🙋 feature request
Add optional fsync support to @parcel/fs-write-stream-atomic to improve durability when using atomic writes with streams.
🤔 Expected Behavior
When the stream finishes writing the temp file, the library should be able to:
fsyncthe temp file to ensure data is on disk.renamethe temp file to the target path.- Optionally
fsyncthe parent directory to persist the name/metadata.
This could be opt-in via options, e.g.:
writeStreamAtomic(dest, { fsync: true, fsyncDir: true });😯 Current Behavior
The library writes to a temp file and then renames it to the target, but does not call fsync on the temp file or the parent directory. After a crash or power loss right after rename, the file contents may not be fully durable.
💁 Possible Solution
-
Add options to enable
fsync:- On stream close: call
fdatasync/fsyncon the temp file beforerename. - After
rename: attempt to open andfsyncthe parent directory (best-effort; skip on platforms/filesystems that don’t support it).
- On stream close: call
-
Keep the default behavior unchanged for performance, but allow users to opt in per use case.
🔦 Context
We use the package to copy/write large files via streams and need stronger guarantees against data loss on sudden crashes. Atomic rename protects the filename switch, but without fsync we can still end up with a truncated or not-yet-flushed file after a power failure.
💻 Examples
Proposed usage:
import writeStreamAtomic from '@parcel/fs-write-stream-atomic';
const ws = writeStreamAtomic('/path/to/file', { fsync: true, fsyncDir: true });
// pipe(...).on('finish', ...)Thanks for maintaining this package - it’s very helpful for me.
If this proposal makes sense, I’d be happy to open a PR implementing it.