Skip to content

Add optional fsync support for fs-write-stream-atomic #10235

@Adi1231234

Description

@Adi1231234

🙋 feature request

Add optional fsync support to @parcel/fs-write-stream-atomic to improve durability when using atomic writes with streams.

🤔 Expected Behavior

When the stream finishes writing the temp file, the library should be able to:

  1. fsync the temp file to ensure data is on disk.
  2. rename the temp file to the target path.
  3. Optionally fsync the parent directory to persist the name/metadata.

This could be opt-in via options, e.g.:

writeStreamAtomic(dest, { fsync: true, fsyncDir: true });

😯 Current Behavior

The library writes to a temp file and then renames it to the target, but does not call fsync on the temp file or the parent directory. After a crash or power loss right after rename, the file contents may not be fully durable.

💁 Possible Solution

  • Add options to enable fsync:

    • On stream close: call fdatasync/fsync on the temp file before rename.
    • After rename: attempt to open and fsync the parent directory (best-effort; skip on platforms/filesystems that don’t support it).
  • Keep the default behavior unchanged for performance, but allow users to opt in per use case.

🔦 Context

We use the package to copy/write large files via streams and need stronger guarantees against data loss on sudden crashes. Atomic rename protects the filename switch, but without fsync we can still end up with a truncated or not-yet-flushed file after a power failure.

💻 Examples

Proposed usage:

import writeStreamAtomic from '@parcel/fs-write-stream-atomic';

const ws = writeStreamAtomic('/path/to/file', { fsync: true, fsyncDir: true });
// pipe(...).on('finish', ...)

Thanks for maintaining this package - it’s very helpful for me.
If this proposal makes sense, I’d be happy to open a PR implementing it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions