Skip to content

OAK-12325: Use heap based comparison of blob content#3045

Merged
nfsantos merged 1 commit into
apache:trunkfrom
nfsantos:OAK-12325
Jul 21, 2026
Merged

OAK-12325: Use heap based comparison of blob content#3045
nfsantos merged 1 commit into
apache:trunkfrom
nfsantos:OAK-12325

Conversation

@nfsantos

@nfsantos nfsantos commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Jira: https://issues.apache.org/jira/browse/OAK-12325

Problem

AbstractBlob.equal()'s byte-wise fallback comparison (IOUtils.contentEquals(InputStream, InputStream)) allocates two direct ByteBuffers per call when the underlying streams are file-backed (e.g. FileChannel-based NIO copies). This comparison runs on hot paths such as the segment writer's dedup check when writing an unchanged property and node-state diffing (rebase, commit diff). Under write-heavy load this becomes a significant source of native (off-heap) memory allocation — direct buffers are reclaimed only via GC-driven Cleaner/PhantomReference callbacks, which can lag behind allocation and drive up process RSS outside the JVM heap.

Summary

  • Replace IOUtils.contentEquals in AbstractBlob.equal() with a manual buffered comparison using InputStream.readNBytes and Arrays.equals, working entirely with heap byte[] buffers that are reclaimed by ordinary GC, avoiding the commons-io dependency on this hot path.
  • Also close both input streams via try-with-resources, fixing a related stream leak in the previous implementation.

Replace IOUtils.contentEquals in AbstractBlob.equal() with a manual
buffered comparison using InputStream.readNBytes and Arrays.equals,
avoiding the commons-io dependency for this hot path.
@nfsantos
nfsantos merged commit 44eff55 into apache:trunk Jul 21, 2026
3 of 5 checks passed
@nfsantos
nfsantos deleted the OAK-12325 branch July 21, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants