feat: Implement Primitive Writer for Streaming and Chunked ZTDF Creation

[jrschumacher]

Problem
The Java SDK currently lacks support for creating ZTDFs via streaming or chunked writes. This requires buffering entire files in memory during encryption, which is inefficient for large-scale data processing. This limitation prevents its use in memory-constrained environments and complicates integrations with cloud services that rely on chunked or streamed data uploads.

User Story
As a Java developer, I want to use a primitive writer interface to construct a ZTDF file. This interface should allow me to write encrypted data segments in any order and then finalize the TDF by generating the manifest and wrapping keys. This would enable me to build efficient, low-memory applications for encrypting and streaming large files.

Misc
This work should align with the primitive interface being introduced in the core Go platform. The goal is to expose a low-level writer that provides flexibility for chunked I/O without compromising security.

To prevent misuse, core cryptographic and policy controls should be abstracted away and remain non-configurable through this interface. Any highly privileged controls that must be exposed should be designated for internal Virtru use and excluded from public documentation.

Acceptance Criteria

Functional Requirements

• A primitive writer interface, consistent with the platforms design, is implemented in the Java SDK for ZTDF creation.
• The interface supports writing payload segments in a non-sequential order.
• A finalization method is provided to complete the TDF creation by writing the manifest and closing the archive.
• The generated ZTDF files are well-formed and can be decrypted by other TDF SDKs.
• It is possible to calculate the final TDF size ahead of its creation.

Non-Functional Requirements

• Performance: The writer should handle data as streams to avoid high memory consumption.
• Memory: The memory footprint during TDF creation should remain low and not be dependent on the total file size.
• Security: The public API must not expose controls that could compromise the integrity of the TDFs encryption or policy.
• Documentation: The public-facing API should be clearly documented, while any internal-only methods are omitted from the public documentation.

[mkleene]

I think that the interface that we have already satisfies this:

java-sdk/sdk/src/main/java/io/opentdf/platform/sdk/SDK.java

Lines 109 to 112 in 95c20b3

	public Manifest createTDF(InputStream payload, OutputStream outputStream, Config.TDFConfig config) throws SDKException, IOException {
	var tdf = new TDF(services);
	return tdf.createTDF(payload, outputStream, config).getManifest();
	}

, except for:

• being able to calculate the size of the ZTDF ahead of creation
• not having a finalization method (but to me this seems extremely similar to closing a stream, which is how we finish things now)
• being able to write segments out of order
  • I don't fully understand this one; do we want to write segments in one order and then read them in another? I'd be interested in the use case here since the segment is a low-level TDF thing that seems unrelated to what people would want to do at an application level.

[jrschumacher]

being able to calculate the size of the ZTDF ahead of creation

This might only be required for specific edge-cases. If the inbound data segment size is incongruent with the encrypted segment size there could be buffer overflow when dealing with out-of-order inbound data.

not having a finalization method (but to me this seems extremely similar to closing a stream, which is how we finish things now)

This makes sense. It might be overkill, but just expressing that as we are writing the encrypted bits we want to make sure to add attributes and wrap the key at some future state given that we might be waiting on an async event to determine the attributes.

being able to write segments out of order

It's more focused on the inbound stream being out of order. Consider some bespoke, legacy client that has intelligence on transmitting data in the most efficient way and in parallel. We also have a limited amount of free memory so we cannot buffer until all the data is available (in order).