Type usages of Uint8Array as Uint8Array<ArrayBuffer>

[peter-cardenas-ai]

Describe the bug
With the most recent version of TypeScript, there was a change to make the type of Uint8Array such that the buffer can be either ArrayBuffer or SharedArrayBuffer. While controversial, it seems as though this change will stick. Thus, to minimize burden on usages of this package and from a cursory look at the implementation, it seems as though we should change our usages of Uint8Array to Uint8Array<ArrayBuffer>.

To Reproduce
Steps to reproduce the behavior:
See how this causes a type error in TypeScript 5.9+

message Test {
  bytes test = 1;
}

const response = await rpcClient.method(...);
new Blob([response.response.test]); // type error

Expected behavior
Should not have a type error

Environment:

• TypeScript 5.9+

[jcready]

It's not totally clear to me that protobuf-ts should prevent decoding of typed arrays with SharedArrayBuffer backing them. FWIW protobuf-es has a nearly identical issue open and protobuf-ts would likely want to follow its lead.

Some Background

Part of why protobuf-ts decoding is so fast is because it only creates typed array views over the original Uint8Array's backing buffer rather than creating entirely new ArrayBuffer instances with the bytes copied over: 58b774b

While this approach is fast it's not entirely without faults. For one it cause the entire Uint8Array backing buffer to stay in memory for the lifetime of any byte or unknown fields. And since it only creates "views" over the underlying buffer it means that if the underlying buffer's bytes are mutated, it can corrupt those "views" as well.

const input = new Uint8Array(8); // Uint8Array(8) [ 0, 0, 0, 0, 0, 0, 0, 0 ]
const view = input.subarray(4); // Uint8Array(4) [ 0, 0, 0, 0 ]
input.set([1, 2, 3, 4], 4);
view; // Uint8Array(4) [ 1, 2, 3, 4 ] - Yikes

I bring these issues up to point out that these issues already exist for ArrayBuffer-backed Uint8Arrays and thus those same issues also would exist for ones backed by SharedArrayBuffers. The only difference is that the SharedArrayBuffer could have its bytes mutated in another "thread".

Updating Just the Return Types

If protobuf-ts was to change JUST the return type of things from Uint8Array to Uint8Array<ArrayBuffer> (without any change to the actual runtime code) and you did something like:

const mySharedArrayBuffer = new SharedArrayBuffer(1024);
// ...populate the shared array buffer somehow

const myTestMessage = Test.fromBinary(new Uint8Array(mySharedArrayBuffer));
new Blob([myTestMessage.test]);

Then you'd get no type error, but you would get a runtime error.

Possible Solutions

As far as I can see the only way to resolve that issue would be one of these options:

1. Only allow Uint8Array<ArrayBuffer> inputs which seems limiting as there's really no technical reason to impose such a limitation given that protobuf-ts already accepts the risk of the input buffer having its bytes changed and potentially corrupting any byte or unknown fields.
2. In the case that the input Uint8Array was backed by a SharedArrayBuffer: copy its bytes to a new ArrayBuffer-backed Uint8Array. Doubles the amount of memory required to deserialize.
3. In the case that the BinaryReader's buf was backed by a SharedArrayBuffer have its skip() and bytes() methods copy the memory into a new ArrayBuffer-backed Uint8Array. Less memory overhead compared to Regression: Custom field options are missing from reflection data #2, but likely slower as it requires creating several individual chunks of memory as the fields are being deserialized vs. creating a single larger chunk of memory up front.
4. Make our interfaces/classes generic such that they take a type parameter that extends (and defaults to) Uint8Array and then propagate the correct/narrow version (ArrayBuffer-backed) whenever possible. E.g. have rpc methods forcefully use the Uint8Array<ArrayBuffer> version. This may not be possible to do in a safe way that also avoids breaking backwards type compatibility.
5. Do nothing and force the burden of manual type casting (and potential runtime issues) onto the user.

[peter-cardenas-ai]

for the grpcweb transport at least it seems like an ArrayBuffer is backing it. So maybe a generic defaulting to ArrayBufferLike (the current value) is in order and particular transports could possibly enforce ArrayBuffer.

another note, the reason why this change is controversial is that adding a generic to Uint8Array is not backwards compatible with versions earlier than 5.7. there are ways around it but something to keep in mind