perf: JSON-RPC: faster eventLoop: request buffer

[WofWca]

Before this commit we would not start another getNextEvent()
before we have received a response for the previous one.
This commit makes a pool of getNextEvent() requests
so that the server always has a request
that it can readily respond to.

Measurements

I have tested this on Delta Chat desktop.
This seems to provide a measurable speedup in handling event bursts.

I did console.time() when we start
a BackendRemote.rpc.maybeNetwork() request
(which we do every time the main window is focused),
and a console.timeEnd() when we receive the first
"IDLE entering wait-on-remote state" entry, inside of
ipcBackend.on('json-rpc-message'.
For these measurements I also disabled request-response logging
with config['log-debug'] = false.

Each such maybeNetwork() resulted in ~1000 getNextEvent() responses.

With the original event loop (without a pool) the average time
based on 150 measurements was 1152.28 ms.
With the new event loop with a pool of 20
based on 150 measurements it was 774.58 ms.

That is 67.22% of the original average duration.

TODO:

• Please check that this doesn't break stuff.
  I heard that there was some issue with Delta Chat bots
  trying to not miss a single event.

  If that's the case, or if in doubt, we can set
  the default pool size to 1 instead of 20,
  and only set it to 20 in Delta Chat Desktop.

Related:

• perf: stop handling JSON-RPC Info events deltachat/deltachat-desktop#5282

[link2xt]

I'm not sure what exactly the problem is and how this pooling improves performance. We already have concurrent calls working on the JSON-RPC level, tested with a Python test. Maybe there are some large responses and just reading them blocks reception of events?

The other option is adding a batched API that fetches multiple events at once (up to some limit) and returns them all at once. This way as soon as you get an event call response, you get all events that were in the queue at the moment of the call processing and not have to wait for every other response to be read. Getting all events at once makes it easier to deduplicate them, e.g. if you have 10 events in the queue about some chat being modified, you are only interested in the last one.

[WofWca]

I'm not sure what exactly the problem is and how this pooling improves performance.

IDK, but it just makes sense that there is a non-zero delay between the getNextEvent() call on the client and the time the Core receives that request. This is what this pooling eliminates. The Core now doesn't have to wait to receive a get_next_event request before it can send a response.

[link2xt]

IDK, but it just makes sense that there is a non-zero delay between the getNextEvent() call on the client and the time the Core receives that request.

I doubt there is any significant delay in passing the event from one process to another via stdio even with JSON serialization. More likely the difference comes from different scheduling, if you get events faster than heavy API responses and have some debouncing logic in desktop, you may do less work.