-
Notifications
You must be signed in to change notification settings - Fork 27
Open
Labels
Description
Currently we send every work into a single processor queue. This is not ideal, as it puts messages we need to act on (messages for clusters we are part of) on the same level as messages we only want to relay (other messages in the subnet).
This is not ideal. Actually, as we currently queue messages that are relevant to us again after processing (in the same queue), we actively decide to first handle potentially irrelevant messages before going on with processing messages that are relevant to us:
Rx = relevant message validation
Ox = other message validation
Px = processing of a validated relevant message
| Position | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|
| t = 1 | R1 | O2 | O3 | O4 | O5 | R6 | O7 |
| t = 2 | O2 | O3 | O4 | O5 | R6 | O7 | P1 |
| t = 3 | O3 | O4 | O5 | R6 | O7 | P1 | |
| ... | |||||||
| t = 6 | R6 | O7 | P1 | ||||
| t = 7 | O7 | P1 | P2 | ||||
| t = 8 | P1 | P2 |
We should instead:
- either
- immediately process relevant messages after validating them (as part of the same job)
- or create a priority queue for them
- consider whether we want to quickly determine ASAP whether a message is relevant, and actively deprioritize irrelevant messages.