Introduce cluster support #830
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduce TypeDB Cluster support. Including:
Drivers can be created using a single address, multiple addresses, or
address translation (mapping between public user-facing addresses and
internal cluster addresses for special needs). Address translation can
be updated after the driver is created.
Server version and distribution, as well as the list of active replicas,
can be retrieved using TypeDB Driver.
Read operations can be executed using one of three strategies:
* Strong consistency level -- Strongest consistency, always up-to-date
due to the guarantee of the primary replica usage.
* Eventual consistency level -- Allow stale reads from any replica. May
not reflect latest writes (will not be supported in the first Cluster
release on the server side, so will probably return errors)
* Replica dependent consistency level -- The operation is executed
against the provided replica address only. Can be especially useful for
testing.
By default, all operations are executed using the strongest consistency
level. However, all read operations now support consistency level
specification. Read transactions can be configured through transaction
options.
Driver options can be used to configure the failover strategies used
with strong and eventual consistency levels.
`primary_failover_retries` -- Limits the number of attempts to redirect
a strongly consistent request to another primary replica in case of a
failure due to the change of replica roles. Defaults to 1.
`replica_discovery_attempts` -- Limits the number of driver attempts to
discover a single working replica to perform an operation in case of a
replica unavailability. Every replica is tested once, which means that
at most:
- {limit} operations are performed if the limit <= the number of
replicas.
- {number of replicas} operations are performed if the limit > the
number of replicas.
- {number of replicas} operations are performed if the limit is None.
Affects every eventually consistent operation, including redirect
failover, when the new primary replica is unknown. Defaults to None.
**Missing parts**:
* Address translation runtime update is not fully implemented, it
probably requires more updates, or can be removed for the first release.
* Testing cannot be automated without Cluster snapshots. Instead, for
local testing, use `temp-cluster-server` (instructions below).
* The driver works itself, the tests for Core pass. However, its
replication features are untested. There is a new `clustering`
integration test, which successfully boots up servers, but there are
some missing functionalities of the server that prevent the testing
progress. Once the PR referenced below is completed and merged, it can
be completed, as well. More BDD tests should be added here
typedb/typedb-behaviour#373
This driver can be tested using the server from
typedb/typedb-cluster#608. To do so, run `cargo
build` in `typedb-cluster`, and then copy the resulting binary to
`typedb-driver` like:
```
cp path-to-typedb/typedb-cluster/target/debug/typedb_server_bin path-to-typedb/typedb-driver/tool/test/temp-cluster-server/typedb
```
Then, test scripts like
`./tool/test/temp-cluster-server/start-cluster-servers.sh 3` can be
used.
Note that it is a temporary directory, created to use the server while
we are not able to use Cluster snapshots from Bazel.
farost
force-pushed
the
cluster-support-feature-branch
branch
from
ec84221 to
a804da7
Compare
## Usage and product changes Allow using `http` and `https` schemes in connection URLs, introducing a temporary comparison of the specified scheme and the TLS driver option. This is a temporary resolution until we decide how to make the GRPC endpoint safe and easy to understand, yet comfortable for Cloud and Cluster. ## Implementation Additionally, introduce more Java interfaces as a part of its build fix.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Usage and product changes
TODO
Implementation
TODO