N3 Cluster

feat~srv.clusters.n3~1

N3 is a 3-node cluster.

Needs: adsn, story

Motivation

The need for new Standard Edition architecture arose from challenges encountered in the previous architecture:

• Peer-to-peer cluster implementation revealed load distribution issues:

  • In a 5-node setup with one application server and 3 database servers, one application node remained idle

  • In a 3-node setup with one application server and 3 database servers, uneven load distribution led to potential node overload

• The initiative to Design "peer nodes" ctool principles was unsuccessful

Server

N3 Cluster implements:

• A 3-node peer-to-peer cluster

• 3 routers

• 6 VVMs (fixed configuration, not modifiable currently)

• Clean Ubuntu nodes as a requirement

  • This decision aims to minimize software conflicts and reduce operational costs

• grafana 8.3.4 becasue of reverse proxy problem

Nodes

Swarm stacks

• grafana always work with local prometheus task (instance that runs on the same node)

  • Each host has its own grafana configuration

• router is one service three instances

  • Three services configuration leads to port conflicts.

• Networks

  • "voedger.net" 10.10.0.0

  • Scylla: host mode network

Alternative configurations

Server: N3: Load Balancer

adsn~srv.clusters.n3.load-balancer~1

The system’s load balancing layer must be provided by a cloud-managed load balancer solution, such as Amazon Elastic Load Balancer, Google Cloud Load Balancer, or Hetzner Load Balancer.

Covers:

• feat~srv.clusters.n3~1

Server: N3: Swarm

adsn~srv.clusters.n3.swarm~1

The system uses Docker Swarm for orchestration.

Covers:

• feat~srv.clusters.n3~1

Needs: impl

Server: N3: Swarm: All Managers

adsn~srv.clusters.n3.swarm.allmgrs~1

All nodes function as managers.

Covers:

• feat~srv.clusters.n3~1

Needs: impl

clusters.n3.monitoring

• Monitoring:

  • 3 Prometheus instances

  • 3 Grafana instances

• Database:

  • DBMS: Scylla

  • Scylla cluster configuration:

    • Physical deployment: One or three datacenters

    • Logical configuration: ??? Always maintains two datacenters (Scylla configuration)

• Routing implementation:

  • Router task runs on each node

  • Current solution uses Voedger image with specific CLI options for routing

  • Note: The possibility of using Nginx is being considered, but the implementation has been postponed due to the complexity of development and maintenance

Covers:

• feat~srv.clusters.n3~1