Skip to main content

Server: N3 Cluster

feat~srv.clusters.n3~1

N3 is a 3-node cluster.

Needs: adsn, story

Motivation​

The need for new Standard Edition architecture arose from challenges encountered in the previous architecture:

  • Peer-to-peer cluster implementation revealed load distribution issues:
    • In a 5-node setup with one application server and 3 database servers, one application node remained idle
    • In a 3-node setup with one application server and 3 database servers, uneven load distribution led to potential node overload
  • The initiative to Design "peer nodes" ctool principles was unsuccessful

Server​

N3 Cluster implements:

  • A 3-node peer-to-peer cluster
  • 3 routers
  • 6 VVMs (fixed configuration, not modifiable currently)
  • Clean Ubuntu nodes as a requirement
    • This decision aims to minimize software conflicts and reduce operational costs
  • grafana 8.3.4 becasue of reverse proxy problem

Nodes​

Swarm stacks​

  • grafana always work with local prometheus task (instance that runs on the same node)
    • Each host has its own grafana configuration
  • router is one service three instances
    • Three services configuration leads to port conflicts.
  • Networks
    • "voedger.net" 10.10.0.0
    • Scylla: host mode network

Alternative configurations​

Server: N3: Load Balancer​

adsn~srv.clusters.n3.load-balancer~1

The system’s load balancing layer must be provided by a cloud-managed load balancer solution, such as Amazon Elastic Load Balancer, Google Cloud Load Balancer, or Hetzner Load Balancer.

Covers:

  • feat~srv.clusters.n3~1

Server: N3: Swarm​

adsn~srv.clusters.n3.swarm~1

The system uses Docker Swarm for orchestration.

Covers:

  • feat~srv.clusters.n3~1

Needs: impl

Server: N3: Swarm: All Managers​

adsn~srv.clusters.n3.swarm.allmgrs~1

All nodes function as managers.

Covers:

  • feat~srv.clusters.n3~1

Needs: impl

clusters.n3.monitoring​

  • Monitoring:

    • 3 Prometheus instances
    • 3 Grafana instances

  • Database:

    • DBMS: Scylla
    • Scylla cluster configuration:
      • Physical deployment: One or three datacenters
      • Logical configuration: ??? Always maintains two datacenters (Scylla configuration)

  • Routing implementation:

    • Router task runs on each node
    • Current solution uses Voedger image with specific CLI options for routing
    • Note: The possibility of using Nginx is being considered, but the implementation has been postponed due to the complexity of development and maintenance

Covers:

  • feat~srv.clusters.n3~1