723b1b8c05
A new Hetzner account is capped at 10 primary IPs (IPv4 and IPv6 both count), 20 shared vCPUs, 8 dedicated vCPUs and no Arm, and a network cannot span zones. So: one private network per zone (10.20.<zone>.0/24), the lowest-index node of each zone keeps a public IPv4 and is its gateway (NAT, MSS clamp, one DNAT port 27000+index per private node, persisted as igneum-nat.service), every other node has no public address. nodes.tsv gains access, pub and port columns; lib resolves same-zone vs cross-zone dial addresses and jumps ssh through the gateway for private nodes. All nodes.tsv loops read on fd 3 (a backgrounded ssh drained the file). TYPE_BY_INDEX puts nodes 8 to 11 on ccx13; node 12 is the last shared one the account allows. create.sh prints the plan's cost from the live API. provision.sh install takes node names and skips binaries whose sha256 matches. Binaries copied from the seed's staged v4 build (same sources), no vCPU for a builder. Results 2026-10-04: RTT matrix (hel1-fsn1 35 ms, ash-sin 289 ms) and a 10-minute propagation window of 644 blocks: p50 343 ms, p90 497 ms, p99 666 ms across 12 nodes in 5 locations. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
76 lines
5.9 KiB
Bash
Executable file
76 lines
5.9 KiB
Bash
Executable file
# Igneum cloud devnet: settings. Sourced by every script in this directory through lib/common.sh.
|
|
# Override any value in the environment, for example `N=10 PROVIDER=digitalocean ./create.sh`.
|
|
# Nothing here spends money by itself; create.sh asks before it creates anything.
|
|
|
|
PROVIDER="${PROVIDER:-hetzner}" # hetzner (hcloud CLI) or digitalocean (doctl CLI)
|
|
# 4 Oct 2026: a new Hetzner account is capped at 20 shared vCPUs (the seed node takes 2), 8 dedicated vCPUs, no Arm
|
|
# ("unsupported location for server type" for every cax type) and 10 primary IPs. 12 nodes is the most that fits:
|
|
# 8 shared (cx23/cpx21/cpx22) plus 4 dedicated (ccx13). N=20 needs a limit increase (console: Limits, "shared vCPU").
|
|
N="${N:-12}" # node count
|
|
PREFIX="${PREFIX:-igneum}" # server names igneum-01 .. igneum-20, the builder is igneum-builder
|
|
|
|
# Regions, round-robin over the node index. Hetzner locations: hel1 Helsinki, fsn1 Falkenstein, ash Ashburn (US east),
|
|
# hil Hillsboro (US west), sin Singapore. With 20 nodes that is 4 per location: 8 in the EU, 4 US east, 4 US west, 4 Asia.
|
|
REGIONS="${REGIONS:-hel1,fsn1,ash,hil,sin}"
|
|
DO_REGIONS="${DO_REGIONS:-lon1,fra1,nyc3,sfo3,sgp1}"
|
|
|
|
# Instance size per location. The node holds up to four 256 MiB lottery caches (M15 cap), rocksdb and the CPU
|
|
# miner's own 256 MiB cache, so 2 GB plans are too small. What this Hetzner project can create (API, 3 Oct 2026, net
|
|
# USD per month): cx23 (2 Intel vCPU, 4 GB) 6.49 in fsn1/nbg1/hel1 only; cpx22 (2 AMD vCPU, 4 GB) 22.99 EU, 30.99 sin;
|
|
# cpx21 (3 vCPU, 4 GB) 37.49 in ash/hil only (the old cpx line is deprecated in the EU and Singapore since Dec 2025);
|
|
# cpx11 (2 GB) 20.49 in ash/hil. So: cx23 in Europe, cpx22 in Singapore, cpx21 in the US. SERVER_TYPE overrides all.
|
|
TYPE_BY_LOCATION="${TYPE_BY_LOCATION:-fsn1=cx23,nbg1=cx23,hel1=cx23,sin=cpx22,ash=cpx21,hil=cpx21}"
|
|
SERVER_TYPE="${SERVER_TYPE:-}" # empty = pick from TYPE_BY_LOCATION
|
|
# Index ranges with their own type, "lo-hi:type,lo-hi:type"; other indexes use TYPE_BY_LOCATION. The 8th shared server
|
|
# failed with "shared core limit exceeded" (4 Oct 2026), dedicated vCPUs are a separate limit of 8, so nodes 8 to 11
|
|
# are ccx13 (2 dedicated AMD cores, 8 GB; USD 0.0809/h EU, 0.0817 US, 0.1017 sin) and node 12 is the last shared one.
|
|
TYPE_BY_INDEX="${TYPE_BY_INDEX-8-11:ccx13}"
|
|
DO_SIZE="${DO_SIZE:-s-2vcpu-4gb}"
|
|
BUILDER_TYPE="${BUILDER_TYPE:-cx43}" # 8 Intel vCPU, 16 GB, USD 0.0296/h, for the one-off node build (deleted afterwards)
|
|
DO_BUILDER_SIZE="${DO_BUILDER_SIZE:-c-8}"
|
|
IMAGE="${IMAGE:-debian-12}"
|
|
DO_IMAGE="${DO_IMAGE:-debian-12-x64}"
|
|
|
|
# SSH. The operations key ~/.ssh/igneum_ed25519 (3 Oct 2026, uploaded to Hetzner as "igneum-ops" by the seed-node
|
|
# scripts). If SSH_KEY_FILE is missing, create.sh generates an ed25519 pair there and uploads the public half.
|
|
SSH_KEY_NAME="${SSH_KEY_NAME:-igneum-ops}"
|
|
SSH_KEY_FILE="${SSH_KEY_FILE:-$HOME/.ssh/igneum_ed25519}"
|
|
SSH_USER="${SSH_USER:-root}"
|
|
|
|
# The network. A suffixed devnet (igneum-devnet-20) has its own handshake magic and its own genesis (the bits
|
|
# override recomputes the genesis hash), so it can never peer with or pollute the live devnet on the Mac.
|
|
DEVNET_SUFFIX="${DEVNET_SUFFIX:-20}"
|
|
# 0x1e400000 = 2^18 expected hashes per block. Three 6-thread CPU miners on the M5 Max held about 1 block/s at this
|
|
# value (fork-divergence, genesis row); 20 one-thread cloud vCPUs are in the same range. The DAA takes over after that.
|
|
GENESIS_BITS="${GENESIS_BITS:-0x1e400000}"
|
|
MESH_OUT="${MESH_OUT:-2}" # outbound --addpeer links per node (ring plus a chord); inbound doubles it, about 4 peers each
|
|
MINER_THREADS="${MINER_THREADS:-1}" # the trickle: one CPU thread per node
|
|
MINER_STATUS_SECS="${MINER_STATUS_SECS:-60}"
|
|
|
|
# Source to ship. NODE_SRC is the fork worktree whose code the network runs. master (vendor/igneum-node) carries the
|
|
# finality v2 work as uncommitted changes; the dual-lane difficulty controller is on the `difficulty` worktree
|
|
# (vendor/igneum-node-diff). Point NODE_SRC at the tree you want measured.
|
|
NODE_SRC="${NODE_SRC:-$REPO/vendor/igneum-node}"
|
|
POW_SRC="${POW_SRC:-$REPO/igneum-pow}"
|
|
# head = `git archive HEAD` only; head+dirty = HEAD plus every modified and untracked file of the worktree (target*/ excluded).
|
|
# All seven worktrees sit at d62708a8 with their branch work uncommitted (3 Oct 2026), so head+dirty is the default.
|
|
SRC_MODE="${SRC_MODE:-head+dirty}"
|
|
|
|
# Private-network mode (4 Oct 2026). A new Hetzner account is limited to 10 primary IPs and IPv4 and IPv6 both count
|
|
# (both "Primary IP limit exceeded" on the 5th node, 4 Oct 2026; Hetzner does not raise it for new accounts). So:
|
|
# one Hetzner private network per network zone (a network cannot span zones: "subnetwork zones are not aligned"),
|
|
# one public IPv4 gateway node per zone (the lowest index in the zone; IPv4 only, no IPv6 primary), every other node
|
|
# without public addresses. The gateway NATs the zone's outbound traffic (apt, chrony) and forwards one TCP port per
|
|
# private node (FWD_PORT_BASE + index) to that node's p2p port, so every node stays dialable from every zone and the
|
|
# ring mesh is unchanged. Same-zone links use the private IPs directly. ssh to a private node jumps through its
|
|
# gateway (lib/common.sh does it). NET_MODE=public is the old layout (every node public; needs N+1 primary IPs).
|
|
NET_MODE="${NET_MODE:-private}"
|
|
NET_PREFIX="${NET_PREFIX:-10.20}" # networks NET_PREFIX.<zone index>.0/24 (see ZONES), network name igneum-net-<zone>
|
|
FWD_PORT_BASE="${FWD_PORT_BASE:-27000}" # private node i is dialled at <gateway public ip>:<FWD_PORT_BASE + i>
|
|
# Hetzner network zones (hcloud location list, 4 Oct 2026): zone:index:locations
|
|
ZONES="${ZONES:-eu-central:1:fsn1 nbg1 hel1;us-east:2:ash;us-west:3:hil;ap-southeast:4:sin}"
|
|
|
|
# Ports (consensus/core/src/network.rs, devnet). RPC stays on loopback; only p2p is open to the world.
|
|
P2P_PORT=26611
|
|
RPC_PORT=26610
|
|
RPC_JSON_PORT=28610
|