Skip to main content

Radicle Sync Architecture

AC/DC repositories are automatically synchronized to the Radicle peer-to-peer network for decentralized, censorship-resistant code distribution.

Architecture​

The sync architecture uses a two-server model to work around network topology constraints:

┌─────────────────────────────────────────────────────────────────────┐
│                        Radicle Sync Flow                             │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  1. Push to Forgejo                                                  │
│     ────────────────                                                 │
│     Developer pushes to Forgejo (source.ac-dc.network)               │
│                                                                      │
│  2. CI Workflow Triggered                                            │
│     ─────────────────────                                            │
│     Forgejo triggers CI workflow on CI Runner (ci.ac-dc.network)     │
│                                                                      │
│  3. Local Radicle Push                                               │
│     ───────────────────                                              │
│     CI workflow pushes to CI Runner's local Radicle storage          │
│                                                                      │
│  4. Storage Rsync                                                    │
│     ──────────────                                                   │
│     CI Runner rsyncs storage to Source Server via private network    │
│     (10.106.0.3 → 10.106.0.2)                                        │
│                                                                      │
│  5. Network Announce                                                 │
│     ────────────────                                                 │
│     Source Server announces to Radicle seeds (network-connected)     │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Why This Architecture?​

The Problem​

Both servers share the same Radicle Node ID but have separate storage:

  • CI Runner: Executes CI jobs and pushes to local Radicle storage
  • Source Server: Has working connectivity to Radicle seeds

The CI Runner cannot directly announce to Radicle seeds (connection timeouts), while the Source Server can successfully connect to 10+ seeds.

The Solution​

Instead of direct announcement from CI Runner:

  1. CI Runner pushes to its local storage (fast, local operation)
  2. Storage is rsynced to Source Server over private network (fast, reliable)
  3. Source Server announces to Radicle network (working connectivity)

Components​

Scripts on CI Runner (10.106.0.3)​

/home/devops/radicle-full-sync.sh​

Main sync script called by CI workflows:

#!/bin/bash
# Usage: radicle-full-sync.sh <rid>
# Example: radicle-full-sync.sh rad:z3XCPA2jQz5Fhh6LnYKxHMCAyoAMG

RID="${1:-}"
REPO_ID="${RID#rad:}"
SOURCE_SERVER="10.106.0.2"
RADICLE_STORAGE="/home/devops/.radicle/storage"

# Step 1: Rsync storage to source server
rsync -az --delete "${RADICLE_STORAGE}/${REPO_ID}/" \
  "${SOURCE_SERVER}:${RADICLE_STORAGE}/${REPO_ID}/"

# Step 2: Trigger announce on source server
ssh ${SOURCE_SERVER} "rad sync ${RID} --announce"

/home/devops/push-radicle-to-source.sh​

Single-repo sync with verbose output:

#!/bin/bash
# Usage: push-radicle-to-source.sh <rid>

Scripts on Source Server (10.106.0.2)​

/home/devops/sync-radicle-all.sh​

Cron job script that fetches from network:

#!/bin/bash
# Syncs all tracked repos from Radicle network
for rid in $(rad ls --format json | jq -r ".[].rid"); do
  rad sync "$rid" --fetch
done

Cron Jobs​

On Source Server:

# Sync all Radicle repos from network every 5 minutes
*/5 * * * * /home/devops/sync-radicle-all.sh

CI Workflow Integration​

Each repository's CI workflow includes a Radicle sync step:

# .forgejo/workflows/sync.yml (or ci.yml)
jobs:
  radicle-push:
    runs-on: native
    steps:
      - name: Sync to Radicle
        run: |
          # ... push to local storage ...

      - name: Sync to network via source server
        run: |
          RID="rad:z3XCPA2jQz5Fhh6LnYKxHMCAyoAMG"
          $HOME/radicle-full-sync.sh "$RID"

Monitoring​

Check Sync Status​

# On source server - check all repo sync status
curl -s http://localhost:8081/api/repos | \
  python3 -c "import sys,json; d=json.load(sys.stdin); \
  synced=sum(1 for r in d['repos'] if r.get('sync_status')=='synced'); \
  print(f'Synced: {synced}/{len(d[\"repos\"])} repos')"

Verify Specific Repo​

# Check Forgejo HEAD
git ls-remote https://source.ac-dc.network/alpha-delta-network/REPO.git HEAD

# Check Radicle HEAD on source server
ssh -p 2584 devops@source.ac-dc.network "rad ls | grep REPO"

# Check Radicle HEAD on CI runner
ssh -p 2584 devops@ci.ac-dc.network "rad ls | grep REPO"

Manual Sync​

# Force sync a specific repo
ssh -p 2584 devops@ci.ac-dc.network \
  "/home/devops/push-radicle-to-source.sh rad:z3XCPA2jQz5Fhh6LnYKxHMCAyoAMG"

# Sync all repos
ssh -p 2584 devops@ci.ac-dc.network \
  'for rid in $(rad ls 2>&1 | grep -oE "rad:[a-zA-Z0-9]+" | sort -u); do
    /home/devops/push-radicle-to-source.sh "$rid"
  done'

Troubleshooting​

Repo Shows as "Unsynced"​

  1. Check if CI workflow completed successfully
  2. Verify storage was rsynced: compare HEADs on both servers
  3. Check if announce succeeded: look for "Synced with N seed(s)" message

Announce Fails with "All seeds timed out"​

This is expected on CI Runner. The source server handles network announces. If it happens on source server:

  • Check Radicle node status: rad node status
  • Verify seed connectivity: nc -zv iris.radicle.xyz 8776
  • Restart node if needed: rad node stop && rad node start

Storage Out of Sync​

# Force rsync from CI Runner to Source Server
ssh -p 2584 devops@ci.ac-dc.network \
  "rsync -az --delete ~/.radicle/storage/REPO_ID/ \
   10.106.0.2:~/.radicle/storage/REPO_ID/"

Network Information​

Radicle Node ID​

Both servers share: z6MkqVELL15xFNq4FfnJ87shQGXBnJekUgmrhgk491DQd2MP

Connected Seeds (Source Server)​

The source server maintains connections to 10+ Radicle seeds including:

  • iris.radicle.xyz:8776
  • rosa.radicle.xyz:8776
  • seed.radicle.at:8776
  • radicle.dpc.pw:8776

Storage Locations​

  • CI Runner: /home/devops/.radicle/storage/
  • Source Server: /home/devops/.radicle/storage/