Skip to content
Distributed Erlang and Nodes
step 1/7

Reading — step 1 of 7

Learn

~3 min readETS, Applications, with

BEAM was designed for distributed computing from day one. Connecting nodes lets processes on different machines communicate AS IF they were on one — a feature few other languages match natively. Programming Elixir's distribution chapter and OTP docs are the references.

Note: Judge0 runs single-node — these features are conceptual.

Naming a node

Start Elixir with a node name:

iex --name node1@hostname
# or short name (single hostname segment):
iex --sname node1

The shell prompt now shows the node:

iex(node1@hostname)1>

Connecting nodes

In one shell:

iex --sname node1 --cookie shared_secret

In another shell on the same machine:

iex --sname node2 --cookie shared_secret

Then from node2:

Node.connect(:"node1@hostname")
Node.list()        # [:"node1@hostname"]

The --cookie is a shared secret. Nodes refuse connections from differing-cookie peers. Set in production via the VM args file.

Sending messages across nodes

# On node1, register a process:
pid = spawn(fn ->
    receive do
        {:ping, sender} -> send(sender, :pong)
    end
end)
:global.register_name(:pinger, pid)

# On node2:
GenServer.cast({:pinger, :"node1@hostname"}, :ping)
# OR via :global.whereis_name

Messages just work across nodes. The BEAM serializes the term, sends it, deserializes on the receiver. PIDs from remote nodes are first-class — you can hold them, forward them, monitor them.

Process spawning

Node.spawn(:"node2@hostname", fn ->
    IO.puts "running on node2!"
end)

Spawn a function on a remote node. The result depends on what the function does — often it sends results back as messages.

Distributed GenServer

# Start a named GenServer:
GenServer.start_link(MyServer, init, name: {:global, :my_server})

# From any connected node:
GenServer.call({:global, :my_server}, :get)

{:global, name} works across all connected nodes. name: ModuleName is local-only.

:global vs :rpc vs Phoenix.PubSub

  • :global — built-in but slow at scale; uses a centralized name registry
  • :rpc — Remote Procedure Call; one-shot remote function calls
  • Phoenix.PubSub — distributed pub/sub built on top; what real apps use
  • Horde — modern distributed Registry/Supervisor

Erlang's distribution model — caveats

Designed for trusted networks — the cookie isn't strong security. Use within a private VPC or behind a TLS-enabled epmd.

Mesh topology by default — every node connects to every other. For 50+ nodes, this gets expensive (use --hidden connections or a different distribution strategy).

Network partitions — when nodes lose connectivity, both halves think the other died. Resolve via :erlang.set_cookie + custom heartbeats, or use libcluster + libcluster_kubernetes for orchestration.

libcluster is the standard library for Kubernetes/cloud node discovery — it watches for new pods and connects them automatically.

Real-world: Phoenix Channels at scale

Phoenix Channels uses distributed Erlang under the hood:

  • WebSockets terminate on individual nodes
  • PubSub broadcasts to all subscribers across the cluster
  • Presence tracking is CRDT-based across nodes

Discord runs on this. So does WhatsApp's signaling. Real production-scale distributed Elixir.

When to use distribution

  • Yes: stateful services that need fault tolerance, real-time pub/sub, low-latency cross-node messaging
  • No: stateless web requests (load-balance via standard HTTP), short-lived jobs (use a queue), connection-heavy workloads where Mnesia/ETS isn't sufficient

Common mistakes

  • Default cookie in production~/.erlang.cookie defaults to a random value but is sometimes left as a known value. Use a strong shared secret.
  • Distributed Erlang without TLS — traffic is in plaintext. Enable TLS distribution for production.
  • Assuming distributed = magic scaling — coordination still has costs. Some workloads scale better with stateless services + queues.
  • Forgetting partitions — split-brain is real. Use libcluster strategies, monitoring, manual reconciliation.
  • Using :global for high-throughput — central bottleneck. Phoenix.PubSub or Horde scale better.

Discussion

Ask a question, share an insight, or help someone who’s stuck.

Sign in to post a comment or reply.

Loading…