Start a Node

This page explains the cockroach start command, which you use to start nodes as a new cluster or add nodes to an existing cluster. For a full walk-through of the cluster startup and initialization process, see one of the Manual Deployment tutorials.

Note:

Node-level settings are defined by flags passed to the cockroach start command and cannot be changed without stopping and restarting the node. In contrast, some cluster-wide settings are defined via SQL statements and can be updated anytime after a cluster has been started. For more details, see Cluster Settings.

Synopsis

# Start a single-node cluster:
$ cockroach start <flags, excluding --join>
# Start a multi-node cluster:
$ cockroach start <flags, including --join> &
$ cockroach init <flags>
# Add a node to a cluster:
$ cockroach start <flags, including --join>
# View help:
$ cockroach start --help

Flags

The start command supports the following general-use, networking, security, and logging flags. All flags must be specified each time thenode is started, as they will not be remembered, with the exception of the —join flag. Nevertheless, we recommend specifying all flags every time, including the —join flag, as that willallow restarted nodes to join the cluster even if their data directory was destroyed.

Tip:

When adding a node to an existing cluster, include the —join flag.

General

Networking

Security

Locality

The —locality flag accepts arbitrary key-value pairs that describe the location of the node. Locality might include country, region, datacenter, rack, etc. The key-value pairs should be ordered from most to least inclusive, and the keys and order of key-value pairs must be the same on all nodes. It's typically better to include more pairs than fewer.

• CockroachDB spreads the replicas of each piece of data across as diverse a set of localities as possible, with the order determining the priority. However, locality can also be used to influence the location of data replicas in various ways using replication zones.

• When there is high latency between nodes (e.g., cross-datacenter deployments), CockroachDB uses locality to move range leases closer to the current workload, reducing network round trips and improving read performance, also known as "follow-the-workload". In a deployment across more than 3 datacenters, however, to ensure that all data benefits from "follow-the-workload", you must increase your replication factor to match the total number of datacenters.

• Locality is also a prerequisite for using the table partitioning and Node Map enterprise features.

Example

# Locality flag for nodes in US East datacenter:
--locality=region=us,datacenter=us-east
# Locality flag for nodes in US Central datacenter:
--locality=region=us,datacenter=us-central
# Locality flag for nodes in US West datacenter:
--locality=region=us,datacenter=us-west

Store

The —store flag supports the following fields. Note that commas are used to separate fields, and so are forbidden in all field values.

Note:

In-memory storage is not suitable for production deployments at this time.

Warning:

In the absence of special replication constraints, CockroachDB rebalances replicas to take advantage of available storage capacity. However, in a 3-node cluster with multiple stores per node, CockroachDB is not able to rebalance replicas from one store to another store on the same node because this would temporarily result in the node having multiple replicas of the same range, which is not allowed. This is due to the mechanics of rebalancing, where the cluster first creates a copy of the replica at the target destination before removing the source replica. To allow this type of cross-store rebalancing, the cluster must have 4 or more nodes; this allows the cluster to create a copy of the replica on a node that doesn't already have a replica of the range before removing the source replica and then migrating the new replica to the store with more capacity on the original node.

Logging

By default, cockroach start writes all messages to log files, and prints nothing to stderr. However, you can control the process's logging behavior with the following flags:

Defaults

cockroach start uses the equivalent values for these logging flags by default:

• —log-dir=<first store dir>/logs
• —logtostderr=NONE
  This means, by default, CockroachDB writes all messages to log files, and never prints to stderr.

Standard output

When you run cockroach start, some helpful details are printed to the standard output:

CockroachDB node starting at 2018-04-30 15:10:52.34274101 +0000 UTC
build:               CCL v19.1.0 @ 2019/04/30 14:48:26 (go1.11.6)
webui:               http://localhost:8080
sql:                 postgresql://root@localhost:26257?sslmode=disable
client flags:        cockroach <client cmd> --listen-addr=localhost:26257 --insecure
logs:                /cockroach-data/logs
temp dir:            /cockroach-data/cockroach-temp430873933
external I/O path:   /cockroach-data/extern
attrs:               ram:64gb
locality:            datacenter=us-east1
store[0]:            path=cockroach-data,attrs=ssd
status:              initialized new cluster
clusterID:           7b9329d0-580d-4035-8319-53ba8b74b213
nodeID:              1

Tip:

These details are also written to the INFO log in the /logs directory in case you need to refer to them at a later time.

Examples

Start a single-node cluster

To start a single-node cluster, run the cockroach start command without the —join flag:

$ cockroach start \
--certs-dir=certs \
--advertise-addr=<node1 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--insecure \
--advertise-addr=<node1 address> \
--cache=.25 \
--max-sql-memory=.25

Start a multi-node cluster

To start a multi-node cluster, run the cockroach start command for each node, setting the —join flag to the addresses of 3-5 of the initial nodes:

$ cockroach start \
--certs-dir=certs \
--advertise-addr=<node1 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--certs-dir=certs \
--advertise-addr=<node2 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--certs-dir=certs \
--advertise-addr=<node3 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--insecure \
--advertise-addr=<node1 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--insecure \
--advertise-addr=<node2 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--insecure \
--advertise-addr=<node3 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

Then run the cockroach init command against any node to perform a one-time cluster initialization:

$ cockroach init \
--certs-dir=certs \
--host=<address of any node>

$ cockroach init \
--insecure \
--host=<address of any node>

Start a multi-node cluster across private networks

Scenario:

• You have a cluster that spans GCE and AWS.

• The nodes on each cloud can reach each other on private addresses, but the private addresses aren't reachable from the other cloud.
  Approach:

• Start each node on GCE with —locality set to describe its location, —locality-advertise-addr set to advertise its private address to other nodes in on GCE, —advertise-addr set to advertise its public address to nodes on AWS, and —join set to the public addresses of 3-5 of the initial nodes:

$ cockroach start \
--certs-dir=certs \
--locality=cloud=gce \
--locality-advertise-addr=cloud=gce@<private address of node> \
--advertise-addr=<public address of node> \
--join=<public address of node1>,<public address of node2>,<public address of node3> \
--cache=.25 \
--max-sql-memory=.25

• Start each node on AWS with —locality set to describe its location, —locality-advertise-addr set to advertise its private address to other nodes on AWS, —advertise-addr set to advertise its public address to nodes on GCE, and —join set to the public addresses of 3-5 of the initial nodes:

$ cockroach start \
--certs-dir=certs \
--locality=cloud=aws \
--locality-advertise-addr=cloud=aws@<private address of node> \
--advertise-addr=<public address of node> \
--join=<public address of node1>,<public address of node2>,<public address of node3> \
--cache=.25 \
--max-sql-memory=.25

• Run the cockroach init command against any node to perform a one-time cluster initialization:

$ cockroach init \
--certs-dir=certs \
--host=<address of any node>

Add a node to a cluster

To add a node to an existing cluster, run the cockroach start command, setting the —join flag to the addresses of 3-5 of the nodes already in the cluster:

$ cockroach start \
--certs-dir=certs \
--advertise-addr=<node4 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

$ cockroach start \
--insecure \
--advertise-addr=<node4 address> \
--join=<node1 address>,<node2 address>,<node3 address> \
--cache=.25 \
--max-sql-memory=.25

See also

• Initialize a Cluster
• Manual Deployment
• Orchestrated Deployment
• Test Deployment
• Local Deployment
• Other Cockroach Commands

Was this page helpful?
YesNo