Metrics Reference

Orderer Metrics

Prometheus

The following orderer metrics are exported for consumption by Prometheus.

Name

Type

Description

Labels

blockcutter_block_fill_duration

histogram

The time from first transaction enqueing to the block being cut in seconds.

channel

broadcast_enqueue_duration

histogram

The time to enqueue a transaction in seconds.

channel

type

status

broadcast_processed_count

counter

The number of transactions processed.

channel

type

status

broadcast_validate_duration

histogram

The time to validate a transaction in seconds.

channel

type

status

cluster_comm_egress_queue_capacity

gauge

Capacity of the egress queue.

host

msg_type

channel

cluster_comm_egress_queue_length

gauge

Length of the egress queue.

host

msg_type

channel

cluster_comm_egress_queue_workers

gauge

Count of egress queue workers.

channel

cluster_comm_egress_stream_count

gauge

Count of streams to other nodes.

channel

cluster_comm_egress_tls_connection_count

gauge

Count of TLS connections to other nodes.

cluster_comm_ingress_stream_count

gauge

Count of streams from other nodes.

cluster_comm_msg_dropped_count

counter

Count of messages dropped.

host

channel

cluster_comm_msg_send_time

histogram

The time it takes to send a message in seconds.

host

channel

consensus_etcdraft_active_nodes

gauge

Number of active nodes in this channel.

channel

consensus_etcdraft_cluster_size

gauge

Number of nodes in this channel.

channel

consensus_etcdraft_committed_block_number

gauge

The block number of the latest block committed.

channel

consensus_etcdraft_config_proposals_received

counter

The total number of proposals received for config type transactions.

channel

consensus_etcdraft_data_persist_duration

histogram

The time taken for etcd/raft data to be persisted in storage (in seconds).

channel

consensus_etcdraft_is_leader

gauge

The leadership status of the current node: 1 if it is the leader else 0.

channel

consensus_etcdraft_leader_changes

counter

The number of leader changes since process start.

channel

consensus_etcdraft_normal_proposals_received

counter

The total number of proposals received for normal type transactions.

channel

consensus_etcdraft_proposal_failures

counter

The number of proposal failures.

channel

consensus_etcdraft_snapshot_block_number

gauge

The block number of the latest snapshot.

channel

consensus_kafka_batch_size

gauge

The mean batch size in bytes sent to topics.

topic

consensus_kafka_compression_ratio

gauge

The mean compression ratio (as percentage) for topics.

topic

consensus_kafka_incoming_byte_rate

gauge

Bytes/second read off brokers.

broker_id

consensus_kafka_last_offset_persisted

gauge

The offset specified in the block metadata of the most recently committed block.

channel

consensus_kafka_outgoing_byte_rate

gauge

Bytes/second written to brokers.

broker_id

consensus_kafka_record_send_rate

gauge

The number of records per second sent to topics.

topic

consensus_kafka_records_per_request

gauge

The mean number of records sent per request to topics.

topic

consensus_kafka_request_latency

gauge

The mean request latency in ms to brokers.

broker_id

consensus_kafka_request_rate

gauge

Requests/second sent to brokers.

broker_id

consensus_kafka_request_size

gauge

The mean request size in bytes to brokers.

broker_id

consensus_kafka_response_rate

gauge

Requests/second sent to brokers.

broker_id

consensus_kafka_response_size

gauge

The mean response size in bytes from brokers.

broker_id

deliver_blocks_sent

counter

The number of blocks sent by the deliver service.

channel

filtered

data_type

deliver_requests_completed

counter

The number of deliver requests that have been completed.

channel

filtered

data_type

success

deliver_requests_received

counter

The number of deliver requests that have been received.

channel

filtered

data_type

deliver_streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver_streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

fabric_version

gauge

The active version of Fabric.

version

grpc_comm_conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc_comm_conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc_server_stream_messages_received

counter

The number of stream messages received.

service

method

grpc_server_stream_messages_sent

counter

The number of stream messages sent.

service

method

grpc_server_stream_request_duration

histogram

The time to complete a stream request.

service

method

code

grpc_server_stream_requests_completed

counter

The number of stream requests completed.

service

method

code

grpc_server_stream_requests_received

counter

The number of stream requests received.

service

method

grpc_server_unary_request_duration

histogram

The time to complete a unary request.

service

method

code

grpc_server_unary_requests_completed

counter

The number of unary requests completed.

service

method

code

grpc_server_unary_requests_received

counter

The number of unary requests received.

service

method

ledger_blockchain_height

gauge

Height of the chain in blocks.

channel

ledger_blockstorage_commit_time

histogram

Time taken in seconds for committing the block to storage.

channel

logging_entries_checked

counter

Number of log entries checked against the active logging level

level

logging_entries_written

counter

Number of log entries that are written

level

participation_consensus_relation

gauge

The channel participation consensus relation of the node: 0 if other, 1 if consenter, 2 if follower, 3 if config-tracker.

channel

participation_status

gauge

The channel participation status of the node: 0 if inactive, 1 if active, 2 if onboarding, 3 if failed.

channel

StatsD

The following orderer metrics are emitted for consumption by StatsD. The %{variable_name} nomenclature represents segments that vary based on context.

For example, %{channel} will be replaced with the name of the channel associated with the metric.

Bucket

Type

Description

blockcutter.block_fill_duration.%{channel}

histogram

The time from first transaction enqueing to the block being cut in seconds.

broadcast.enqueue_duration.%{channel}.%{type}.%{status}

histogram

The time to enqueue a transaction in seconds.

broadcast.processed_count.%{channel}.%{type}.%{status}

counter

The number of transactions processed.

broadcast.validate_duration.%{channel}.%{type}.%{status}

histogram

The time to validate a transaction in seconds.

cluster.comm.egress_queue_capacity.%{host}.%{msg_type}.%{channel}

gauge

Capacity of the egress queue.

cluster.comm.egress_queue_length.%{host}.%{msg_type}.%{channel}

gauge

Length of the egress queue.

cluster.comm.egress_queue_workers.%{channel}

gauge

Count of egress queue workers.

cluster.comm.egress_stream_count.%{channel}

gauge

Count of streams to other nodes.

cluster.comm.egress_tls_connection_count

gauge

Count of TLS connections to other nodes.

cluster.comm.ingress_stream_count

gauge

Count of streams from other nodes.

cluster.comm.msg_dropped_count.%{host}.%{channel}

counter

Count of messages dropped.

cluster.comm.msg_send_time.%{host}.%{channel}

histogram

The time it takes to send a message in seconds.

consensus.etcdraft.active_nodes.%{channel}

gauge

Number of active nodes in this channel.

consensus.etcdraft.cluster_size.%{channel}

gauge

Number of nodes in this channel.

consensus.etcdraft.committed_block_number.%{channel}

gauge

The block number of the latest block committed.

consensus.etcdraft.config_proposals_received.%{channel}

counter

The total number of proposals received for config type transactions.

consensus.etcdraft.data_persist_duration.%{channel}

histogram

The time taken for etcd/raft data to be persisted in storage (in seconds).

consensus.etcdraft.is_leader.%{channel}

gauge

The leadership status of the current node: 1 if it is the leader else 0.

consensus.etcdraft.leader_changes.%{channel}

counter

The number of leader changes since process start.

consensus.etcdraft.normal_proposals_received.%{channel}

counter

The total number of proposals received for normal type transactions.

consensus.etcdraft.proposal_failures.%{channel}

counter

The number of proposal failures.

consensus.etcdraft.snapshot_block_number.%{channel}

gauge

The block number of the latest snapshot.

consensus.kafka.batch_size.%{topic}

gauge

The mean batch size in bytes sent to topics.

consensus.kafka.compression_ratio.%{topic}

gauge

The mean compression ratio (as percentage) for topics.

consensus.kafka.incoming_byte_rate.%{broker_id}

gauge

Bytes/second read off brokers.

consensus.kafka.last_offset_persisted.%{channel}

gauge

The offset specified in the block metadata of the most recently committed block.

consensus.kafka.outgoing_byte_rate.%{broker_id}

gauge

Bytes/second written to brokers.

consensus.kafka.record_send_rate.%{topic}

gauge

The number of records per second sent to topics.

consensus.kafka.records_per_request.%{topic}

gauge

The mean number of records sent per request to topics.

consensus.kafka.request_latency.%{broker_id}

gauge

The mean request latency in ms to brokers.

consensus.kafka.request_rate.%{broker_id}

gauge

Requests/second sent to brokers.

consensus.kafka.request_size.%{broker_id}

gauge

The mean request size in bytes to brokers.

consensus.kafka.response_rate.%{broker_id}

gauge

Requests/second sent to brokers.

consensus.kafka.response_size.%{broker_id}

gauge

The mean response size in bytes from brokers.

deliver.blocks_sent.%{channel}.%{filtered}.%{data_type}

counter

The number of blocks sent by the deliver service.

deliver.requests_completed.%{channel}.%{filtered}.%{data_type}.%{success}

counter

The number of deliver requests that have been completed.

deliver.requests_received.%{channel}.%{filtered}.%{data_type}

counter

The number of deliver requests that have been received.

deliver.streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver.streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

fabric_version.%{version}

gauge

The active version of Fabric.

grpc.comm.conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc.comm.conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc.server.stream_messages_received.%{service}.%{method}

counter

The number of stream messages received.

grpc.server.stream_messages_sent.%{service}.%{method}

counter

The number of stream messages sent.

grpc.server.stream_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a stream request.

grpc.server.stream_requests_completed.%{service}.%{method}.%{code}

counter

The number of stream requests completed.

grpc.server.stream_requests_received.%{service}.%{method}

counter

The number of stream requests received.

grpc.server.unary_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a unary request.

grpc.server.unary_requests_completed.%{service}.%{method}.%{code}

counter

The number of unary requests completed.

grpc.server.unary_requests_received.%{service}.%{method}

counter

The number of unary requests received.

ledger.blockchain_height.%{channel}

gauge

Height of the chain in blocks.

ledger.blockstorage_commit_time.%{channel}

histogram

Time taken in seconds for committing the block to storage.

logging.entries_checked.%{level}

counter

Number of log entries checked against the active logging level

logging.entries_written.%{level}

counter

Number of log entries that are written

participation.consensus_relation.%{channel}

gauge

The channel participation consensus relation of the node: 0 if other, 1 if consenter, 2 if follower, 3 if config-tracker.

participation.status.%{channel}

gauge

The channel participation status of the node: 0 if inactive, 1 if active, 2 if onboarding, 3 if failed.

Peer Metrics

Prometheus

The following peer metrics are exported for consumption by Prometheus.

Name

Type

Description

Labels

chaincode_execute_timeouts

counter

The number of chaincode executions (Init or Invoke) that have timed out.

chaincode

chaincode_launch_duration

histogram

The time to launch a chaincode.

chaincode

success

chaincode_launch_failures

counter

The number of chaincode launches that have failed.

chaincode

chaincode_launch_timeouts

counter

The number of chaincode launches that have timed out.

chaincode

chaincode_shim_request_duration

histogram

The time to complete chaincode shim requests.

type

channel

chaincode

success

chaincode_shim_requests_completed

counter

The number of chaincode shim requests completed.

type

channel

chaincode

success

chaincode_shim_requests_received

counter

The number of chaincode shim requests received.

type

channel

chaincode

couchdb_processing_time

histogram

Time taken in seconds for the function to complete request to CouchDB

database

function_name

result

deliver_blocks_sent

counter

The number of blocks sent by the deliver service.

channel

filtered

data_type

deliver_requests_completed

counter

The number of deliver requests that have been completed.

channel

filtered

data_type

success

deliver_requests_received

counter

The number of deliver requests that have been received.

channel

filtered

data_type

deliver_streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver_streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

dockercontroller_chaincode_container_build_duration

histogram

The time to build a chaincode image in seconds.

chaincode

success

endorser_chaincode_instantiation_failures

counter

The number of chaincode instantiations or upgrade that have failed.

channel

chaincode

endorser_duplicate_transaction_failures

counter

The number of failed proposals due to duplicate transaction ID.

channel

chaincode

endorser_endorsement_failures

counter

The number of failed endorsements.

channel

chaincode

chaincodeerror

endorser_proposal_acl_failures

counter

The number of proposals that failed ACL checks.

channel

chaincode

endorser_proposal_duration

histogram

The time to complete a proposal.

channel

chaincode

success

endorser_proposal_simulation_failures

counter

The number of failed proposal simulations

channel

chaincode

endorser_proposal_validation_failures

counter

The number of proposals that have failed initial validation.

endorser_proposals_received

counter

The number of proposals received.

endorser_successful_proposals

counter

The number of successful proposals.

fabric_version

gauge

The active version of Fabric.

version

gossip_comm_messages_received

counter

Number of messages received

gossip_comm_messages_sent

counter

Number of messages sent

gossip_comm_overflow_count

counter

Number of outgoing queue buffer overflows

gossip_leader_election_leader

gauge

Peer is leader (1) or follower (0)

channel

gossip_membership_total_peers_known

gauge

Total known peers

channel

gossip_payload_buffer_size

gauge

Size of the payload buffer

channel

gossip_privdata_commit_block_duration

histogram

Time it takes to commit private data and the corresponding block (in seconds)

channel

gossip_privdata_fetch_duration

histogram

Time it takes to fetch missing private data from peers (in seconds)

channel

gossip_privdata_list_missing_duration

histogram

Time it takes to list the missing private data (in seconds)

channel

gossip_privdata_pull_duration

histogram

Time it takes to pull a missing private data element (in seconds)

channel

gossip_privdata_purge_duration

histogram

Time it takes to purge private data (in seconds)

channel

gossip_privdata_reconciliation_duration

histogram

Time it takes for reconciliation to complete (in seconds)

channel

gossip_privdata_retrieve_duration

histogram

Time it takes to retrieve missing private data elements from the ledger (in seconds)

channel

gossip_privdata_send_duration

histogram

Time it takes to send a missing private data element (in seconds)

channel

gossip_privdata_validation_duration

histogram

Time it takes to validate a block (in seconds)

channel

gossip_state_commit_duration

histogram

Time it takes to commit a block in seconds

channel

gossip_state_height

gauge

Current ledger height

channel

grpc_comm_conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc_comm_conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc_server_stream_messages_received

counter

The number of stream messages received.

service

method

grpc_server_stream_messages_sent

counter

The number of stream messages sent.

service

method

grpc_server_stream_request_duration

histogram

The time to complete a stream request.

service

method

code

grpc_server_stream_requests_completed

counter

The number of stream requests completed.

service

method

code

grpc_server_stream_requests_received

counter

The number of stream requests received.

service

method

grpc_server_unary_request_duration

histogram

The time to complete a unary request.

service

method

code

grpc_server_unary_requests_completed

counter

The number of unary requests completed.

service

method

code

grpc_server_unary_requests_received

counter

The number of unary requests received.

service

method

ledger_block_processing_time

histogram

Time taken in seconds for ledger block processing.

channel

ledger_blockchain_height

gauge

Height of the chain in blocks.

channel

ledger_blockstorage_and_pvtdata_commit_time

histogram

Time taken in seconds for committing the block and private data to storage.

channel

ledger_blockstorage_commit_time

histogram

Time taken in seconds for committing the block to storage.

channel

ledger_statedb_commit_time

histogram

Time taken in seconds for committing block changes to state db.

channel

ledger_transaction_count

counter

Number of transactions processed.

channel

transaction_type

chaincode

validation_code

logging_entries_checked

counter

Number of log entries checked against the active logging level

level

logging_entries_written

counter

Number of log entries that are written

level

StatsD

The following peer metrics are emitted for consumption by StatsD. The %{variable_name} nomenclature represents segments that vary based on context.

For example, %{channel} will be replaced with the name of the channel associated with the metric.

Bucket

Type

Description

chaincode.execute_timeouts.%{chaincode}

counter

The number of chaincode executions (Init or Invoke) that have timed out.

chaincode.launch_duration.%{chaincode}.%{success}

histogram

The time to launch a chaincode.

chaincode.launch_failures.%{chaincode}

counter

The number of chaincode launches that have failed.

chaincode.launch_timeouts.%{chaincode}

counter

The number of chaincode launches that have timed out.

chaincode.shim_request_duration.%{type}.%{channel}.%{chaincode}.%{success}

histogram

The time to complete chaincode shim requests.

chaincode.shim_requests_completed.%{type}.%{channel}.%{chaincode}.%{success}

counter

The number of chaincode shim requests completed.

chaincode.shim_requests_received.%{type}.%{channel}.%{chaincode}

counter

The number of chaincode shim requests received.

couchdb.processing_time.%{database}.%{function_name}.%{result}

histogram

Time taken in seconds for the function to complete request to CouchDB

deliver.blocks_sent.%{channel}.%{filtered}.%{data_type}

counter

The number of blocks sent by the deliver service.

deliver.requests_completed.%{channel}.%{filtered}.%{data_type}.%{success}

counter

The number of deliver requests that have been completed.

deliver.requests_received.%{channel}.%{filtered}.%{data_type}

counter

The number of deliver requests that have been received.

deliver.streams_closed

counter

The number of GRPC streams that have been closed for the deliver service.

deliver.streams_opened

counter

The number of GRPC streams that have been opened for the deliver service.

dockercontroller.chaincode_container_build_duration.%{chaincode}.%{success}

histogram

The time to build a chaincode image in seconds.

endorser.chaincode_instantiation_failures.%{channel}.%{chaincode}

counter

The number of chaincode instantiations or upgrade that have failed.

endorser.duplicate_transaction_failures.%{channel}.%{chaincode}

counter

The number of failed proposals due to duplicate transaction ID.

endorser.endorsement_failures.%{channel}.%{chaincode}.%{chaincodeerror}

counter

The number of failed endorsements.

endorser.proposal_acl_failures.%{channel}.%{chaincode}

counter

The number of proposals that failed ACL checks.

endorser.proposal_duration.%{channel}.%{chaincode}.%{success}

histogram

The time to complete a proposal.

endorser.proposal_simulation_failures.%{channel}.%{chaincode}

counter

The number of failed proposal simulations

endorser.proposal_validation_failures

counter

The number of proposals that have failed initial validation.

endorser.proposals_received

counter

The number of proposals received.

endorser.successful_proposals

counter

The number of successful proposals.

fabric_version.%{version}

gauge

The active version of Fabric.

gossip.comm.messages_received

counter

Number of messages received

gossip.comm.messages_sent

counter

Number of messages sent

gossip.comm.overflow_count

counter

Number of outgoing queue buffer overflows

gossip.leader_election.leader.%{channel}

gauge

Peer is leader (1) or follower (0)

gossip.membership.total_peers_known.%{channel}

gauge

Total known peers

gossip.payload_buffer.size.%{channel}

gauge

Size of the payload buffer

gossip.privdata.commit_block_duration.%{channel}

histogram

Time it takes to commit private data and the corresponding block (in seconds)

gossip.privdata.fetch_duration.%{channel}

histogram

Time it takes to fetch missing private data from peers (in seconds)

gossip.privdata.list_missing_duration.%{channel}

histogram

Time it takes to list the missing private data (in seconds)

gossip.privdata.pull_duration.%{channel}

histogram

Time it takes to pull a missing private data element (in seconds)

gossip.privdata.purge_duration.%{channel}

histogram

Time it takes to purge private data (in seconds)

gossip.privdata.reconciliation_duration.%{channel}

histogram

Time it takes for reconciliation to complete (in seconds)

gossip.privdata.retrieve_duration.%{channel}

histogram

Time it takes to retrieve missing private data elements from the ledger (in seconds)

gossip.privdata.send_duration.%{channel}

histogram

Time it takes to send a missing private data element (in seconds)

gossip.privdata.validation_duration.%{channel}

histogram

Time it takes to validate a block (in seconds)

gossip.state.commit_duration.%{channel}

histogram

Time it takes to commit a block in seconds

gossip.state.height.%{channel}

gauge

Current ledger height

grpc.comm.conn_closed

counter

gRPC connections closed. Open minus closed is the active number of connections.

grpc.comm.conn_opened

counter

gRPC connections opened. Open minus closed is the active number of connections.

grpc.server.stream_messages_received.%{service}.%{method}

counter

The number of stream messages received.

grpc.server.stream_messages_sent.%{service}.%{method}

counter

The number of stream messages sent.

grpc.server.stream_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a stream request.

grpc.server.stream_requests_completed.%{service}.%{method}.%{code}

counter

The number of stream requests completed.

grpc.server.stream_requests_received.%{service}.%{method}

counter

The number of stream requests received.

grpc.server.unary_request_duration.%{service}.%{method}.%{code}

histogram

The time to complete a unary request.

grpc.server.unary_requests_completed.%{service}.%{method}.%{code}

counter

The number of unary requests completed.

grpc.server.unary_requests_received.%{service}.%{method}

counter

The number of unary requests received.

ledger.block_processing_time.%{channel}

histogram

Time taken in seconds for ledger block processing.

ledger.blockchain_height.%{channel}

gauge

Height of the chain in blocks.

ledger.blockstorage_and_pvtdata_commit_time.%{channel}

histogram

Time taken in seconds for committing the block and private data to storage.

ledger.blockstorage_commit_time.%{channel}

histogram

Time taken in seconds for committing the block to storage.

ledger.statedb_commit_time.%{channel}

histogram

Time taken in seconds for committing block changes to state db.

ledger.transaction_count.%{channel}.%{transaction_type}.%{chaincode}.%{validation_code}

counter

Number of transactions processed.

logging.entries_checked.%{level}

counter

Number of log entries checked against the active logging level

logging.entries_written.%{level}

counter

Number of log entries that are written