Observability

Agent

Compile source code

Download Apache ShardingSphere from GitHub,Then compile.

git clone --depth 1 https://github.com/apache/shardingsphere.git
cd shardingsphere
mvn clean install -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Drat.skip=true -Djacoco.skip=true -DskipITs -DskipTests -Prelease

Artifact is distribution/agent/target/apache-shardingsphere-${latest.release.version}-shardingsphere-agent-bin.tar.gz

Directory structure

Create agent directory, and unzip agent distribution package to the directory.

mkdir agent
tar -zxvf apache-shardingsphere-${latest.release.version}-shardingsphere-agent-bin.tar.gz -C agent
cd agent
tree 
├── LICENSE
├── NOTICE
├── conf
│   └── agent.yaml
├── plugins
│   ├── lib
│   │   ├── shardingsphere-agent-metrics-core-${latest.release.version}.jar
│   │   └── shardingsphere-agent-plugin-core-${latest.release.version}.jar
│   ├── logging
│   │   └── shardingsphere-agent-logging-file-${latest.release.version}.jar
│   ├── metrics
│   │   └── shardingsphere-agent-metrics-prometheus-${latest.release.version}.jar
│   └── tracing
│       ├── shardingsphere-agent-tracing-opentelemetry-${latest.release.version}.jar
│       └── shardingsphere-agent-tracing-opentracing-${latest.release.version}.jar
└── shardingsphere-agent-${latest.release.version}.jar

Agent log output location is agent/logs/stdout.log.

Configuration

conf/agent.yaml is used to manage agent configuration. Built-in plugins include File, Prometheus, OpenTelemetry, OpenTracing.

plugins:
#  logging:
#    File:
#      props:
#        level: "INFO"
#  metrics:
#    Prometheus:
#      host:  "localhost"
#      port: 9090
#      props:
#        jvm-information-collector-enabled: "true"
#  tracing:
#    OpenTelemetry:
#      props:
#        otel.service.name: "shardingsphere"
#        otel.traces.exporter: "jaeger"
#        otel.exporter.otlp.traces.endpoint: "http://localhost:14250"
#        otel.traces.sampler: "always_on"
#    OpenTracing:
#      props:
#        opentracing-tracer-class-name: "org.apache.skywalking.apm.toolkit.opentracing.SkywalkingTracer"

Plugin description

File

Currently, the File plugin only outputs the time-consuming log output of building metadata, and has no other log output for the time being.

Prometheus

Used for exposure monitoring metrics.

• Parameter description

OpenTelemetry

OpenTelemetry can export tracing data to Jaeger, Zipkin.

• Parameter description

Parameter reference OpenTelemetry SDK Autoconfigure

Usage in ShardingSphere-Proxy

Using via a non-container environment

• Edit the startup script

Configure the absolute path of shardingsphere-agent-${latest.release.version}.jar to the start.sh startup script of shardingsphere proxy.

nohup java ${JAVA_OPTS} ${JAVA_MEM_OPTS} \
-javaagent:/xxxxx/agent/shardingsphere-agent-${latest.release.version}.jar \
-classpath ${CLASS_PATH} ${MAIN_CLASS} >> ${STDOUT_FILE} 2>&1 &

• Start ShardingSphere-Proxy

bin/start.sh

After startup, you can find the plugin info in the log of ShardingSphere-Proxy, Metric and Tracing data can be viewed through the configured monitoring address.

Use via container environment

• Assume that the following corresponding configurations have been completed locally.

  • Folder ./custom/agent/ that contains all files after unpacking ShardingSphere-Agent binary package
  • The folder containing the configuration files of ShardingSphere-Proxy such as server.yaml is ./custom/conf/
• At this point, the use of ShardingSphere-Agent can be configured through the environment variable JVM_OPT. Taking starting in the Docker Compose environment as an example, a reasonable docker-compose.yml example is as follows.

version: "3.8"
services:
  apache-shardingsphere-proxy:
    image: apache/shardingsphere-proxy:latest
    environment:
      JVM_OPTS: "-javaagent:/agent/shardingsphere-agent-${latest.release.version}.jar"
      PORT: 3308
    volumes:
      - ./custom/agent:/agent/
      - ./custom/conf:/opt/shardingsphere-proxy/conf/
    ports:
      - "13308:3308"

Metrics