Overview of the Metadata Center

Overview of the Metadata Center

The Metadata Center provides access to two types of metadata in Dubbo:

    1. Address discovery metadata, used for application-level service discovery.
    1. Service operation and maintenance metadata, used for peripheral operation and maintenance systems such as visualization consoles for service queries, testing, etc.

1 Address Discovery Metadata

Dubbo3 introduces the application-level service discovery mechanism to address performance issues in heterogeneous microservice systems and large-scale cluster practices. Application-level service discovery will fully replace the interface-level service discovery of version 2.x. To maintain Dubbo’s usability oriented towards services/interfaces and flexibility in service governance, Dubbo has built a metadata mechanism around application-level service discovery, specifically interface-application mapping relations and interface configuration metadata.

1.1 Interface-Application Mapping Relation

Dubbo has always been capable of precise address discovery, subscribing only to the list of services and addresses declared by the Consumer. Achieving precise address subscription in the application-level service discovery model is not easy, as the Dubbo Consumer only declares the list of interfaces to consume. Therefore, the Consumer needs to convert the interface into the Provider’s application name for accurate service subscription.

For this, Dubbo must maintain this mapping of interface name -> application name in the metadata center, reported actively by the provider during startup in Dubbo3. The mapping relation of interfaces (service names) to applications (Provider application names) can be one-to-many, where one service name may correspond to multiple application names.

Taking Zookeeper as an example, the mapping is stored in the following location:

  1. $ ./zkCli.sh
  2. $ get /dubbo/mapping/org.apache.dubbo.demo.DemoService
  3. $ demo-provider,two-demo-provider,dubbo-demo-annotation-provider

① The node path is /dubbo/mapping/{interface name}

② Multiple application names are separated by a comma ,

1.2 Interface Configuration Metadata

Interface-level configuration metadata supplements address discovery. Unlike address discovery models like Spring Cloud, which can only sync IP and port information, Dubbo’s service discovery mechanism can sync interface lists, interface definitions, and interface-level parameter configurations. This part of the content is calculated based on the application’s current information and interface information. Moreover, from a performance perspective, it generates metadata revisions for metadata aggregation across different machine instances.

Metadata reporting can be disabled by setting dubbo.metadata-report.report-metadata=false.

Using Zookeeper as an example, the interface configuration metadata is stored in the following location. If multiple instances generate the same revision, they will eventually share the same metadata configuration:

/dubbo/metadata/{application name}/{revision}

  1. [zk: localhost:2181(CONNECTED) 33] get /dubbo/metadata/demo-provider/da3be833baa2088c5f6776fb7ab1a436
  1. {
  2.     "app":"demo-provider",
  3.     "revision":"da3be833baa2088c5f6776fb7ab1a436",
  4.     "services":{
  5.         "org.apache.dubbo.demo.DemoService:dubbo":{
  6.             "name":"org.apache.dubbo.demo.DemoService",
  7.             "protocol":"dubbo",
  8.             "path":"org.apache.dubbo.demo.DemoService",
  9.             "params":{
  10.                 "side":"provider",
  11.                 "release":"",
  12.                 "methods":"sayHello,sayHelloAsync",
  13.                 "deprecated":"false",
  14.                 "dubbo":"2.0.2",
  15.                 "pid":"38298",
  16.                 "interface":"org.apache.dubbo.demo.DemoService",
  17.                 "service-name-mapping":"true",
  18.                 "timeout":"3000",
  19.                 "generic":"false",
  20.                 "metadata-type":"remote",
  21.                 "delay":"5000",
  22.                 "application":"demo-provider",
  23.                 "dynamic":"true",
  24.                 "REGISTRY_CLUSTER":"registry1",
  25.                 "anyhost":"true",
  26.                 "timestamp":"1626887121829"
  27.             }
  28.         },
  29.         "org.apache.dubbo.demo.RestDemoService:1.0.0:rest":{
  30.             "name":"org.apache.dubbo.demo.RestDemoService",
  31.             "version":"1.0.0",
  32.             "protocol":"rest",
  33.             "path":"org.apache.dubbo.demo.RestDemoService",
  34.             "params":{
  35.                 "side":"provider",
  36.                 "release":"",
  37.                 "methods":"getRemoteApplicationName,sayHello,hello,error",
  38.                 "deprecated":"false",
  39.                 "dubbo":"2.0.2",
  40.                 "pid":"38298",
  41.                 "interface":"org.apache.dubbo.demo.RestDemoService",
  42.                 "service-name-mapping":"true",
  43.                 "version":"1.0.0",
  44.                 "timeout":"5000",
  45.                 "generic":"false",
  46.                 "revision":"1.0.0",
  47.                 "metadata-type":"remote",
  48.                 "delay":"5000",
  49.                 "application":"demo-provider",
  50.                 "dynamic":"true",
  51.                 "REGISTRY_CLUSTER":"registry1",
  52.                 "anyhost":"true",
  53.                 "timestamp":"1626887120943"
  54.             }
  55.         }
  56.     }
  57. }

2 Service Operation and Maintenance Metadata

The service operation and maintenance metadata reported by Dubbo is typically used by various operation and maintenance systems, such as service testing, gateway data mapping, and service static dependency analysis.

Various third-party systems can directly read and use this part of the data; specific integration methods can refer to the several third-party systems mentioned in this chapter.

Metadata reporting can be disabled by setting dubbo.metadata-report.report-definition=false.

2.1 Metadata Reported by Providers

The metadata content stored on the provider side is as follows:

  1. {
  2.  "parameters": {
  3.   "side": "provider",
  4.   "methods": "sayHello",
  5.   "dubbo": "2.0.2",
  6.   "threads": "100",
  7.   "interface": "org.apache.dubbo.samples.metadatareport.configcenter.api.AnnotationService",
  8.   "threadpool": "fixed",
  9.   "version": "1.1.1",
  10.   "generic": "false",
  11.   "revision": "1.1.1",
  12.   "valid": "true",
  13.   "application": "metadatareport-configcenter-provider",
  14.   "default.timeout": "5000",
  15.   "group": "d-test",
  16.   "anyhost": "true"
  17.  },
  18.  "canonicalName": "org.apache.dubbo.samples.metadatareport.configcenter.api.AnnotationService",
  19.  "codeSource": "file:/Users/cvictory/workspace/work-mw/dubbo-samples/dubbo-samples-metadata-report/dubbo-samples-metadata-report-configcenter/target/classes/",
  20.  "methods": [{
  21.   "name": "sayHello",
  22.   "parameterTypes": ["java.lang.String"],
  23.   "returnType": "java.lang.String"
  24.  }],
  25.  "types": [{
  26.   "type": "java.lang.String",
  27.   "properties": {
  28.    "value": {
  29.     "type": "char[]"
  30.    },
  31.    "hash": {
  32.     "type": "int"
  33.    }
  34.   }
  35.  }, {
  36.   "type": "int"
  37.  }, {
  38.   "type": "char"
  39.  }]
  40. }

① parameters are details of service configuration and parameters.

② types contains service definition information.

Metadata Reported by Consumers:
  1. {
  2.  "valid": "true",
  3.  "side": "consumer",
  4.  "application": "metadatareport-configcenter-consumer",
  5.  "methods": "sayHello",
  6.  "default.timeout": "6666",
  7.  "dubbo": "2.0.2",
  8.  "interface": "org.apache.dubbo.samples.metadatareport.configcenter.api.AnnotationService",
  9.  "version": "1.1.1",
  10.  "revision": "1.1.1",
  11.  "group": "d-test"
  12. }

Configuration metadata used when the Consumer process subscribes.

3 Metadata Reporting Work Mechanism

Metadata reporting is an asynchronous process by default. To better control asynchronous behavior, the metadata reporting component (metadata-report) exposes two configuration items:

  • Failure retry
  • Daily scheduled retry refresh

3.1 retrytimes Failure Retry

Failure retry can be set through retrytimes (number of retries, default 100) and retryperiod (retry period, default 3000ms).

3.2 Scheduled Refresh

Enabled by default, can be disabled by setting cycleReport=false.

3.3 Complete Configuration Items

  1. dubbo.metadata-report.address=zookeeper://127.0.0.1:2181
  2. dubbo.metadata-report.username=xxx         ## optional
  3. dubbo.metadata-report.password=xxx         ## optional
  4. dubbo.metadata-report.retry-times=30       ## optional, default is 100
  5. dubbo.metadata-report.retry-period=5000    ## optional, default is 3000
  6. dubbo.metadata-report.cycle-report=false   ## optional, default is true
  7. dubbo.metadata-report.sync.report=false    ## optional, default is false

If the metadata address (dubbo.metadata-report.address) is not configured, it will judge whether the protocol of the registry center supports the metadata center. If supported, it will use the registry center’s address as the metadata center.

Please refer to metadata-report for details on how to extend custom third-party implementations.

Feedback

Was this page helpful?

Yes No

Last modified September 30, 2024: Update & Translate Overview Docs (#3040) (d37ebceaea7)