Using CouchDB

This tutorial will describe the steps required to use CouchDB as the state database with Hyperledger Fabric. By now, you should be familiar with Fabric concepts and have explored some of the samples and tutorials.

Note

These instructions use the new Fabric chaincode lifecycle introduced in the Fabric v2.0 release. If you would like to use the previous lifecycle model to use indexes with chaincode, visit the v1.4 version of the Using CouchDB.

The tutorial will take you through the following steps:

  1. Enable CouchDB in Hyperledger Fabric

  2. Create an index

  3. Add the index to your chaincode folder

  4. Deploy the smart contract

  5. Query the CouchDB State Database

  6. Use best practices for queries and indexes

  7. Query the CouchDB State Database With Pagination

  8. Update an Index

  9. Delete an Index

For a deeper dive into CouchDB refer to CouchDB as the State Database and for more information on the Fabric ledger refer to the Ledger topic. Follow the tutorial below for details on how to leverage CouchDB in your blockchain network.

Throughout this tutorial, we will use the Asset transfer ledger queries sample as our use case to demonstrate how to use CouchDB with Fabric, including the execution of JSON queries against the state database. You should have completed the task Install Fabric and Fabric Samples.

Why CouchDB?

Fabric supports two types of peer state databases. LevelDB is the default state database embedded in the peer node. LevelDB stores chaincode data as simple key-value pairs and only supports key, key range, and composite key queries. CouchDB is an optional, alternate state database that allows you to model data on the ledger as JSON and issue rich queries against data values rather than the keys. The CouchDB support also allows you to deploy indexes with your chaincode to make queries more efficient and enable you to query large datasets.

In order to leverage the benefits of CouchDB, namely content-based JSON queries, your data must be modeled in JSON format. You must decide whether to use LevelDB or CouchDB before setting up your network. Switching a peer from using LevelDB to CouchDB is not supported due to data compatibility issues. All peers on the network must use the same database type. If you have a mix of JSON and binary data values, you can still use CouchDB, however the binary values can only be queried based on key, key range, and composite key queries.

Enable CouchDB in Hyperledger Fabric

CouchDB runs as a separate database process alongside the peer. There are additional considerations in terms of setup, management, and operations. A Docker image of CouchDB is available and we recommend that it be run on the same server as the peer. You will need to setup one CouchDB container per peer and update each peer container by changing the configuration found in core.yaml to point to the CouchDB container. The core.yaml file must be located in the directory specified by the environment variable FABRIC_CFG_PATH:

  • For Docker deployments, core.yaml is pre-configured and located in the peer container FABRIC_CFG_PATH folder. However, when using Docker environments, you can pass environment variables to override the core.yaml properties, for example CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS to set the CouchDB address.

  • For native binary deployments, core.yaml is included with the release artifact distribution.

Edit the stateDatabase section of core.yaml. Specify CouchDB as the stateDatabase and fill in the associated couchDBConfig properties. For more information, see CouchDB configuration.

Create an index

Why are indexes important?

Indexes allow a database to be queried without having to examine every row with every query, making them run faster and more efficiently. Normally, indexes are built for frequently occurring query criteria allowing the data to be queried more efficiently. To leverage the major benefit of CouchDB – the ability to perform rich queries against JSON data – indexes are not required, but they are strongly recommended for performance. Also, if sorting is required in a query, CouchDB requires an index that includes the sorted fields.

Note

JSON queries that do not have an index may work but will throw a warning in the peer log that the index was not found. However, if a rich query includes a sort specification, then an index on that field is required; otherwise, the query will fail and an error will be thrown.

To demonstrate building an index, we will use the data from the Asset transfer ledger queries sample. In this example, the Asset data structure is defined as:

  1. type Asset struct {
  2.         DocType        string `json:"docType"` //docType is used to distinguish the various types of objects in state database
  3.         ID             string `json:"ID"`      //the field tags are needed to keep case from bouncing around
  4.         Color          string `json:"color"`
  5.         Size           int    `json:"size"`
  6.         Owner          string `json:"owner"`
  7.         AppraisedValue int    `json:"appraisedValue"`
  8. }

In this structure, the attributes (docType, ID, color, size, owner, appraisedValue) define the ledger data associated with the asset. The attribute docType is a pattern that can be used in chaincode to differentiate different data types within the chaincode namespace that may need to be queried separately. When using CouchDB, each chaincode is represented as its own CouchDB database, that is, each chaincode has its own namespace for keys.

With respect to the Asset data structure, docType is used to identify that this JSON document represents an asset. Potentially there could be other JSON document types in the chaincode namespace. Any of the JSON fields can be used in CouchDB JSON queries.

When defining an index for use in chaincode queries, each one must be defined in its own text file with the extension *.json and the index definition must be formatted in the CouchDB index JSON format.

To define an index, three pieces of information are required:

  • fields: these are the fields to query

  • name: name of the index

  • type: always “json” in this context

For example, a simple index named foo-index for a field named foo.

  1. {
  2.     "index": {
  3.         "fields": ["foo"]
  4.     },
  5.     "name" : "foo-index",
  6.     "type" : "json"
  7. }

Optionally the design document attribute ddoc can be specified on the index definition. A design document is a CouchDB construct designed to contain indexes. Indexes can be grouped into design documents for efficiency but CouchDB recommends one index per design document.

Tip

When defining an index it is a good practice to include the ddoc attribute and value along with the index name. It is important to include this attribute to ensure that you can update the index later if needed. Also it gives you the ability to explicitly specify which index to use on a query.

Here is another example of an index definition from the Asset transfer ledger queries sample with the index name indexOwner using multiple fields docType and owner and includes the ddoc attribute:

  1. {
  2.   "index":{
  3.       "fields":["docType","owner"] // Names of the fields to be queried
  4.   },
  5.   "ddoc":"indexOwnerDoc", // (optional) Name of the design document in which the index will be created.
  6.   "name":"indexOwner",
  7.   "type":"json"
  8. }

In the example above, if the design document indexOwnerDoc does not already exist, it is automatically created when the index is deployed. An index can be constructed with one or more attributes specified in the list of fields and any combination of attributes can be specified. An attribute can exist in multiple indexes for the same docType. In the following example, index1 only includes the attribute owner, index2 includes the attributes owner and color, and index3 includes the attributes owner, color, and size. Also, notice each index definition has its own ddoc value, following the CouchDB recommended practice.

  1. {
  2.   "index":{
  3.       "fields":["owner"] // Names of the fields to be queried
  4.   },
  5.   "ddoc":"index1Doc", // (optional) Name of the design document in which the index will be created.
  6.   "name":"index1",
  7.   "type":"json"
  8. }
  10. {
  11.   "index":{
  12.       "fields":["owner", "color"] // Names of the fields to be queried
  13.   },
  14.   "ddoc":"index2Doc", // (optional) Name of the design document in which the index will be created.
  15.   "name":"index2",
  16.   "type":"json"
  17. }
  19. {
  20.   "index":{
  21.       "fields":["owner", "color", "size"] // Names of the fields to be queried
  22.   },
  23.   "ddoc":"index3Doc", // (optional) Name of the design document in which the index will be created.
  24.   "name":"index3",
  25.   "type":"json"
  26. }

If you want to add sorting to your indexing, add a sort type. Then you can query result at needed sort. Example below.

  1. {
  2.   "index":{
  3.     "fields":[
  4.        {
  5.           "owner":"desc"
  6.        },
  7.        {
  8.           "color":"desc"
  9.        },
  10.        {
  11.           "size":"desc"
  12.        }
  13.     ] // Names of the fields to be queried and sorted for descending.
  14.  },
  15.   "ddoc":"index3Doc",
  16.   "name":"index3",
  17.   "type":"json"
  18. }

Tip

Be aware if you want to have good performance in your query, put the most restrictive field in the first place to limit the results the most. In our case owner should be the first value in index fields. Because the owner carries a unique user identifier.

In general, you should model index fields to match the fields that will be used in query filters and sorts. For more details on building an index in JSON format refer to the CouchDB documentation.

Add the index to your chaincode folder

Once you finalize an index, you need to package it with your chaincode for deployment by placing it in the appropriate metadata folder. You can package and install the chaincode using the peer lifecycle chaincode commands. The JSON index files must be located under the path META-INF/statedb/couchdb/indexes which is located inside the directory where the chaincode resides.

The Asset transfer ledger queries sample below illustrates how the index is packaged with the chaincode.

Marbles Chaincode Index Package

This sample includes one index named indexOwnerDoc, to support queries by asset owner:

  1. {"index":{"fields":["docType","owner"]},"ddoc":"indexOwnerDoc", "name":"indexOwner","type":"json"}

Start the network

Try it yourself

We will bring up the Fabric test network and use it to deploy the asset transfer ledger queries chaincode. Use the following command to navigate to the test-network directory in the Fabric samples:

  1. cd fabric-samples/test-network

For this tutorial, we want to operate from a known initial state. The following command will kill any active or stale Docker containers and remove previously generated artifacts:

  1. ./network.sh down

If you have not run through the tutorial before, you will need to vendor the chaincode dependencies before we can deploy it to the network. Run the following commands:

  1. cd ../asset-transfer-ledger-queries/chaincode-go
  2. GO111MODULE=on go mod vendor
  3. cd ../../test-network

From the test-network directory, deploy the test network with CouchDB with the following command:

  1. ./network.sh up createChannel -s couchdb

This will create two fabric peer nodes that use CouchDB as the state database. It will also create one ordering node and a single channel named mychannel.

Deploy the smart contract

You can use the test network script to deploy the asset transfer ledger queries smart contract to the channel. Run the following command to deploy the smart contract to mychannel:

  1. ./network.sh deployCC -ccn ledger -ccp ../asset-transfer-ledger-queries/chaincode-go/ -ccl go -ccep "OR('Org1MSP.peer','Org2MSP.peer')"

Note that we are using the -ccep flag to deploy the smart contract with an endorsement policy of “OR(‘Org1MSP.peer’,’Org2MSP.peer’)”. This allows either organization to create an asset without receiving an endorsement from the other organization.

Verify index was deployed

Indexes will be deployed to each peer’s CouchDB state database once the chaincode has been installed on the peer and deployed to the channel. You can verify that the CouchDB index was created successfully by examining the peer log in the Docker container.

Try it yourself

To view the logs in the peer Docker container, open a new Terminal window and run the following command to grep for message confirmation that the index was created.

  1. docker logs peer0.org1.example.com  2>&1 | grep "CouchDB index"

You should see a result that looks like the following:

  1. [couchdb] createIndex -> INFO 072 Created CouchDB index [indexOwner] in state database [mychannel_ledger] using design document [_design/indexOwnerDoc]

Query the CouchDB State Database

Now that the index has been defined in the JSON file and deployed alongside the chaincode, chaincode functions can execute JSON queries against the CouchDB state database.

Specifying an index name on a query is optional. If not specified, and an index already exists for the fields being queried, the existing index will be automatically used.

Tip

It is a good practice to explicitly include an index name on a query using the use_index keyword. Without it, CouchDB may pick a less optimal index. Also CouchDB may not use an index at all and you may not realize it, at the low volumes during testing. Only upon higher volumes you may realize slow performance because CouchDB is not using an index.

Build the query in chaincode

You can perform JSON queries against the data on the ledger using queries defined within your chaincode. The Asset transfer ledger queries sample includes two JSON query functions:

  • QueryAssets

    Example of an ad hoc JSON query. This is a query where a selector JSON query string can be passed into the function. This query would be useful to client applications that need to dynamically build their own queries at runtime. For more information on query selectors refer to CouchDB selector syntax.

  • QueryAssetsByOwner

    Example of a parameterized query where the query is defined in the chaincode but allows a query parameter to be passed in. In this case the function accepts a single argument, the asset owner. It then queries the state database for JSON documents matching the docType of “asset” and the owner id using the JSON query syntax.

Run the query using the peer command

In absence of a client application, we can use the peer command to test the queries defined in the chaincode. We will use the peer chaincode query command to use the Assets index indexOwner and query for all assets owned by “tom” using the QueryAssets function.

Try it yourself

Before querying the database, we should add some data. Run the following command as Org1 to create an asset owned by “tom”:

  1. export CORE_PEER_TLS_ENABLED=true
  2. export CORE_PEER_LOCALMSPID="Org1MSP"
  3. export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
  4. export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
  5. export CORE_PEER_ADDRESS=localhost:7051
  6. peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n ledger -c '{"Args":["CreateAsset","asset1","blue","5","tom","35"]}'

Next, query for all assets owned by tom:

  1. // Rich Query with index name explicitly specified:
  2. peer chaincode query -C mychannel -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}"]}'

Delving into the query command above, there are three arguments of interest:

  • QueryAssets

Name of the function in the Assets chaincode. As you can see in the chaincode function below, QueryAssets() calls getQueryResultForQueryString(), which then passes the queryString to the getQueryResult() shim API that executes the JSON query against the state database.

  1. func (t *SimpleChaincode) QueryAssets(ctx contractapi.TransactionContextInterface, queryString string) ([]*Asset, error) {
  2.         return getQueryResultForQueryString(ctx, queryString)
  3. }
  • {"selector":{"docType":"asset","owner":"tom"}

This is an example of an ad hoc selector string which query for all documents of type asset where the owner attribute has a value of tom.

  • "use_index":["_design/indexOwnerDoc", "indexOwner"]

Specifies both the design doc name indexOwnerDoc and index name indexOwner. In this example the selector query explicitly includes the index name, specified by using the use_index keyword. Recalling the index definition above Create an index, it contains a design doc, "ddoc":"indexOwnerDoc". With CouchDB, if you plan to explicitly include the index name on the query, then the index definition must include the ddoc value, so it can be referenced with the use_index keyword.

The query runs successfully and the index is leveraged with the following results:

  1. [{"docType":"asset","ID":"asset1","color":"blue","size":5,"owner":"tom","appraisedValue":35}]

Use best practices for queries and indexes

Queries that use indexes will complete faster, without having to scan the full database in CouchDB. Understanding indexes will allow you to write your queries for better performance and help your application handle larger amounts of data.

It is also important to plan the indexes you install with your chaincode. You should install only a few indexes per chaincode that support most of your queries. Adding too many indexes, or using an excessive number of fields in an index, will degrade the performance of your network. This is because the indexes are updated after each block is committed.

The examples in this section will help demonstrate how queries use indexes and what type of queries will have the best performance. Remember the following when writing your queries:

  • All fields in the index must also be in the selector or sort sections of your query for the index to be used.

  • More complex queries will have a lower performance and will be less likely to use an index.

  • You should avoid operators that will result in a full table scan or a full index scan such as $or, $in and $regex.

In the previous section of this tutorial, you issued the following query against the assets chaincode:

  1. // Example one: query fully supported by the index
  2. export CHANNEL_NAME=mychannel
  3. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'

The asset transfer ledger queries chaincode was installed with the indexOwnerDoc index:

  1. {"index":{"fields":["docType","owner"]},"ddoc":"indexOwnerDoc", "name":"indexOwner","type":"json"}

Notice that both the fields in the query, docType and owner, are included in the index, making it a fully supported query. As a result this query will be able to use the data in the index, without having to search the full database. Fully supported queries such as this one will return faster than other queries from your chaincode.

If you add extra fields to the query above, it will still use the index. However, the query will additionally have to scan the database for the extra fields, resulting in a longer response time. As an example, the query below will still use the index, but will take a longer time to return than the previous example.

  1. // Example two: query fully supported by the index with additional data
  2. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\",\"color\":\"blue\"}, \"use_index\":[\"/indexOwnerDoc\", \"indexOwner\"]}"]}'

A query that does not include all fields in the index will have to scan the full database instead. For example, the query below searches for the owner, without specifying the type of item owned. Since the indexOwnerDoc contains both the owner and docType fields, this query will not be able to use the index.

  1. // Example three: query not supported by the index
  2. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"owner\":\"tom\"}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'

In general, more complex queries will have a longer response time, and have a lower chance of being supported by an index. Operators such as $or, $in, and $regex will often cause the query to scan the full index or not use the index at all.

As an example, the query below contains an $or term that will search for every asset and every item owned by tom.

  1. // Example four: query with $or supported by the index
  2. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"$or\":[{\"docType\":\"asset\"},{\"owner\":\"tom\"}]}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'

This query will still use the index because it searches for fields that are included in indexOwnerDoc. However, the $or condition in the query requires a scan of all the items in the index, resulting in a longer response time.

Below is an example of a complex query that is not supported by the index.

  1. // Example five: Query with $or not supported by the index
  2. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssets", "{\"selector\":{\"$or\":[{\"docType\":\"asset\",\"owner\":\"tom\"},{\"color\":\"yellow\"}]}, \"use_index\":[\"indexOwnerDoc\", \"indexOwner\"]}"]}'

The query searches for all assets owned by tom or any other items that are yellow. This query will not use the index because it will need to search the entire table to meet the $or condition. Depending the amount of data on your ledger, this query will take a long time to respond or may timeout.

While it is important to follow best practices with your queries, using indexes is not a solution for collecting large amounts of data. The blockchain data structure is optimized to validate and confirm transactions and is not suited for data analytics or reporting. If you want to build a dashboard as part of your application or analyze the data from your network, the best practice is to query an off chain database that replicates the data from your peers. This will allow you to understand the data on the blockchain without degrading the performance of your network or disrupting transactions.

You can use block or chaincode events from your application to write transaction data to an off-chain database or analytics engine. For each block received, the block listener application would iterate through the block transactions and build a data store using the key/value writes from each valid transaction’s rwset. The Peer channel-based event services provide replayable events to ensure the integrity of downstream data stores. For an example of how you can use an event listener to write data to an external database, visit the Off chain data sample in the Fabric Samples.

Query the CouchDB State Database With Pagination

When large result sets are returned by CouchDB queries, a set of APIs is available which can be called by chaincode to paginate the list of results. Pagination provides a mechanism to partition the result set by specifying a pagesize and a start point – a bookmark which indicates where to begin the result set. The client application iteratively invokes the chaincode that executes the query until no more results are returned. For more information refer to this topic on pagination with CouchDB.

We will use the Asset transfer ledger queries sample function QueryAssetsWithPagination to demonstrate how pagination can be implemented in chaincode and the client application.

  • QueryAssetsWithPagination –

    Example of an ad hoc JSON query with pagination. This is a query where a selector string can be passed into the function similar to the above example. In this case, a pageSize is also included with the query as well as a bookmark.

In order to demonstrate pagination, more data is required. This example assumes that you have already added asset1 from above. Run the following commands in the peer container to create four more assets owned by “tom”, to create a total of five assets owned by “tom”:

Try it yourself

  1. export CORE_PEER_LOCALMSPID="Org1MSP"
  2. export CORE_PEER_TLS_ROOTCERT_FILE=${PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
  3. export CORE_PEER_MSPCONFIGPATH=${PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
  4. export CORE_PEER_ADDRESS=localhost:7051
  5. peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n ledger -c '{"Args":["CreateAsset","asset2","yellow","5","tom","35"]}'
  6. peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n ledger -c '{"Args":["CreateAsset","asset3","green","6","tom","20"]}'
  7. peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n ledger -c '{"Args":["CreateAsset","asset4","purple","7","tom","20"]}'
  8. peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls --cafile  "${PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem" -C mychannel -n ledger -c '{"Args":["CreateAsset","asset5","blue","8","tom","40"]}'

In addition to the arguments for the query in the previous example, QueryAssetsWithPagination adds pagesize and bookmark. PageSize specifies the number of records to return per query. The bookmark is an “anchor” telling couchDB where to begin the page. (Each page of results returns a unique bookmark.)

  • QueryAssetsWithPagination

As you can see in the chaincode function below, QueryAssetsWithPagination() calls getQueryResultForQueryStringWithPagination(), which then passes the queryString as well as the bookmark and pagesize to the GetQueryResultWithPagination() shim API that executes the paginated JSON query against the state database.

  1. func (t *SimpleChaincode) QueryAssetsWithPagination(
  2.         ctx contractapi.TransactionContextInterface,
  3.         queryString,
  4.         pageSize int,
  5.         bookmark string) (*PaginatedQueryResult, error) {
  7.         return getQueryResultForQueryStringWithPagination(ctx, queryString, int32(pageSize), bookmark)
  8. }

The following example is a peer command which calls QueryAssetsWithPagination with a pageSize of 3 and no bookmark specified.

Tip

When no bookmark is specified, the query starts with the “first” page of records.

Try it yourself

  1. // Rich Query with index name explicitly specified and a page size of 3:
  2. peer chaincode query -C mychannel -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","3",""]}'

The following response is received (carriage returns added for clarity), three of the five assets are returned because the pagsize was set to 3:

  1. {
  2.   "records":[
  3.     {"docType":"asset","ID":"asset1","color":"blue","size":5,"owner":"tom","appraisedValue":35},
  4.     {"docType":"asset","ID":"asset2","color":"yellow","size":5,"owner":"tom","appraisedValue":35},
  5.     {"docType":"asset","ID":"asset3","color":"green","size":6,"owner":"tom","appraisedValue":20}],
  6.   "fetchedRecordsCount":3,
  7.   "bookmark":"g1AAAABJeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqzJRYXp5YYg2Q5YLI5IPUgSVawJIjFXJKfm5UFANozE8s"
  8. }

Note

Bookmarks are uniquely generated by CouchDB for each query and represent a placeholder in the result set. Pass the returned bookmark on the subsequent iteration of the query to retrieve the next set of results.

The following is a peer command to call QueryAssetsWithPagination with a pageSize of 3. Notice this time, the query includes the bookmark returned from the previous query.

Try it yourself

  1. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","3","g1AAAABJeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqzJRYXp5YYg2Q5YLI5IPUgSVawJIjFXJKfm5UFANozE8s"]}'

The following response is received (carriage returns added for clarity). The last two records are retrieved:

  1. {
  2.   "records":[
  3.     {"docType":"asset","ID":"asset4","color":"purple","size":7,"owner":"tom","appraisedValue":20},
  4.     {"docType":"asset","ID":"asset5","color":"blue","size":8,"owner":"tom","appraisedValue":40}],
  5.   "fetchedRecordsCount":2,
  6.   "bookmark":"g1AAAABJeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqzJRYXp5aYgmQ5YLI5IPUgSVawJIjFXJKfm5UFANqBE80"
  7. }

The returned bookmark marks the end of the result set. If we attempt to query with this bookmark, no more results will get returned.

Try it yourself

  1. peer chaincode query -C $CHANNEL_NAME -n ledger -c '{"Args":["QueryAssetsWithPagination", "{\"selector\":{\"docType\":\"asset\",\"owner\":\"tom\"}, \"use_index\":[\"_design/indexOwnerDoc\", \"indexOwner\"]}","3","g1AAAABJeJzLYWBgYMpgSmHgKy5JLCrJTq2MT8lPzkzJBYqzJRYXp5aYgmQ5YLI5IPUgSVawJIjFXJKfm5UFANqBE80"]}'

For an example of how a client application can iterate over JSON query result sets using pagination, search for the getQueryResultForQueryStringWithPagination function in the Asset transfer ledger queries sample.

Range Query Pagination

Bookmarks are also returned by the GetStateByRangeWithPagination shim API so that applications can page through range query results when using either LevelDB or CouchDB state database. The returned bookmark represents the next startKey that can be used to retrieve the next page of range query results. Once the results are exhausted the returned bookmark will be an empty string. If an endKey was specified in the range query and the results are exhausted, the returned bookmark will be the passed endKey when using CouchDB, and will be an empty string when using LevelDB.

For an example of how a client application can iterate over range query result sets using pagination, search for the GetAssetsByRangeWithPagination function in the Asset transfer ledger queries sample.

Update an Index

It may be necessary to update an index over time. The same index may exist in subsequent versions of the chaincode that gets installed. In order for an index to be updated, the original index definition must have included the design document ddoc attribute and an index name. To update an index definition, use the same index name but alter the index definition. Simply edit the index JSON file and add or remove fields from the index. Fabric only supports the index type JSON. Changing the index type is not supported. The updated index definition gets redeployed to the peer’s state database when the chaincode definition is committed to the channel. Changes to the index name or ddoc attributes will result in a new index being created and the original index remains unchanged in CouchDB until it is removed.

Note

If the state database has a significant volume of data, it will take some time for the index to be re-built, during which time chaincode invokes that issue queries may fail or timeout.

Iterating on your index definition

If you have access to your peer’s CouchDB state database in a development environment, you can iteratively test various indexes in support of your chaincode queries. Any changes to chaincode though would require redeployment. Use the CouchDB Fauxton interface or a command line curl utility to create and update indexes.

Note

The Fauxton interface is a web UI for the creation, update, and deployment of indexes to CouchDB. If you want to try out this interface, there is an example of the format of the Fauxton version of the index in Assets sample. If you have deployed the test network with CouchDB, the Fauxton interface can be loaded by opening a browser and navigating to http://localhost:5984/_utils.

Alternatively, if you prefer not use the Fauxton UI, the following is an example of a curl command which can be used to create the index on the database mychannel_ledger:

  1. // Index for docType, owner.
  2. // Example curl command line to define index in the CouchDB channel_chaincode database
  3.  curl -i -X POST -H "Content-Type: application/json" -d \
  4.         "{\"index\":{\"fields\":[\"docType\",\"owner\"]}, \
  5.           \"name\":\"indexOwner\", \
  6.           \"ddoc\":\"indexOwnerDoc\", \
  7.           \"type\":\"json\"}" http://username:password@hostname:port/mychannel_ledger/_index

Note

If you are using the test network configured with CouchDB, replace hostname:port with localhost:5984 and username:password with admin:adminpw.

Delete an Index

Index deletion is not managed by Fabric tooling. If you need to delete an index, manually issue a curl command against the database or delete it using the Fauxton interface.

The format of the curl command to delete an index would be:

  1. curl -X DELETE http://admin:adminpw@localhost:5984/{database_name}/_index/{design_doc}/json/{index_name} -H  "accept: */*" -H  "Host: localhost:5984"

To delete the index used in this tutorial, the curl command would be:

  1. curl -X DELETE http://admin:adminpw@localhost:5984/mychannel_ledger/_index/indexOwnerDoc/json/indexOwner -H  "accept: */*" -H  "Host: localhost:5984"

Clean up

When you are finished using the tutorial, you can bring down the test network using network.sh script.

  1. ./network.sh down

This command will bring down the CAs, peers, and ordering node of the network that we created. Note that all of the data on the ledger will be lost. If you want to go through the tutorial again, you will start from a clean initial state.