Consume data

This page shows how to consume data from HStreamDB using Java SDK.

Prerequisites

Make sure you have HStreamDB running and accessible.

Concepts

A client can consume data from a consumer object. A consumer object will join a consumer group with a subscriptionID, and then the client can fetch data from the subscribed stream.

Each consumer object will contain a RawRecordReceiver and a HRecordReceiver, so users can consume raw records or HRecords according to their needs.

When you received a record from server, you should use responder.ack() to send an ack response to server. HStreamDB now support at-least-once consume, if you don’t send acks, records will be retransmitted by the server after ackTimeoutSecond.

Consume Records

You can consume records like this:

  2. // first, create a subscription for the stream
  3. Subscription subscription =
  4.     Subscription
  5.         .newBuilder()
  6.         .subscription("my_subscription")
  7.         .stream("my_stream")
  8.         .offset(new SubscriptionOffset(SubscriptionOffset.SpecialOffset.LATEST))
  9.         .ackTimeoutSeconds(600)
  10.         .build();
  11. client.createSubscription(subscription);
  13. // second, create a consumer attach to the subscription
  14. Consumer consumer =
  15.     client
  16.         .newConsumer()
  17.         .subscription("my_subscription")
  18.         .rawRecordReceiver(
  19.             ((receivedRawRecord, responder) -> {
  20.                 System.out.println(receivedRawRecord.getRecordId());
  21.                 responder.ack();
  22.             }))
  23.         .build();
  25. // third, start the consumer
  26. consumer.startAsync().awaitRunning();
  • the example use rawRecordReceiver() to consume raw records, if you want to consume HReacord, just use hRecordReceiver() instead.