Set up a standalone Pulsar locally

可以通过运行单机模式 Pulsar 进行本地开发和测试。 单机模式的 Pulsar 需有 Pulsar broker、必要的 ZooKeeper 和 BookKeeper 组件,在单独的 Java 虚拟机(JVM)中运行。

在生产环境中运行 Pulsar?

了解如何在生产环境中运行完整的 Pulsar,请参阅 Pulsar 的集群部署

安装单机模式 Pulsar

本教程将详细介绍安装的每一步。

系统要求

Currently, Pulsar is available for 64-bit macOS, Linux, and Windows. To use Pulsar, you need to install 64-bit JRE/JDK 8 or later versions.

提示

By default, Pulsar allocates 2G JVM heap memory to start. It can be changed in conf/pulsar_env.sh file under PULSAR_MEM. This is extra options passed into JVM.

Note

Broker is only supported on 64-bit JVM.

安装二进制版本 Pulsar

首先,通过以下方式下载二进制安装包:

下载好压缩文件后,解压缩并使用 cd 命令进入文件所在位置:

  1. $ tar xvfz apache-pulsar-2.8.0-bin.tar.gz
  2. $ cd apache-pulsar-2.8.0

软件包包含的内容:

Pulsar 二进制包包含下列目录:

目录内容
binPulsar’s command-line tools, such as pulsar and pulsar-admin.
confPulsar 的配置文件,包含 broker 配置ZooKeeper 配置等。
examplesJava JAR 包,包含 Pulsar Functions 的示例。
libPulsar使用到的 JAR 文件
licenses开源许可文件,.txt 格式,用于规范 Pulsar 代码库的各个组件。

运行 Pulsar 会立即生成以下目录。

目录内容
dataZooKeeper和BookKeeper使用的数据存储目录
instancesPulsar Functions 创建的组件。
logs安装时生成的日志文件

提示

想要使用内置连接器和分层存储卸载程序,可以按照以下说明进行安装:

不安装上述程序,则跳至下一步启动单机 Pulsar。 Pulsar can be successfully installed without installing bulitin connectors and tiered storage offloaders.

安装内置连接器(可选)

2.1.0-孵化 发布以后,Pulsar 发布了一个单独的二进制分布,其中包含所有的内置连接器。 要启用这些内置连接器,可以通过以下方式下载连接器安装包:

下载 nar 文件后,将其复制到 pulsar 目录下的 connectors 目录中。 比如,要下载 pulsar-io-aerospike-2.8.0.nar 连接器文件,则输入以下命令:

  1. $ mkdir connectors
  2. $ mv pulsar-io-aerospike-2.8.0.nar connectors
  3. $ ls connectors
  4. pulsar-io-aerospike-2.8.0.nar
  5. ...

Note

  • If you are running Pulsar in a bare metal cluster, make sure connectors tarball is unzipped in every pulsar directory of the broker (or in every pulsar directory of function-worker if you are running a separate worker cluster for Pulsar Functions).

  • If you are running Pulsar in Docker or deploying Pulsar using a docker image (e.g. K8S or DCOS), you can use the apachepulsar/pulsar-all image instead of the apachepulsar/pulsar image. apachepulsar/pulsar-all 镜像已经捆绑了所有内置 connector

安装分层存储卸载程序(可选)

提示

Since 2.2.0 release, Pulsar releases a separate binary distribution, containing the tiered storage offloaders. To enable tiered storage feature, follow the instructions below; otherwise skip this section.

要使用分层存储卸载程序,需要在所有 broker 节点上下载卸载程序的压缩文件,下载方式有以下几种:

下载卸载程序压缩包后,解压缩,并将卸载程序复制为 offloaders 存储在 pulsar 目录中:

  1. $ tar xvfz apache-pulsar-offloaders-2.8.0-bin.tar.gz
  2. // you will find a directory named `apache-pulsar-offloaders-2.8.0` in the pulsar directory
  3. // then copy the offloaders
  4. $ mv apache-pulsar-offloaders-2.8.0/offloaders offloaders
  5. $ ls offloaders
  6. tiered-storage-jcloud-2.8.0.nar

关于如何配置分层存储的详细信息,请参阅分层存储指南

Note

  • If you are running Pulsar in a bare metal cluster, make sure that offloaders tarball is unzipped in every broker’s pulsar directory.

  • If you are running Pulsar in Docker or deploying Pulsar using a docker image (e.g. K8S or DCOS), you can use the apachepulsar/pulsar-all image instead of the apachepulsar/pulsar image. apachepulsar/pulsar-all image has already bundled tiered storage offloaders.

启动单机模式 Pulsar

当有最新版本的本地副本后,可以使用 pulsar 命令启动本地集群(该命令存储在 bin 目录中),并且可以指定为以单机模式启动 Pulsar。

  1. $ bin/pulsar standalone

成功启动 Pulsar 后,可以看到如下所示的 INFO 级日志消息:

  1. 2017-06-01 14:46:29,192 - INFO - [main:WebSocketService@95] - Configuration Store cache started
  2. 2017-06-01 14:46:29,192 - INFO - [main:AuthenticationService@61] - Authentication is disabled
  3. 2017-06-01 14:46:29,192 - INFO - [main:WebSocketService@108] - Pulsar WebSocket Service started

提示

  • The service is running on your terminal, which is under your direct control. If you need to run other commands, open a new terminal window.
    You can also run the service as a background process using the pulsar-daemon start standalone command. For more information, see pulsar-daemon.

  • By default, there is no encryption, authentication, or authorization configured. Apache Pulsar 允许在没有任何授权的情况下从远程服务器访问。 请检查安全概述文件以确保部署安全。

  • When you start a local standalone cluster, a public/default namespace is created automatically. 自动创建的命名空间将用于开发用途。 所有Pulsar的topic主题都在命名空间中进行管理。 了解更多信息,参阅 Topics

使用单机模式 Pulsar

Pulsar 中有一个名为 pulsar-client 的 CLI 工具。 Pulsar-client 工具允许使用者在运行的集群中 consume 并 produce 消息到 Pulsar topic。

Consume 一条消息

first-subscription 订阅中 consume 一条消息到 my-topic 的命令如下所示:

  1. $ bin/pulsar-client consume my-topic -s "first-subscription"

如果消息成功发送到 topic,则会在 pulsar-client 日志中出现一个确认,如下所示:

  1. 09:56:55.566 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.MultiTopicsConsumerImpl - [TopicsConsumerFakeTopicNamee2df9] [first-subscription] Success subscribe new topic my-topic in topics consumer, partitions: 4, allTopicPartitionsNumber: 4

提示

As you have noticed that we do not explicitly create the my-topic topic, to which we consume the message. When you consume a message to a topic that does not yet exist, Pulsar creates that topic for you automatically. Producing a message to a topic that does not exist will automatically create that topic for you as well.

Produce 一条消息

向名称为 my-topic 的 topic 发送一条简单的消息 hello-pulsar,命令如下所示:

  1. $ bin/pulsar-client produce my-topic --messages "hello-pulsar"

如果消息成功发送到 topic,则会在 pulsar-client 日志中出现一个确认,如下所示:

  1. 13:09:39.356 [main] INFO org.apache.pulsar.client.cli.PulsarClientTool - 1 messages successfully produced

终止单机模式 Pulsar

使用 Ctrl+C 终止单机模式 Pulsar 的运行。

提示

If the service runs as a background process using the pulsar-daemon start standalone command, then use the pulsar-daemon stop standalone command to stop the service.

For more information, see pulsar-daemon.