Kubernetes

Prerequisites

• Kubernetes 1.10+
• Helm 3.0+
• InLong Helm Chart
• Apache Flink 1.13.x or 1.15.x
• Apache Pulsar 2.8.x
• A dynamic provisioner for the PersistentVolumes(production environment)

Usage

Configure

• configure the external.pulsar for Apache Pulsar info in values.yaml
• configure the external.flink for Apache Flink info in values.yaml

Install

If the namespace named inlong does not exist, create it first by running:

kubectl create namespace inlong

To install the chart with a namespace named inlong, try:

helm upgrade inlong --install -n inlong ./

Access InLong Dashboard

If ingress.enabled in values.yaml is set to true, you just access http://${ingress.host}/dashboard in browser.

Otherwise, when dashboard.service.type is set to ClusterIP, you need to execute the port-forward command like:

export DASHBOARD_POD_NAME=$(kubectl get pods -l "component=dashboard" -o jsonpath="{.items[0].metadata.name}" -n inlong)
export DASHBOARD_CONTAINER_PORT=$(kubectl get pod $DASHBOARD_POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}" -n inlong)
kubectl port-forward $DASHBOARD_POD_NAME 80:$DASHBOARD_CONTAINER_PORT --address='0.0.0.0' -n inlong

And then access http://127.0.0.1:80, and use the following default account to log in:

User: admin
Password: inlong

note

If the error of unable to do port forwarding: socat not found appears, you need to install socat at first.

Or when dashboard.service.type is set to NodePort, you need to execute the following commands:

export DASHBOARD_NODE_IP=$(kubectl get nodes -o jsonpath="{.items[0].status.addresses[0].address}" -n inlong)
export DASHBOARD_NODE_PORT=$(kubectl get svc inlong-dashboard -o jsonpath="{.spec.ports[0].nodePort}" -n inlong)

And then access http://$DASHBOARD_NODE_IP:$DASHBOARD_NODE_PORT

When dashboard.service.type is set to LoadBalancer, you need to execute the following command:

export DASHBOARD_SERVICE_IP=$(kubectl get svc inlong-dashboard --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}"  -n inlong)

And then access http://$DASHBOARD_SERVICE_IP:30080

note

It may take a few minutes for the LoadBalancer IP to be available. You can check the status by running kubectl get svc inlong-dashboard -n inlong -w

The default username is admin and the default password is inlong. You can access the InLong Dashboard through them.

Configuration

The configuration file is values.yaml, and the following tables lists the configurable parameters of InLong and their default values.

The optional components include agent, audit, dashboard, dataproxy, manager, tubemq-manager, tubemq-master, tubemq-broker, zookeeper and mysql.

Uninstall

To uninstall the release, try:

helm uninstall inlong -n inlong

The above command removes all the Kubernetes components except the PVC associated with the chart, and deletes the release. You can delete all PVC if any persistent volume claims used, it will lose all data.

kubectl delete pvc -n inlong --all

Note: Deleting the PVC also delete all data. Please be cautious before doing it.

Development

A Kubernetes cluster with helm is required before development. But it doesn’t matter if you don’t have one, the kind is recommended. It runs a local Kubernetes cluster in Docker container. Therefore, it requires very little time to up and stop the Kubernetes node.

Quick start with kind

You can install kind by following the Quick Start section of their official documentation.

After installing kind, you can create a Kubernetes cluster with the kind.yml, try:

kind create cluster --config kind.yml

To specify another image use the --image flag – kind create cluster --image=..... Using a different image allows you to change the Kubernetes version of the created cluster. To find images suitable for a given release currently you should check the release notes for your given kind version (check with kind version) where you’ll find a complete listing of images created for a kind release.

After installing kind, you can interact with the created cluster, try:

kubectl cluster-info --context kind-inlong-cluster

Now, you have a running Kubernetes cluster for local development.

Install Helm

Please follow the installation guide in the official documentation to install Helm.

Install the chart

To create the namespace and Install the chart, try:

kubectl create namespace inlong
helm upgrade inlong --install -n inlong ./

It may take a few minutes. Confirm the pods are up:

watch kubectl get pods -n inlong -o wide

Develop and debug

Follow the template debugging guide in the official documentation to debug your chart.

Besides, you can save the rendered templates by:

helm template ./ --output-dir ./result

Then, you can check the rendered templates in the result directory.

Troubleshooting

We’ve done our best to make these charts as seamless as possible, but occasionally there are circumstances beyond our control. We’ve collected tips and tricks for troubleshooting common issues. Please examine these first before raising an issue, and feel free to make a Pull Request!