Docker Software

Run battery-included softwares & tools with Docker

You can use Docker to deploy and launch software applications quickly. You can directly access the PostgreSQL/Redis database deployed on the host in the container using the connection string.

  • PgAdmin4: A GUI tool for managing PostgreSQL instances.
  • PGWeb: A tool automatically generates back-end API services based on PG database schema.
  • PostgREST: A tool to automatically generate backend API services based on PG database schema.
  • ByteBase: A GUI tool for making PostgreSQL schema changes.
  • Jupyter Lab: A battery-included Python lab environment for data analysis and processing.

You can also use Docker to execute some battery-included command tools.

  • SchemaSPY: Generates detailed visual reports of database schemas.
  • Pgbadger: Generate database log report.

You can also use Docker to pull up some battery-included open-source SaaS services.

  • Gitlab: open-source code hosting platform.
  • Habour: open-source mirror repo
  • Jira: open-source project management platform.
  • Confluence: open-source knowledge hosting platform.
  • Odoo: open-source ERP
  • Mastodon: PG-based social network
  • Discourse: open-source forum based on PG and Redis

Add Upstream to Nginx

Most of the software described in this article provides a web interface to the public. While it can be accessed directly via IP: Port, we recommend using a domain name and unifying access from the Nginx proxy. Use the following configuration and commands to register a new service with Nginx.

  1. # Add new Nginx services definition
  2. nginx_upstreams:
  3.   - { name: kong         , domain: api.pigsty , endpoint: "127.0.0.1:8880"   } #== v optional ==#
  4.   - { name: pgadmin      , domain: adm.pigsty , endpoint: "127.0.0.1:8885"   }
  5.   - { name: pgweb        , domain: cli.pigsty , endpoint: "127.0.0.1:8886"   }
  6.   - { name: bytebase     , domain: ddl.pigsty , endpoint: "127.0.0.1:8887"   }
  7.   - { name: jupyter      , domain: lab.pigsty , endpoint: "127.0.0.1:8888"   }
  9. ./infra.yml -t nginx_config,nginx_restart    # refresh nginx conf, and restart to apply

PgAdmin4

PGAdmin4 is the popular PG control tool; use the following command to pull up the PgAdmin4 service on the meta node, default to host 8885 port, username admin@pigsty.cc, password: pigsty.

  1. cd ~/pigsty/app/pgadmin ; make up
  2. # docker-compose up -d , which is
  3. docker run --init --name pgadmin --restart always --detach --publish 8885:80 \
  4.     -e PGADMIN_DEFAULT_EMAIL=admin@pigsty.cc -e PGADMIN_DEFAULT_PASSWORD=pigsty \
  5.     dpage/pgadmin4

Copy the server access information to the /tmp/servers.json file and re-import it.

  1. make conf       # provision pgadmin with pigsty pg servers list 
  2. make dump       # dump servers.json from pgadmin container

PGWeb

PGWeb is a browser-based PG client tool. Use the following command to pull up the PGWEB service on the meta node, defaulting to the host 8081 port.

  1. cd ~/pigsty/app/pgweb ; docker-compose up -d
  2. docker run --init --name pgweb --restart always --detach --publish 8886:8081 sosedoff/pgweb

Users need to fill in the database connection string, for example, the default CMDB:

postgres://dbuser_dba:DBUser.DBA@10.10.10.10:5432/meta?sslmode=disable.

PostgREST

PostgREST is a binary component that automatically generates a REST API based on the PostgreSQL database schema.

The following command will pull up postgrest using docker (local port 8082, using the default admin user, exposing the Pigsty CMDB schema).

  1. cd ~/pigsty/app/postgrest ; docker-compose up -d
  2. docker run --init --name postgrest --restart always --detach --publish 8884:8081 postgrest/postgrest

http://home.pigsty.cc:8883/ shows the available API that exposed by PostgREST

Visiting http://10.10.10.10:8884 will show all the definitions of the auto-generated APIs, which can be automatically generated in the Swagger Editor.

curl http://10.10.10.10:8884/pg_cluster will anonymously access the data table pigsty.pg_cluster.

If you want to add, delete, check and design more fine-grained privilege control, please refer to Tutorial 1 - The Golden Key to generate a signed JWT.

ByteBase

ByteBase is a tool for making database schema changes. The following command will start a ByteBase on meta node port 8887.

  1. cd app/bytebase; docker-compose up -d
  2. docker run --init --name bytebase \
  3.         --restart always --detach \
  4.         --publish 8887:8887 \
  5.         --volume /data/bytebase/data:/var/opt/bytebase \
  6.         bytebase/bytebase:1.0.4 \
  7.         --data /var/opt/bytebase \
  8.         --host http://ddl.pigsty \
  9.         --port 8887 \
  10.         --pg postgres://dbuser_bytebase:DBUser.Bytebase@10.10.10.10:5432/bytebase

Visit http://10.10.10.10:8887/ to use ByteBase. To start schema changes, you need to create the project, environment, instance, and database.

Jupyter

Jupyter Lab is a data analysis environment. The following command will start a Jupyter Server on port 8084.

  1. docker run -it --restart always --detach --name jupyter -p 8083:8888 -v "${PWD}":/tmp/notebook jupyter/scipy-notebook
  2. docker logs jupyter # Print logs and get Token of login

Visit http://10.10.10.10:8888/ to use JupyterLab, (you need to fill in the auto-generated Token). Note that Pigsty also has JupyterLab installed on the host.

SchemaSPY

Generate a database schema report using CMDB as an example. The following docker, using.

  1. docker run -v /www/schema/pg-meta/meta/pigsty:/output andrewjones/schemaspy-postgres:latest \
  2.     -host 10.10.10.10 -port 5432 -u dbuser_dba -p DBUser.DBA -db meta -s pigsty

Then visit http://pigsty/schema/pg-meta/meta/pigsty to access the Schema report.

Gitlab

Please refer to the Gitlab Docker Deploy Doc to complete the Docker deployment.

  1. export GITLAB_HOME=/data/gitlab
  3. sudo docker run --detach \
  4.   --hostname gitlab.example.com \
  5.   --publish 443:443 --publish 80:80 --publish 23:22 \
  6.   --name gitlab \
  7.   --restart always \
  8.   --volume $GITLAB_HOME/config:/etc/gitlab \
  9.   --volume $GITLAB_HOME/logs:/var/log/gitlab \
  10.   --volume $GITLAB_HOME/data:/var/opt/gitlab \
  11.   --shm-size 256m \
  12.   gitlab/gitlab-ee:latest
  14. sudo docker exec -it gitlab grep 'Password:' /etc/gitlab/initial_root_password

Discourse

Build open source forum Discourse. You need to adjust the config app.yml, focusing on the SMTP part of the config.

Sample Discourse config

  1. templates:
  2.   - "templates/web.china.template.yml"
  3.   - "templates/postgres.template.yml"
  4.   - "templates/redis.template.yml"
  5.   - "templates/web.template.yml"
  6.   - "templates/web.ratelimited.template.yml"
  7. ## Uncomment these two lines if you wish to add Lets Encrypt (https)
  8. # - "templates/web.ssl.template.yml"
  9. # - "templates/web.letsencrypt.ssl.template.yml"
  10. expose:
  11.   - "80:80"   # http
  12.   - "443:443" # https
  13. params:
  14.   db_default_text_search_config: "pg_catalog.english"
  15.   db_shared_buffers: "768MB"
  16. env:
  17.   LC_ALL: en_US.UTF-8
  18.   LANG: en_US.UTF-8
  19.   LANGUAGE: en_US.UTF-8
  20.   EMBER_CLI_PROD_ASSETS: 1
  21.   UNICORN_WORKERS: 4
  22.   DISCOURSE_HOSTNAME: forum.pigsty
  23.   DISCOURSE_DEVELOPER_EMAILS: 'fengruohang@outlook.com,rh@vonng.com'
  24.   DISCOURSE_SMTP_ENABLE_START_TLS: false
  25.   DISCOURSE_SMTP_AUTHENTICATION: login
  26.   DISCOURSE_SMTP_OPENSSL_VERIFY_MODE: none
  27.   DISCOURSE_SMTP_ADDRESS: smtpdm.server.address
  28.   DISCOURSE_SMTP_PORT: 80
  29.   DISCOURSE_SMTP_USER_NAME: no_reply@mail.pigsty.cc
  30.   DISCOURSE_SMTP_PASSWORD: "<password>"
  31.   DISCOURSE_SMTP_DOMAIN: mail.pigsty.cc
  32. volumes:
  33.   - volume:
  34.       host: /var/discourse/shared/standalone
  35.       guest: /shared
  36.   - volume:
  37.       host: /var/discourse/shared/standalone/log/var-log
  38.       guest: /var/log
  40. hooks:
  41.   after_code:
  42.     - exec:
  43.         cd: $home/plugins
  44.         cmd:
  45.           - git clone https://github.com/discourse/docker_manager.git
  46. run:
  47.   - exec: echo "Beginning of custom commands"
  48.   # - exec: rails r "SiteSetting.notification_email='no_reply@mail.pigsty.cc'"
  49.   - exec: echo "End of custom commands"

Then, just execute the following command and pull up Discourse.

  1. ./launcher rebuild app
PGAdmin4

Pull up pgadmin4 with Docker and load pigsty server list

Gitea

Launch gitea with pigsty pg as meta database

Kong API Gateway

使用Docker拉起KONG API网关，使用外部PG作为元数据存储，对已有API进行管理

PostgREST Backend RESTAPI auto-gen

Launch PostgREST with docker and generate backend API automatically.

Schema Migration with Bytebase

使用Docker拉起Bytebase，对PG的模式进行版本化管理

Minio S3 Services

Launch minio with docker in pigsty

Browser data with PGWEB

Launch pgweb with docker for small batch query from browser

Last modified 2022-06-03: add scaffold for en docs (6a6eded)