Installation directory structure

The directory structure of Linkis 1.0 is very different from the 0.X version. Each microservice in 0.X has a root directory that exists independently. The main advantage of this directory structure is that it is easy to distinguish microservices and facilitate individual Microservices are managed, but there are some obvious problems:

  1. The microservice catalog is too complicated and it is not convenient to switch catalog management
  2. There is no unified startup script, which makes it more troublesome to start and stop microservices
  3. There are a large number of duplicate service configurations, and the same configuration often needs to be modified in many places
  4. There are a large number of repeated Lib dependencies, which increases the size of the installation package and the risk of dependency conflicts

Therefore, in Linkis 1.0, we have greatly optimized and adjusted the installation directory structure, reducing the number of microservice directories, reducing the jar packages that are repeatedly dependent, and reusing configuration files and microservice management scripts as much as possible. Mainly reflected in the following aspects:

1.The bin folder is no longer provided for each microservice, and modified to be shared by all microservices.

The Bin folder is modified to the installation directory, which is mainly used to install Linkis 1.0 and check the environment status. The new sbin directory provides one-click start and stop for Linkis, and provides independent start and stop for all microservices by changing parameters.

2.No longer provide a separate conf directory for each microservice, and modify it to be shared by all microservices.

The Conf folder contains two aspects of content. On the one hand, it is the configuration information shared by all microservices. This type of configuration information contains information that users can customize configuration according to their own environment; on the other hand, it is the special characteristics of each microservice. Configuration, under normal circumstances, users do not need to change by themselves.

3.The lib folder is no longer provided for each microservice, and modified to be shared by all microservices

The Lib folder also contains two aspects of content, on the one hand, the common dependencies required by all microservices; on the other hand, the special dependencies required by each microservice.

4.The log directory is no longer provided for each microservice, modified to be shared by all microservices

The Log directory contains log files of all microservices.

The simplified directory structure of Linkis 1.0 is as follows.

  1. ├── bin ──installation directory
  2. │ ├── checkEnv.sh ── Environmental variable detection
  3. │ ├── checkServices.sh ── Microservice status check
  4. │ ├── common.sh ── Some public shell functions
  5. │ ├── install-io.sh ── Used for dependency replacement during installation
  6. │ └── install.sh ── Main script of Linkis installation
  7. ├── conf ──configuration directory
  8. │ ├── application-eureka.yml 
  9. │ ├── application-linkis.yml    ──Microservice general yml
  10. │ ├── linkis-cg-engineconnmanager-io.properties
  11. │ ├── linkis-cg-engineconnmanager.properties
  12. │ ├── linkis-cg-engineplugin.properties
  13. │ ├── linkis-cg-entrance.properties
  14. │ ├── linkis-cg-linkismanager.properties
  15. │ ├── linkis-computation-governance
  16. │ │   └── linkis-client
  17. │ │       └── linkis-cli
  18. │ │           ├── linkis-cli.properties
  19. │ │           └── log4j2.xml
  20. │ ├── linkis-env.sh   ──linkis environment properties
  21. │ ├── linkis-et-validator.properties
  22. │ ├── linkis-mg-gateway.properties
  23. │ ├── linkis.properties  ──linkis global properties
  24. │ ├── linkis-ps-bml.properties
  25. │ ├── linkis-ps-cs.properties
  26. │ ├── linkis-ps-datasource.properties
  27. │ ├── linkis-ps-publicservice.properties
  28. │ ├── log4j2.xml
  29. │ ├── proxy.properties(Optional)
  30. │ └── token.properties(Optional)
  31. ├── db ──database DML and DDL file directory
  32. │ ├── linkis\_ddl.sql ──Database table definition SQL
  33. │ ├── linkis\_dml.sql ──Database table initialization SQL
  34. │ └── module ──Contains DML and DDL files of each microservice
  35. ├── lib ──lib directory
  36. │ ├── linkis-commons ──Common dependency package
  37. │ ├── linkis-computation-governance ──The lib directory of the computing governance module
  38. │ ├── linkis-engineconn-plugins ──lib directory of all EngineConnPlugins
  39. │ ├── linkis-public-enhancements ──lib directory of public enhancement services
  40. │ └── linkis-spring-cloud-services ──SpringCloud lib directory
  41. ├── logs ──log directory
  42. │ ├── linkis-cg-engineconnmanager-gc.log
  43. │ ├── linkis-cg-engineconnmanager.log
  44. │ ├── linkis-cg-engineconnmanager.out
  45. │ ├── linkis-cg-engineplugin-gc.log
  46. │ ├── linkis-cg-engineplugin.log
  47. │ ├── linkis-cg-engineplugin.out
  48. │ ├── linkis-cg-entrance-gc.log
  49. │ ├── linkis-cg-entrance.log
  50. │ ├── linkis-cg-entrance.out
  51. │ ├── linkis-cg-linkismanager-gc.log
  52. │ ├── linkis-cg-linkismanager.log
  53. │ ├── linkis-cg-linkismanager.out
  54. │ ├── linkis-et-validator-gc.log
  55. │ ├── linkis-et-validator.log
  56. │ ├── linkis-et-validator.out
  57. │ ├── linkis-mg-eureka-gc.log
  58. │ ├── linkis-mg-eureka.log
  59. │ ├── linkis-mg-eureka.out
  60. │ ├── linkis-mg-gateway-gc.log
  61. │ ├── linkis-mg-gateway.log
  62. │ ├── linkis-mg-gateway.out
  63. │ ├── linkis-ps-bml-gc.log
  64. │ ├── linkis-ps-bml.log
  65. │ ├── linkis-ps-bml.out
  66. │ ├── linkis-ps-cs-gc.log
  67. │ ├── linkis-ps-cs.log
  68. │ ├── linkis-ps-cs.out
  69. │ ├── linkis-ps-datasource-gc.log
  70. │ ├── linkis-ps-datasource.log
  71. │ ├── linkis-ps-datasource.out
  72. │ ├── linkis-ps-publicservice-gc.log
  73. │ ├── linkis-ps-publicservice.log
  74. │ └── linkis-ps-publicservice.out
  75. ├── pid ──Process ID of all microservices
  76. │ ├── linkis\_cg-engineconnmanager.pid ──EngineConnManager microservice
  77. │ ├── linkis\_cg-engineconnplugin.pid ──EngineConnPlugin microservice
  78. │ ├── linkis\_cg-entrance.pid ──Engine entrance microservice
  79. │ ├── linkis\_cg-linkismanager.pid ──linkis manager microservice
  80. │ ├── linkis\_mg-eureka.pid ──eureka microservice
  81. │ ├── linkis\_mg-gateway.pid ──gateway microservice
  82. │ ├── linkis\_ps-bml.pid ──material library microservice
  83. │ ├── linkis\_ps-cs.pid ──Context microservice
  84. │ ├── linkis\_ps-datasource.pid ──Data source microservice
  85. │ └── linkis\_ps-publicservice.pid ──public microservice
  86. └── sbin ──microservice start and stop script directory
  87.     ├── ext ──Start and stop script directory of each microservice
  88.     ├── linkis-daemon.sh ── Quick start and stop, restart a single microservice script
  89.     ├── linkis-start-all.sh ── Start all microservice scripts with one click
  90.     └── linkis-stop-all.sh ── Stop all microservice scripts with one click

Configuration item modification

After executing the install.sh in the bin directory to complete the Linkis installation, you need to modify the configuration items. All configuration items are located in the con directory. Normally, you need to modify the three configurations of db.sh, linkis.properties, and linkis-env.sh For documentation, project installation and configuration, please refer to the article “Linkis1.0 Installation”

Microservice start and stop

After modifying the configuration items, you can start the microservice in the sbin directory. The names of all microservices are as follows:

  1. ├── linkis-cg-engineconnmanager  ──engine management service
  2. ├── linkis-cg-engineplugin  ──EngineConnPlugin management service
  3. ├── linkis-cg-entrance  ──computing governance entrance service
  4. ├── linkis-cg-linkismanager  ──computing governance management service
  5. ├── linkis-mg-eureka  ──microservice registry service
  6. ├── linkis-mg-gateway  ──Linkis gateway service
  7. ├── linkis-ps-bml  ──material library service
  8. ├── linkis-ps-cs  ──context service
  9. ├── linkis-ps-datasource  ──data source service
  10. └── linkis-ps-publicservice  ──public service

Microservice abbreviation:

AbbreviationFull English NameFull Chinese Name
cgComputation GovernanceComputing Governance
mgMicroservice GovernanceMicroservice Governance
psPublic Enhancement ServicePublic Enhancement Service

In the past, to start and stop a single microservice, you need to enter the bin directory of each microservice and execute the start/stop script. When there are many microservices, it is troublesome to start and stop. A lot of additional directory switching operations are added. Linkis1.0 will all The scripts related to the start and stop of microservices are placed in the sbin directory, and only a single entry script needs to be executed.

Under the Linkis/sbin directory:

1.Start all microservices at once:

  1. sh linkis-start-all.sh

2.Shut down all microservices at once

  1. sh linkis-stop-all.sh

3.Start a single microservice (the service name needs to be removed from the Linkis prefix, such as mg-eureka)

  1. sh linkis-daemon.sh start service-name

For example:

  1. sh linkis-daemon.sh start mg-eureka

4.Shut down a single microservice

  1. sh linkis-daemon.sh stop service-name

For example:

  1. sh linkis-daemon.sh stop mg-eureka

5.Restart a single microservice

  1. sh linkis-daemon.sh restart service-name

For example:

  1. sh linkis-daemon.sh restart mg-eureka

6.View the status of a single microservice

  1. sh linkis-daemon.sh status service-name

For example:

  1. sh linkis-daemon.sh status mg-eureka