linkis directory structure

linkis code hierarchy structure, as well as package structure and deployment directory structure description description, if you explain, if you want to explain, if you want to know more about each module module module module

  1. ├── docs
  2. │ ├── configuration //linkis configuration item documents for all modules
  3. │ ├── errorcode // error code document of all modules of linkis
  4. │ ├── configuration-change-records.md
  5. │ ├── index.md
  6. │ ├── info-1.1.3.md
  7. │ ├── info-1.2.1.md
  8. │ ├── info-1.3.1.md
  9. │ └── trino-usage.md
  10. ├── linkis-commons //Core abstraction, which contains all common modules
  11. │ ├── linkis-common //Common module, many built-in common tools
  12. │ ├── linkis-hadoop-common
  13. │ ├── linkis-httpclient //Java SDK top-level interface further encapsulates httpclient
  14. │ ├── linkis-module // The top-level public module of linkis service involves parameters and service initialization when the service starts, unified Restful processing, login status verification, etc.
  15. │ ├── linkis-mybatis //Mybatis module of SpringCloud
  16. │ ├── linkis-protocol //Some interfaces and entity classes of service request/response
  17. │ ├── linkis-rpc //RPC module, complex two-way communication based on Feign
  18. │ ├── linkis-scheduler //General scheduling module
  19. │ ├── linkis-storage //File operation tool set
  20. ├── linkis-computation-governance //Computation governance service
  21. │ ├── linkis-client //Java SDK, users can directly access Linkis through Client
  22. │ ├── linkis-computation-governance-common
  23. │ ├── linkis-engineconn
  24. │ ├── linkis-engineconn-manager
  25. │ ├── linkis-entrance //General underlying entrance module
  26. │ ├── linkis-jdbc-driver //You can use linkis to connect in a similar way to jdbc sdk
  27. │ ├── linkis-manager
  28. ├── linkis-dist //The final step of compiling and packaging, integrating all lib packages and installation and deployment script configuration, etc.
  29. │ ├── bin
  30. │ │ ├── checkEnv.sh
  31. │ │ ├── common.sh
  32. │ │ └── install.sh //Installation script
  33. │ ├── deploy-config
  34. │ │ ├── db.sh //database configuration
  35. │ │ └── linkis-env.sh //linkis startup related configuration
  36. │ ├── docker
  37. │ │ └── scripts
  38. │ ├── helm
  39. │ │ ├── charts
  40. │ │ ├── scripts
  41. │ │ ├── README_CN.md
  42. │ │ └── README.md
  43. │ ├── package
  44. │ │ ├── bin
  45. │ │ ├── conf
  46. │ │ ├── db
  47. │ │ └── sbin
  48. │ ├── release-docs
  49. │ │ ├── licenses
  50. │ │ ├── LICENSE
  51. │ │ └── NOTICE
  52. │ ├── src
  53. │ └── pom.xml
  54. ├── linkis-engineconn-plugins // engine
  55. │ ├── elasticsearch
  56. │ ├── flink
  57. │ ├──hive
  58. │ ├── io_file
  59. │ ├── jdbc
  60. │ ├── open look
  61. │ ├── pipeline
  62. │ ├── presto
  63. │ ├── python
  64. │ ├── seat tunnel
  65. │ ├── shell
  66. │ ├── spark
  67. │ ├── sqoop
  68. ├── linkis-extensions // extension function enhancement plug-in module
  69. │ ├── linkis-io-file-client // function extension to linkis-storage
  70. ├── linkis-orchestrator //Service orchestration
  71. │ ├── linkis-code-orchestrator
  72. │ ├── linkis-computation-orchestrator
  73. │ ├── linkis-orchestrator-core
  74. ├── linkis-public-enhancements //public enhancement services
  75. │ ├── linkis-baseddata-manager
  76. │ ├── linkis-bml // material library
  77. │ ├── linkis-configuration
  78. │ ├── linkis-context-service //unified context
  79. │ ├── linkis-datasource //data source service
  80. │ ├── linkis-error-code
  81. │ ├── linkis-instance-label
  82. │ ├── linkis-jobhistory
  83. │ ├── linkis-ps-common-lock
  84. │ ├── linkis-script-dev
  85. │ ├── linkis-udf
  86. │ ├── linkis-variable
  87. ├── linkis-spring-cloud-services //Microservice Governance
  88. │ ├── linkis-service-discovery
  89. │ ├── linkis-service-gateway //Gateway Gateway
  90. ├── linkis-web //linkis management console code
  91. │ ├── release-docs
  92. │ │ ├── licenses
  93. │ │ └── LICENSE
  94. │ ├── src
  95. │ ├── config.sh
  96. │ ├── install.sh
  97. │ ├── package.json
  98. │ ├── pom.xml
  99. │ └── vue.config.js
  100. ├── tool
  101. │ ├── dependencies
  102. │ │ ├── known-dependencies.txt
  103. │ │ └── regenerate_konwn_dependencies_txt.sh
  104. │ ├── code-style-idea.xml
  105. │ ├── license-header
  106. │ └── modify_license.sh
  107. ├── CONTRIBUTING_CN.md
  108. ├── CONTRIBUTING.md
  109. ├── DISCLAIMER
  110. ├── linkis-tree.txt
  111. ├── mvnw
  112. ├── mvnw.cmd
  113. ├── pom.xml
  114. ├── README_CN.md
  115. ├── README.md
  116. └── scalastyle-config.xml
  2. ├── bin
  3. │ ├── checkEnv.sh ── environment variable detection
  4. │ ├── common.sh ── some public shell functions
  5. │ └── install.sh ── Main script for Linkis installation
  6. ├── deploy-config
  7. │ ├── db.sh //Database connection configuration
  8. │ └── linkis-env.sh //Related environment configuration information
  9. ├── docker
  10. ├── helm
  11. ├── licenses
  12. ├── linkis-package //Microservice-related startup configuration files, dependencies, scripts, linkis-cli, etc.
  13. │ ├── bin
  14. │ ├── conf
  15. │ ├── db
  16. │ ├── lib
  17. │ └── sbin
  18. ├── NOTICE
  19. ├── DISCLAIMER
  20. ├── LICENSE
  21. ├── README_CN.md
  22. └── README.md
  1. ├── bin ── linkis-cli Shell command line program used to submit tasks to Linkis
  2. │ ├── linkis-cli
  3. │ ├── linkis-cli-hive
  4. │ ├── linkis-cli-pre
  5. │ ├── linkis-cli-spark-sql
  6. │ ├── linkis-cli-spark-submit
  7. │ └── linkis-cli-sqoop
  8. ├── conf configuration directory
  9. │ ├── application-eureka.yml
  10. │ ├── application-linkis.yml ── Microservice general yml
  11. │ ├── linkis-cg-engineconnmanager.properties
  12. │ ├── linkis-cg-engineplugin.properties
  13. │ ├── linkis-cg-linkismanager.properties
  14. │ │── linkis-cli
  15. │ │ ├── linkis-cli.properties
  16. │ │ └── log4j2.xml
  17. │ ├── linkis-env.sh ── linkis environment variable configuration
  18. │ ├── linkis-mg-gateway.properties
  19. │ ├── linkis.properties ── The global coordination of linkis services, all microservices will be loaded and used when starting
  20. │ ├── linkis-ps-publicservice.properties
  21. │ ├── log4j2.xml
  22. ├── db Database DML and DDL file directory
  23. │ ├── linkis_ddl.sql ── database table definition SQL
  24. │ ├── linkis_dml.sql ── database table initialization SQL
  25. │ └── module ── Contains DML and DDL files of each microservice
  26. │ └── upgrade ── Incremental DML/DDL for each version
  27. ├── lib lib directory
  28. │ ├── linkis-commons ── Public dependency package When most services start (except linkis-mg-gateway) -cp path parameter will load this directory
  29. │ ├── linkis-computation-governance ── lib directory of computing governance module
  30. │ ├── linkis-engineconn-plugins ── lib directory of all engine plugins
  31. │ ├── linkis-public-enhancements ── lib directory of public enhancement services
  32. │ └── linkis-spring-cloud-services ── SpringCloud lib directory
  33. ├── logs log directory
  34. │ ├── linkis-cg-engineconnmanager-gc.log
  35. │ ├── linkis-cg-engineconnmanager.log
  36. │ ├── linkis-cg-engineconnmanager.out
  37. │ ├── linkis-cg-engineplugin-gc.log
  38. │ ├── linkis-cg-engineplugin.log
  39. │ ├── linkis-cg-engineplugin.out
  40. │ ├── linkis-cg-entrance-gc.log
  41. │ ├── linkis-cg-entrance.log
  42. │ ├── linkis-cg-entrance.out
  43. │ ├── linkis-cg-linkismanager-gc.log
  44. │ ├── linkis-cg-linkismanager.log
  45. │ ├── linkis-cg-linkismanager.out
  46. │ ├── linkis-cli
  47. │ │ ├── linkis-client.hadoop.log.20220409162400037523664
  48. │ │ ├── linkis-client.hadoop.log.20220409162524417944443
  49. │ ├── linkis-mg-eureka-gc.log
  50. │ ├── linkis-mg-eureka.log
  51. │ ├── linkis-mg-eureka.out
  52. │ ├── linkis-mg-gateway-gc.log
  53. │ ├── linkis-mg-gateway.log
  54. │ ├── linkis-mg-gateway.out
  55. │ ├── linkis-ps-publicservice-gc.log
  56. │ ├── linkis-ps-publicservice.log
  57. │ └── linkis-ps-publicservice.out
  58. ├── pid The process ID of all microservices
  59. │ ├── linkis_cg-engineconnmanager.pid ── engine manager microservice
  60. │ ├── linkis_cg-engineconnplugin.pid ── engine plugin microservice
  61. │ ├── linkis_cg-entrance.pid ── engine entry microservice
  62. │ ├── linkis_cg-linkismanager.pid ── linkis manager microservice
  63. │ ├── linkis_mg-eureka.pid ── eureka microservice
  64. │ ├── linkis_mg-gateway.pid ──gateway microservice
  65. │ └── linkis_ps-publicservice.pid ── public microservice
  66. └── sbin Microservice startup and shutdown script directory
  67. ├── ext ──The start and stop script directory of each microservice
  68.   ├── linkis-daemon.sh ── Quickly start, stop, and restart a single microservice script
  69. ├── linkis-start-all.sh ── Start all microservice scripts with one click
  70. └── linkis-stop-all.sh ── Stop all microservice scripts with one click

After executing Linkis installation, all configuration items are located in the conf directory, If you need to modify the configuration items, after modifying the ${LINKIS_HOME}/conf/*properties file, restart the corresponding service, For example: sh sbin/linkis-daemon.sh start ps-publicservice. If you modify the public configuration file application-eureka.yml/application-linkis.yml/linkis.properties, you need to restart all services sh sbin/linkis-start-all.sh

All microservice names are as follows:

  1. ├── linkis-cg-engineconnmanager engine management service
  2. ├── linkis-cg-engineplugin engine plugin management service
  3. ├── linkis-cg-entrance computing governance entry service
  4. ├── linkis-cg-linkismanager computing governance management service
  5. ├── linkis-mg-eureka microservice registry service
  6. ├── linkis-mg-gateway Linkis gateway service
  7. ├── linkis-ps-publicservice public service

Microservice Abbreviation:

AbbreviationFull name in EnglishFull name in Chinese
cgComputation GovernanceComputing Governance
mgMicroservice CovernanceMicroservice Governance
psPublic Enhancement ServicePublic Enhancement Service
  1. # Start all microservices at once:
  3.      sh linkis-start-all.sh
  5. # Shut down all microservices at once
  7.      sh linkis-stop-all.sh
  9. # Start a single microservice (the service name needs to remove the linkis prefix, such as: mg-eureka)
  11.      sh linkis-daemon.sh start service-name
  13.      For example: sh linkis-daemon.sh start mg-eureka
  15. # Shut down a single microservice
  17.      sh linkis-daemon.sh stop service-name
  19.      For example: sh linkis-daemon.sh stop mg-eureka
  21. # Restart a single microservice
  23.      sh linkis-daemon.sh restart service-name
  25.      For example: sh linkis-daemon.sh restart mg-eureka
  26. # View the status of a single microservice
  28.      sh linkis-daemon.sh status service-name
  30.      For example: sh linkis-daemon.sh status mg-eureka