Debugging related

Because linkis itself has many modules, it is difficult to debug, the following will guide you how to perform a local service debugging (based on version 1.0.3).

Before version 1.0.3, linkis has not yet entered apache incubation. The organization still belongs to webank. The package name of the main class is `org.apache.linkis`. Pay attention to the distinction when debugging.

step 1 Prepare source code and compile

  1. git clone https://github.com/apache/incubator-linkis.git
  2. cd incubator-linkis
  3. #If needed, you can switch to the corresponding branch
  4. #git checkout dev-xxx
  5. mvn -N install
  6. mvn clean Install

step2 Necessary parameter configuration

For the configuration file under incubator-linkis/assembly-combined-package/assembly-combined/conf/, you need to configure the database and hive meta and other necessary startup parameters.

step3 Adjust log4j.xml configuration

In order to facilitate the printing of logs to the console during debugging, you need to modify the default log4j2.xml file and modify the appender to default to console. You need to remove the append of the default RollingFile and add the appender of the console, as shown below: Linkis Debug - 图1 log4j2.xml path incubator-linkis/assembly-combined-package/assembly-combined/conf/log4j2.xml

  1.  <?xml version="1.0" encoding="UTF-8"?>
  2. <configuration status="error" monitorInterval="30">
  3. <appenders>
  4.     <RollingFile name="RollingFile" append="false" fileName="logs/${sys:serviceName}.log"
  5.                  filePattern="logs/$${date:yyyy-MM}/${sys:serviceName}/linkis-log-%d{yyyy-MM-dd}-%i.log">
  6.         <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] [%-40t] %c{1.} (%L) [%M]-%msg%xEx %n"/>
  7.         <SizeBasedTriggeringPolicy size="100MB"/>
  8.         <DefaultRolloverStrategy max="10"/>
  9.     </RollingFile>
  11.     <Console name="Console" target="SYSTEM_OUT">
  12.         <ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
  13.         <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%t] %logger{36} %L %M-%msg%xEx%n"/>
  14.     </Console>
  15. </appenders>
  16. <loggers>
  17.     <root level="INFO">
  18.         <appender-ref ref="RollingFile"/>
  19.         <appender-ref ref="Console"/>
  20.     </root>
  21. </loggers>
  22. </configuration>

step 4 Overall debugging plan

Both Linkis and DSS services rely on Eureka, so you need to start the Eureka service first, and the Eureka service can also use the Eureka you have started. After Eureka is started, other services can be started.

Because linkis internally uses the -DserviceName parameter to set the application name and the configuration file used, so -DserviceName is a necessary startup VM parameter

You can use the “-Xbootclasspath/a: configuration file path” command. Append the configuration file to the end of the search path of the bootloader class, and add the dependent configuration file to the classpath

By checking Include dependencies with “Provided” scope, you can introduce provided-level dependency packages during debugging.

Microservice Governance Services component

Start of linkis-mg-eureka

  1. [main Class]
  2. org.apache.linkis.eureka.SpringCloudEurekaApplication
  5. [VM Opitons]
  6. -DserviceName=linkis-mg-eureka -Xbootclasspath/a:D:\yourDir\incubator-linkis\assembly-combined-package\assembly-combined\conf
  9. [Program arguments]
  10. --spring.profiles.active=eureka --eureka.instance.preferIpAddress=true
  13. [User classpath of module]
  14. linkis-eureka

If you don’t want the default port 20303, you can modify the port configuration:

  1. File path: conf/application-eureka.yml
  2. Modify the port:
  3. server:
  4.   port: 8080 ##Started port

The specific configuration is as follows Linkis Debug - 图2

After startup, you can view the list of eureka services through http://localhost:20303/ Linkis Debug - 图3

Linkis-mg-gateway startup configuration

  1. [main Class]
  2. org.apache.linkis.gateway.springcloud.LinkisGatewayApplication
  5. [VM Opitons]
  6. -DserviceName=linkis-mg-gateway -Xbootclasspath/a:D:\yourDir\incubator-linkis\assembly-combined-package\assembly-combined\conf
  9. [User classpath of module]
  10. linkis-gateway-server-support

Note If there is a problem of’org.apache.logging.log4j.LoggingException: log4j-slf4j-impl cannot be present with log4j-to-slf4j’ Please exclude, the dependency on spring-boot-starter-logging

Public Enhancement Services component

Linkis-ps-publicservice startup configuration

  1. [main Class]
  2. org.apache.linkis.jobhistory.LinkisPublicServiceApp
  5. [VM Opitons]
  6. -DserviceName=linkis-ps-publicservice -Xbootclasspath/a:D:\yourDir\incubator-linkis\assembly-combined-package\assembly-combined\conf
  9. [User classpath of module]
  10. linkis-jobhistory

Linkis-ps-cs startup configuration

  1. [main Class]
  2. org.apache.linkis.cs.server.LinkisCSApplication
  5. [VM Opitons]
  6. -DserviceName=linkis-ps-cs -Xbootclasspath/a:D:\yourDir\incubator-linkis\assembly-combined-package\assembly-combined\conf
  9. [User classpath of module]
  10. linkis-cs-server

Computation Governance Services component

linkis-cg-linkismanager start

  1. [main Class]
  2. org.apache.linkis.manager.am.LinkisManagerApplication
  5. [VM Opitons]
  6. -DserviceName=linkis-cg-linkismanager -Xbootclasspath/a:D:\yourDir\incubator-linkis\assembly-combined-package\assembly-combined\conf
  9. [User classpath of module]
  10. linkis-application-manager

linkis-cg-entrance start

  1. [main Class]
  2. org.apache.linkis.entrance.LinkisEntranceApplication
  5. [VM Opitons]
  6. -DserviceName=linkis-cg-entrance -Xbootclasspath/a:D:\yourDir\incubator-linkis\assembly-combined-package\assembly-combined\conf
  9. [User classpath of module]
  10. linkis-entrance

Note: Windows local debugging service is not currently supported

linkis-cg-engineplugin(ecp): Need to read local ecp materials, local debugging needs to prepare the corresponding materials first, it is recommended to debug remotely

linkis-cg-engineconnmanager(ecm): temporarily ecm starts the engine using the unix method and does not support the windows environment

  1. The following are the specific detailed command parameters for starting the linkis service after a normal and successful installation
  4. LinkisInstallDir: complete linkis installation directory
  7. [linkis-mg-eureka]
  8. nohup java -DserviceName=linkis-mg-eureka -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-mg-eureka-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-spring-cloud-services/linkis-mg-eureka/* org.apache.linkis.eureka.SpringCloudEurekaApplication --eureka.instance.hostname=bdpujes110001 --spring.profiles.active=eureka 2>&1> / data/LinkisInstallDir/logs/linkis-mg-eureka.out &
  11. [linkis-mg-gateway]
  12. nohup java -DserviceName=linkis-mg-gateway -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-mg-gateway-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-spring-cloud-services/linkis-mg-gateway/* org.apache.linkis.gateway.springcloud.LinkisGatewayApplication 2>&1> /data/LinkisInstallDir/logs/linkis-mg-gateway.out &
  15. [linkis-ps-publicservice]
  16. nohup java -DserviceName=linkis-ps-publicservice -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-ps-publicservice-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-commons/public-module/*:/data/LinkisInstallDir/lib/linkis-public-enhancements/linkis-ps-publicservice/* org.apache.linkis.jobhistory.LinkisPublicServiceApp 2>&1> /data/LinkisInstallDir /logs/linkis-ps-publicservice.out &
  19. [linkis-cg-linkismanager]
  20. nohup java -DserviceName=linkis-cg-linkismanager -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-cg-linkismanager-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-commons/public-module/*:/data/LinkisInstallDir/lib/linkis-computation-governance/linkis-cg-linkismanager/* org.apache.linkis.manager.am.LinkisManagerApplication 2>&1> /data /LinkisInstallDir/logs/linkis-cg-linkismanager.out &
  23. [linkis-ps-cs]
  24. nohup java -DserviceName=linkis-ps-cs -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-ps-cs-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-commons/public-module/*:/data/LinkisInstallDir/lib/linkis-public-enhancements/linkis-ps-cs/* org.apache.linkis.cs.server.LinkisCSApplication 2>&1> /data /LinkisInstallDir/logs/linkis-ps-cs.out &
  27. [linkis-cg-entrance]
  28. nohup java -DserviceName=linkis-cg-entrance -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-cg-entrance-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-commons/public-module/*:/data/LinkisInstallDir/lib/linkis-computation-governance/linkis-cg-entrance/* org.apache.linkis.entrance.LinkisEntranceApplication 2>&1> /data/LinkisInstallDir /logs/linkis-cg-entrance.out &
  31. [linkis-cg-engineconnmanager]
  32. nohup java -DserviceName=linkis-cg-engineconnmanager -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-cg-engineconnmanager-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-commons/public-module/*:/data/LinkisInstallDir/lib/linkis-computation-governance/linkis-cg-engineconnmanager/* org.apache.linkis.ecm.server.LinkisECMApplication 2>&1> /data /LinkisInstallDir/logs/linkis-cg-engineconnmanager.out &
  35. [linkis-cg-engineplugin]
  36. nohup java -DserviceName=linkis-cg-engineplugin -Xmx512M -XX:+UseG1GC -Xloggc:/data/LinkisInstallDir/logs/linkis-cg-engineplugin-gc.log -cp /data/LinkisInstallDir/conf/:/data/LinkisInstallDir /lib/linkis-commons/public-module/*:/data/LinkisInstallDir/lib/linkis-computation-governance/linkis-cg-engineplugin/* org.apache.linkis.engineplugin.server.LinkisEngineConnPluginServer 2>&1> /data /LinkisInstallDir/logs/linkis-cg-engineplugin.out &

Remote debugging service steps

todo