Shell Engine Usage document

This article mainly introduces the configuration, deployment and use of Shell EngineConn in Linkis1.0

If you want to use the shell EngineConn on your server, you need to ensure that the user’s PATH has the bash execution directory and execution permissions.

Table 1-1 Environmental configuration list

The shell EngineConn does not need to be compiled by the user, and the compiled shell EngineConn plug-in package can be used directly.

Here you can use the default loading method to be used normally.

Here you can use the default dml.sql to insert it and it can be used normally.

Before submitting the shell on linkis, you only need to ensure that there is the path of the shell in your user’s $PATH.

Linkis provides a client method to call shell tasks. The call method is through the SDK provided by LinkisClient. We provide java and scala two ways to call, the specific usage can refer to JAVA SDK Manual. If you use Hive, you only need to make the following changes:

        Map<String, Object> labels = new HashMap<String, Object>();
        labels.put(LabelKeyConstant.ENGINE_TYPE_KEY, "shell-1"); // required engineType Label
        labels.put(LabelKeyConstant.USER_CREATOR_TYPE_KEY, "hadoop-IDE");// required execute user and creator
        labels.put(LabelKeyConstant.CODE_TYPE_KEY, "shell"); // required codeType

After Linkis 1.0, you can submit tasks through cli. We only need to specify the corresponding EngineConn and CodeType tag types. The use of shell is as follows:

sh ./bin/linkis-cli -engineType shell-1 -codeType shell -code "echo \"hello\" "  -submitUser hadoop -proxyUser hadoop

The specific usage can refer to Linkis CLI Manual.

The use of Scriptis is the simplest. You can directly enter Scriptis, right-click the directory and create a new shell script, write shell code and click Execute.

The execution principle of the shell is that the shell EngineConn starts a system process to execute through the ProcessBuilder that comes with java, and redirects the output of the process to the EngineConn and writes it to the log.

Figure 3-1 Screenshot of shell execution effect

The shell EngineConn can generally set the maximum memory of the EngineConn JVM.