Recommended Configuration

Linkis builds a layer of computing middleware between the upper application and the underlying engine. As an open source distributed computing middleware, it can be well deployed and run on Intel architecture servers and mainstream virtualization environments, and supports mainstream Linux operating system environments

OSVersion
Red Hat Enterprise Linux7.0 and above
CentOS7.0 and above
Oracle Enterprise Linux7.0 and above
Ubuntu LTS16.04 and above

Note: The above Linux operating systems can run on physical servers and mainstream virtualization environments such as VMware, KVM, and XEN

Linkis supports 64-bit general-purpose hardware server platforms running on the Intel x86-64 architecture. The following recommendations are made for the server hardware configuration of the production environment:

CPUMemoryDisk typeNetworkNumber of instances
16 cores +32GB +SASGigabit network card1+

Note:

  • The above recommended configuration is the minimum configuration for deploying Linkis, and a higher configuration is strongly recommended for production environments
  • The hard disk size configuration is recommended to be 50GB+, and the system disk and data disk are separated

Linkis binary packages are compiled based on the following software versions:

ComponentVersionDescription
Hadoop3.3.4
Hive3.1.3
Spark3.2.1
Flink1.12.2
openLooKeng1.5.0
Sqoop1.4.6
ElasticSearch7.6.2
Presto0.234
PythonPython2

Note: If the locally installed component version is inconsistent with the above, you need to modify the corresponding component version and compile the binary package yourself for installation.

Linkis recommends Chrome version 73 for front-end access

The development process requires a password-free interface, which can be replaced or appended to linkis.properties

parameter namedefault valuedescription
wds.linkis.test.modefalseWhether to enable debugging mode, if set to true, all microservices support password-free login, and all EngineConn open remote debugging ports
wds.linkis.test.userhadoopWhen wds.linkis.test.mode=true, the default login user for password-free login

Recommended Configuration - 图1

Apache Linkis uses configuration files to manage admin users by default, and this configuration can be replaced or appended to linkis-mg-gateway.properties. For multi-user access LDAP implementation.

parameter namedefault valuedescription
wds.linkis.admin.userhadoopadmin username
wds.linkis.admin.password123456Admin user password

Recommended Configuration - 图2

Apache Linkis can access LDAP through parameters to achieve multi-user management, and this configuration can be replaced or added in linkis-mg-gateway.properties.

parameter namedefault valuedescription
wds.linkis.ldap.proxy.urlNoneLDAP URL address
wds.linkis.ldap.proxy.baseDNNoneLDAP baseDN address
wds.linkis.ldap.proxy.userNameFormatNone

Recommended Configuration - 图3

Apache Linkis sometimes debugs exceptions when submitting tasks, such as: insufficient resources; you can replace or append this configuration in linkis-cg-linkismanager.properties.

parameter namedefault valuedescription
wds.linkis.manager.rm.request.enabletrueresource check

Recommended Configuration - 图4

Apache Linkis EC can enable debugging mode, and this configuration can be replaced or added in linkis-cg-linkismanager.properties.

parameter namedefault valuedescription
wds.linkis.engineconn.debug.enabletrueWhether to enable engine debugging

Recommended Configuration - 图5

The public-service service of Apache Linkis needs to read hive metadata; this configuration can be replaced or appended in linkis-ps-publicservice.properties.

parameter namedefault valuedescription
hive.meta.urlNoneThe URL of the HiveMetaStore database.
hive.meta.usernoneuser of the HiveMetaStore database
hive.meta.passwordNonepassword for the HiveMetaStore database

Recommended Configuration - 图6

Apache Linkis access uses Mysql as data storage by default, you can replace or append this configuration in linkis.properties.

parameter namedefault valuedescription
wds.linkis.server.mybatis.datasource.urlNoneDatabase connection string, for example: jdbc:mysql://127.0.0.1:3306/dss?characterEncoding=UTF-8
wds.linkis.server.mybatis.datasource.usernameNoneDatabase user name, for example: root
wds.linkis.server.mybatis.datasource.passwordNoneDatabase password, for example: root

Recommended Configuration - 图7

Apache Linkis supports using redis for session sharing; this configuration can be replaced or appended in linkis.properties.

parameter namedefault valuedescription
linkis.session.redis.cache.enabledNoneWhether to enable
linkis.session.redis.host127.0.0.1hostname
linkis.session.redis.port6379Port, eg
linkis.session.redis.passwordNonepassword

Recommended Configuration - 图8

When developing Apache Linkis, you can use this parameter to customize the database, Rest interface, and entity objects of the loading module; you can modify it in linkis-ps-publicservice.properties, and use commas to separate multiple modules.

parameter namedefault valuedescription
wds.linkis.server.restful.scan.packagesNonerestful scan packages, for example: org.apache.linkis.basedatamanager.server.restful
wds.linkis.server.mybatis.mapperLocationsNoneMybatis mapper file path, for example: classpath:org/apache/linkis/basedatamanager/server/dao/mapper/.xml
wds.linkis.server.mybatis.typeAliasesPackageNoneEntity alias scanning package, for example: org.apache.linkis.basedatamanager.server.domain
wds.linkis.server.mybatis.BasePackageNoneDatabase dao layer scan, for example: org.apache.linkis.basedatamanager.server.dao

Recommended Configuration - 图9

This parameter can be used to customize the route of loading modules during Apache Linkis development; it can be modified in linkis.properties, and commas are used to separate multiple modules.

parameter namedefault valuedescription
wds.linkis.gateway.conf.publicservice.listcs,contextservice,data-source-manager,metadataQuery,metadatamanager,query,jobhistory,application,configuration,filesystem,udf,variable,microservice,errorcode,bml,datasource,basedata -managerpublicservice services support routing modules

Recommended Configuration - 图10

This parameter can be used to customize the route of loading modules during Apache Linkis development; it can be modified in linkis.properties, and commas are used to separate multiple modules.

parameter namedefault valuedescription
wds.linkis.filesystem.root.pathfile:///tmp/linkis/Local user directory, a folder named after the user name needs to be created under this directory
wds.linkis.filesystem.hdfs.root.pathhdfs:///tmp/HDFS user directory
wds.linkis.bml.is.hdfstrueWhether to enable hdfs
wds.linkis.bml.hdfs.prefix/apps-datahdfs path
wds.linkis.bml.local.prefix/apps-datalocal path

Recommended Configuration - 图11