Integration with Apache Ranger

Apache Ranger is a security framework used to monitor, enable services, and manage all-round data security access on the Hadoop platform.

In version 2.1.0, Doris supports unified permission management by integrating Apache Ranger.

Note:

  • This feature is currently experimental, and the resource objects and permissions configurable in Ranger may change in subsequent versions.

  • Apache Ranger version needs to be above 2.1.0.

Installation

Install Doris Ranger plug-in

  1. Download the following files

  2. Place the downloaded file in the plugins directory of the Ranger service, such as:

    1. /usr/local/service/ranger/ews/webapp/WEB-INF/classes/ranger-plugins/doris/ranger-doris-plugin-3.0.0-SNAPSHOT.jar
    2. /usr/local/service/ranger/ews/webapp/WEB-INF/classes/ranger-plugins/doris/mysql-connector-java-8.0.25.jar

  3. Restart the Ranger service.

  4. Download ranger-servicedef-doris.json

  5. Execute the following command to upload the definition file to the Ranger service:

    1. curl -u user:password -X POST \
    2.     -H "Accept: application/json" \
    3.     -H "Content-Type: application/json" \
    4.     http://172.21.0.32:6080/service/plugins/definitions \
    5.     -d@ranger-servicedef-doris.json

    The username and password are the username and password used to log in to Ranger WebUI.

    The service address port can be viewed in the ranger.service.http.port configuration item of the ranger-admin-site.xml configuration file.

    If the execution is successful, the service definition in Json format will be returned, such as:

    1. {
    2.   "id": 207,
    3.   "guid": "d3ff9e41-f9dd-4217-bb5f-3fa9996454b6",
    4.   "isEnabled": true,
    5.   "createdBy": "Admin",
    6.   "updatedBy": "Admin",
    7.   "createTime": 1705817398112,
    8.   "updateTime": 1705817398112,
    9.   "version": 1,
    10.   "name": "doris",
    11.   "displayName": "Apache Doris",
    12.   "implClass": "org.apache.ranger.services.doris.RangerServiceDoris",
    13.   "label": "Doris",
    14.   "description": "Apache Doris",
    15.   "options": {
    16.     "enableDenyAndExceptionsInPolicies": "true"
    17.   },
    18.   ...
    19. }

    If you want to recreate it, you can use the following command to delete the service definition and then upload it again:

    1. curl -v -u user:password -X DELETE \
    2. http://172.21.0.32:6080/service/plugins/definitions/207

    Where 207 is the id returned when created. Before deletion, you need to delete the created Doris service in the Ranger WebUI.

    You can also use the following command to list the currently added service definitions in order to obtain the id:

    1. curl -v -u user:password -X GET \
    2. http://172.21.0.32:6080/service/plugins/definitions/

Configure the Doris Ranger plug-in

After the installation is complete, open the Ranger WebUI and you can see the Apache Doris plug-in in the Service Manger interface:

Integration with Apache Ranger - 图1

Click the + button next to the plugin to add a Doris service:

Integration with Apache Ranger - 图2

The meaning of some parameters of Config Properties is as follows:

  • Username/Password: the username and password of the Doris cluster. It is recommended to use the Admin user here.
  • jdbc.driver_class: Connect to the JDBC driver used by Doris. com.mysql.cj.jdbc.Driver
  • jdbc.url: JDBC url connection string of Doris cluster. jdbc:mysql://172.21.0.101:9030?useSSL=false
  • Additional parameters:
    • resource.lookup.timeout.value.in.ms: timeout for obtaining meta-information. It is recommended to fill in 10000, which is 10 seconds.

You can click Test Connection to check whether the connection can be made.

Then click Add to add the service.

Afterwards, you can see the created service in the Apache Doris plug-in on the Service Manger page. Click on the service to start configuring Ranger.

Configure Doris cluster

  1. Create a ranger-doris-security.xml file in the conf directory of all FEs with the following content:

    1. <?xml version="1.0" encoding="UTF-8"?>
    2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    3. <configuration>
    4.     <property>
    5.         <name>ranger.plugin.doris.policy.cache.dir</name>
    6.         <value>/path/to/ranger/cache/</value>
    7.     </property>
    8.     <property>
    9.         <name>ranger.plugin.doris.policy.pollIntervalMs</name>
    10.         <value>30000</value>
    11.     </property>
    12.     <property>
    13.         <name>ranger.plugin.doris.policy.rest.client.connection.timeoutMs</name>
    14.         <value>60000</value>
    15.     </property>
    16.     <property>
    17.         <name>ranger.plugin.doris.policy.rest.client.read.timeoutMs</name>
    18.         <value>60000</value>
    19.     </property>
    20.     <property>
    21.         <name>ranger.plugin.doris.policy.rest.url</name>
    22.         <value>http://172.21.0.32:6080</value>
    23.     </property>
    24.     <property>
    25.         <name>ranger.plugin.doris.policy.source.impl</name>
    26.         <value>org.apache.ranger.admin.client.RangerAdminRESTClient</value>
    27.     </property>
    28.     <property>
    29.         <name>ranger.plugin.doris.service.name</name>
    30.         <value>doris</value>
    31.     </property>
    32. </configuration>

    You need to change ranger.plugin.doris.policy.cache.dir and ranger.plugin.doris.policy.rest.url to actual values.

  2. Create a log4j.properties file in the conf directory of all FEs with the following content:

    1. log4j.rootLogger = debug,stdout,D
    3. log4j.appender.stdout = org.apache.log4j.ConsoleAppender
    4. log4j.appender.stdout.Target = System.out
    5. log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
    6. log4j.appender.stdout.layout.ConversionPattern = [%-5p] %d{yyyy-MM-dd HH:mm:ss,SSS} method:%l%n%m%n
    8. log4j.appender.D = org.apache.log4j.DailyRollingFileAppender
    9. log4j.appender.D.File = /path/to/fe/log/ranger.log
    10. log4j.appender.D.Append = true
    11. log4j.appender.D.Threshold = INFO
    12. log4j.appender.D.layout = org.apache.log4j.PatternLayout
    13. log4j.appender.D.layout.ConversionPattern = %-d{yyyy-MM-dd HH:mm:ss}  [ %t:%r ] - [ %p ]  %m%n

    You need to change log4j.appender.D.File to the actual value, which is used to store the log of the Ranger plug-in.

  3. Add configuration in fe.conf of all FEs:

    access_controller_type=ranger-doris

  4. Restart all FE nodes.

Resources and permissions

  1. Doris resources currently supported in Ranger include:

    • Catalog
    • Database
    • Table
    • Column
    • Resource
    • Workload Group

  2. Doris permissions currently supported in Ranger include:

    • SHOW
    • SHOW_VIEW
    • SHOW_RESOURCES
    • SHOW_WORKLOAD_GROUP
    • LOAD
    • ALTER
    • CREATE
    • ALTER_CREATE
    • ALTER_CREATE_DROP
    • DROP
    • SELECT

Best Practices

Example 1

  1. Create user1 in Doris.

  2. In Doris, first create a Catalog: hive using the admin user.

  3. Create user1 in Ranger.

  4. Add a Policy in Ranger: show_hive_catalog

    Integration with Apache Ranger - 图3

  5. Use user1 to log in to Doris and execute show catalogs. Only the hive catalog can be seen.

  6. Add a Policy in Ranger: select_hive_catalog

    Integration with Apache Ranger - 图4

  7. Log in to Doris using user1. This user can view or query all tables under the hive catalog and all databases starting with tpch.

FAQ

  1. The column permissions, row permissions and Data Mask in Ranger are not supported yet.