Integration with Apache Ranger

Apache Ranger is a security framework used to monitor, enable services, and manage all-round data security access on the Hadoop platform.

In version 2.1.0, Doris supports unified permission management by integrating Apache Ranger.

Note:

  • This feature is currently experimental, and the resource objects and permissions configurable in Ranger may change in subsequent versions.

  • Apache Ranger version needs to be above 2.4.0.

Installation

Install Doris Ranger plug-in

  1. Download the following files

  2. Place the downloaded file in the ranger-plugins/doris directory of the Ranger service, such as:

    1. /usr/local/service/ranger/ews/webapp/WEB-INF/classes/ranger-plugins/doris/ranger-doris-plugin-3.0.0-SNAPSHOT.jar
    2. /usr/local/service/ranger/ews/webapp/WEB-INF/classes/ranger-plugins/doris/mysql-connector-java-8.0.25.jar

  3. Restart the Ranger service.

  4. Download ranger-servicedef-doris.json

  5. Execute the following command to upload the definition file to the Ranger service:

    1. curl -u user:password -X POST \
    2.     -H "Accept: application/json" \
    3.     -H "Content-Type: application/json" \
    4.     http://172.21.0.32:6080/service/plugins/definitions \
    5.     -d@ranger-servicedef-doris.json

    The username and password are the username and password used to log in to Ranger WebUI.

    The service address port can be viewed in the ranger.service.http.port configuration item of the ranger-admin-site.xml configuration file.

    If the execution is successful, the service definition in Json format will be returned, such as:

    1. {
    2.   "id": 207,
    3.   "guid": "d3ff9e41-f9dd-4217-bb5f-3fa9996454b6",
    4.   "isEnabled": true,
    5.   "createdBy": "Admin",
    6.   "updatedBy": "Admin",
    7.   "createTime": 1705817398112,
    8.   "updateTime": 1705817398112,
    9.   "version": 1,
    10.   "name": "doris",
    11.   "displayName": "Apache Doris",
    12.   "implClass": "org.apache.ranger.services.doris.RangerServiceDoris",
    13.   "label": "Doris",
    14.   "description": "Apache Doris",
    15.   "options": {
    16.     "enableDenyAndExceptionsInPolicies": "true"
    17.   },
    18.   ...
    19. }

    If you want to recreate it, you can use the following command to delete the service definition and then upload it again:

    1. curl -v -u user:password -X DELETE \
    2. http://172.21.0.32:6080/service/plugins/definitions/207

    Where 207 is the id returned when created. Before deletion, you need to delete the created Doris service in the Ranger WebUI.

    You can also use the following command to list the currently added service definitions in order to obtain the id:

    1. curl -v -u user:password -X GET \
    2. http://172.21.0.32:6080/service/plugins/definitions/

Configure the Doris Ranger plug-in

After the installation is complete, open the Ranger WebUI and you can see the Apache Doris plug-in in the Service Manger interface:

ranger1

Click the + button next to the plugin to add a Doris service:

ranger2

The meaning of some parameters of Config Properties is as follows:

  • Username/Password: the username and password of the Doris cluster. It is recommended to use the Admin user here.
  • jdbc.driver_class: Connect to the JDBC driver used by Doris. com.mysql.cj.jdbc.Driver
  • jdbc.url: JDBC url connection string of Doris cluster. jdbc:mysql://172.21.0.101:9030?useSSL=false
  • Additional parameters:
    • resource.lookup.timeout.value.in.ms: timeout for obtaining meta-information. It is recommended to fill in 10000, which is 10 seconds.

You can click Test Connection to check whether the connection can be made.

Then click Add to add the service.

Afterwards, you can see the created service in the Apache Doris plug-in on the Service Manger page. Click on the service to start configuring Ranger.

Configure Doris cluster

  1. Create a ranger-doris-security.xml file in the conf directory of all FEs with the following content:

    1. <?xml version="1.0" encoding="UTF-8"?>
    2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    3. <configuration>
    4.     <property>
    5.         <name>ranger.plugin.doris.policy.cache.dir</name>
    6.         <value>/path/to/ranger/cache/</value>
    7.     </property>
    8.     <property>
    9.         <name>ranger.plugin.doris.policy.pollIntervalMs</name>
    10.         <value>30000</value>
    11.     </property>
    12.     <property>
    13.         <name>ranger.plugin.doris.policy.rest.client.connection.timeoutMs</name>
    14.         <value>60000</value>
    15.     </property>
    16.     <property>
    17.         <name>ranger.plugin.doris.policy.rest.client.read.timeoutMs</name>
    18.         <value>60000</value>
    19.     </property>
    20.     <property>
    21.         <name>ranger.plugin.doris.policy.rest.url</name>
    22.         <value>http://172.21.0.32:6080</value>
    23.     </property>
    24.     <property>
    25.         <name>ranger.plugin.doris.policy.source.impl</name>
    26.         <value>org.apache.ranger.admin.client.RangerAdminRESTClient</value>
    27.     </property>
    28.     <property>
    29.         <name>ranger.plugin.doris.service.name</name>
    30.         <value>doris</value>
    31.     </property>
    32. </configuration>

    You need to change ranger.plugin.doris.policy.cache.dir and ranger.plugin.doris.policy.rest.url to actual values.

  2. Create a ranger-doris-audit.xml file in the conf directory of all FEs with the following content:

    1. <?xml version="1.0" encoding="UTF-8"?>
    2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    3. <configuration>
    4. </configuration>

  3. Create a log4j.properties file in the conf directory of all FEs with the following content:

    1. log4j.rootLogger = debug,stdout,D
    3. log4j.appender.stdout = org.apache.log4j.ConsoleAppender
    4. log4j.appender.stdout.Target = System.out
    5. log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
    6. log4j.appender.stdout.layout.ConversionPattern = [%-5p] %d{yyyy-MM-dd HH:mm:ss,SSS} method:%l%n%m%n
    8. log4j.appender.D = org.apache.log4j.DailyRollingFileAppender
    9. log4j.appender.D.File = /path/to/fe/log/ranger.log
    10. log4j.appender.D.Append = true
    11. log4j.appender.D.Threshold = INFO
    12. log4j.appender.D.layout = org.apache.log4j.PatternLayout
    13. log4j.appender.D.layout.ConversionPattern = %-d{yyyy-MM-dd HH:mm:ss}  [ %t:%r ] - [ %p ]  %m%n

    You need to change log4j.appender.D.File to the actual value, which is used to store the log of the Ranger plug-in.

  4. Add configuration in fe.conf of all FEs:

    access_controller_type=ranger-doris

  5. Restart all FE nodes.

Resources and permissions

  1. Doris resources currently supported in Ranger include:

    • Catalog
    • Database
    • Table
    • Column
    • Resource
    • Workload Group

  2. Doris permissions currently supported in Ranger include:

    • SHOW
    • SHOW_VIEW
    • SHOW_RESOURCES
    • SHOW_WORKLOAD_GROUP
    • LOAD
    • ALTER
    • CREATE
    • ALTER_CREATE
    • ALTER_CREATE_DROP
    • DROP
    • SELECT
    • USAGE

Best Practices

Configuring Permissions

  1. Create user1 in Doris.

  2. Using the admin user in Doris, create a Catalog named hive.

  3. Create user1 in Ranger.

  4. Add a Policy in Ranger named show_hive_catalog

    ranger3

  5. Log in to Doris as user1 and execute show catalogs; only the hive catalog should be visible.

  6. Add a Policy in Ranger named select_hive_catalog

    ranger4

  7. Log in to Doris as user1. This user can now view or query all tables under any database starting with tpch in the hive catalog.

Row Policy Example

Supported in version 2.1.3

  1. Refer to Configuring Permissions to assign select permissions for the table internal.db1.user to user1.

  2. Add a Row Level Filter policy in Ranger

    Row Policy Example

  3. Log in to Doris as user1. Execute select * from internal.db1.user and only see data that satisfies id > 3 and age = 2.

Data Mask Example

Supported in version 2.1.3

  1. Refer to Configuring Permissions to assign select permissions for the table internal.db1.user to user1.

  2. Add a Masking policy in Ranger

    Data Mask Example

  3. Log in to Doris as user1. Execute select * from internal.db1.user and observe that the phone numbers are masked according to the specified rules.