第十七章 HUE安装与配置

Hue安装

环境说明

操作系统：Ubuntu 14.04

集群节点：

  • Master
  • slave1
  • slave2

hadoop用户为：root

这里我们将hue安装在Slave2节点上

安装编译hue需要的相关依赖

  1. sudo apt-get install ant gcc g++ libkrb5-dev libffi-dev libmysqlclient-dev libssl-dev libsasl2-dev libsasl2-modules-gssapi-mit libsqlite3-dev libtidy-0.99-0 libxml2-dev libxslt-dev make libldap2-dev maven python-dev python-setuptools libgmp3-dev

下载解压并移动

到官网下载对应tar包

  1. root@slave2:~$ sudo tar zxvf hue-3.10.0.tgz
  2. root@slave2:~$ sudo cp -R hue-3.10.0 /usr/local/hue

编译

  1. root@slave2:~$ cd /usr/local/hue
  2. root@slave2:/usr/local/hue# sudo make apps

添加hue用户并赋权

  1. root@slave2:/usr/local/hue# sudo adduser hue
  2. root@slave2:sudo chmod -R 775 /usr/local/hue
  3. root@slave2:sudo chown -R hue:hue /usr/local/hue

启动hue

  1. root@slave2:/usr/local/hue# ./build/env/bin/supervisor

打开slave2:8888查看到hue界面，代表hue安装成功。

下一步就是配置hue，使它能够管理hdfs、hive、hbase，并能使用Oozie、Pig等，将在下面的文章中给大家介绍。

Hue配置

配置集群的访问权限

由于hue的启动用户是hue，所以需要为hue添加集群的访问权限，在各节点的/usr/local/hadoop/etc/hadoop/core-site.xml，添加如下参数：

  1.  <property>
  2.       <name>hadoop.proxyuser.hue.hosts</name>
  3.       <value>*</value>
  4.  </property>
  5.  <property>
  6.       <name>hadoop.proxyuser.hue.groups</name>
  7.       <value>*</value>
  8.  </property>

配置完，记得重启hadoop集群

配置hdfs

配置/usr/local/hue/desktop/conf/hue.ini

1）配置hdfs的超级用户

  1.   # This should be the hadoop cluster admin
  2.   default_hdfs_superuser=root

2）hdfs相关配置

这里主要配置三项：fs_defaultfs、webhdfs_url、hadoop_conf_dir；

其中，webhdfs_url默认本身就是开启的，不需要在hadoop中特别开启。

  1.   [[hdfs_clusters]]
  2.     # HA support by using HttpFs
  3.     [[[default]]]
  5.       # Enter the filesystem uri
  6.       fs_defaultfs=hdfs://Master:8020
  8.       # NameNode logical name.
  9.       ## logical_name=
  11.       # Use WebHdfs/HttpFs as the communication mechanism.
  12.       # Domain should be the NameNode or HttpFs host.
  13.       # Default port is 14000 for HttpFs.
  14.       webhdfs_url=http://Master:50070/webhdfs/v1
  16.       # Change this if your HDFS cluster is Kerberos-secured
  17.       ## security_enabled=false
  19.       # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
  20.       # have to be verified against certificate authority
  21.       ## ssl_cert_ca_verify=True
  23.       # Directory of the Hadoop configuration
  24.       hadoop_conf_dir=/usr/local/hadoop/etc/hadoop

配置yarn

配置/usr/local/hue/desktop/conf/hue.ini；

主要配置四个地方：resourcemanager_host、resourcemanager_api_url、proxy_api_url、history_server_api_url。

  1. [[yarn_clusters]]
  3.     [[[default]]]
  4.       # Enter the host on which you are running the ResourceManager
  5.       resourcemanager_host=Master
  7.       # The port where the ResourceManager IPC listens on
  8.       ## resourcemanager_port=8032
  10.       # Whether to submit jobs to this cluster
  11.       submit_to=True
  13.       # Resource Manager logical name (required for HA)
  14.       ## logical_name=
  16.       # Change this if your YARN cluster is Kerberos-secured
  17.       ## security_enabled=false
  19.       # URL of the ResourceManager API
  20.       resourcemanager_api_url=http://Master:8088
  22.       # URL of the ProxyServer API
  23.       proxy_api_url=http://Master:8088
  25.       # URL of the HistoryServer API
  26.       history_server_api_url=http://Master:19888
  28.       # URL of the Spark History Server
  29.       ## spark_history_server_url=http://localhost:18088
  31.       # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
  32.       # have to be verified against certificate authority
  33.       ## ssl_cert_ca_verify=True

配置hive

1）首先配置hue.ini

主要配置两个地方：hive_server_host、hive_conf_dir。

  1. [beeswax]
  3.   # Host where HiveServer2 is running.
  4.   # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
  5.   hive_server_host=Master
  7.   # Port where HiveServer2 Thrift server runs on.
  8.   ## hive_server_port=10000
  10.   # Hive configuration directory, where hive-site.xml is located
  11.   hive_conf_dir=/usr/local/hive/conf
  13.   # Timeout in seconds for thrift calls to Hive service
  14.   ## server_conn_timeout=120

2）启动hive2

  1. root@Master:/usr/local/hive/bin# hive --service hiveserver2 &

配置hbase

1）首先配置hue.ini

主要配置两个地方：hbase_clusters、hbase_conf_dir。

  1. [hbase]
  2.   # Comma-separated list of HBase Thrift servers for clusters in the format of '(name|host:port)'.
  3.   # Use full hostname with security.
  4.   # If using Kerberos we assume GSSAPI SASL, not PLAIN.
  5.   hbase_clusters=(Cluster|Master:9090)
  7.   # HBase configuration directory, where hbase-site.xml is located.
  8.   hbase_conf_dir=/usr/local/hbase/conf
  10.   # Hard limit of rows or columns per row fetched before truncating.
  11.   ## truncate_limit = 500
  13.   # 'buffered' is the default of the HBase Thrift Server and supports security.
  14.   # 'framed' can be used to chunk up responses,
  15.   # which is useful when used in conjunction with the nonblocking server in Thrift.
  16.   ## thrift_transport=buffered

2）启动thrift

  1. root@Master:/usr/local/hbase/bin# hbase-daemon.sh start thrift

特别注意：这里的thrift必须是1，而不是thrift2

启动hue

  1. root@slave2:/usr/local/hue# ./build/env/bin/supervisor

打开slave2:8888/about/查看到hue界面，如果页面中没有报hdfs、yarn、hbase、hive相关的警告则代表配置成功，之后就能在hue中使用相关的功能。

但是，我们可能会看到如下警告：

  1. SQLITE_NOT_FOR_PRODUCTION_USE    SQLite is only recommended for small development environments with a few users.
  2. Impala                            No available Impalad to send queries to.
  3. Oozie Editor/Dashboard            The app won't work without a running Oozie server
  4. Pig Editor                        The app won't work without a running Oozie server
  5. Spark                            The app won't work without a running Livy Spark Server

那是由于我们没有安装和配置相应功能，该块内容，将在后续文章中补充。