Connecting to Impala Daemon from impala-shell

Within an impala-shell session, you can only issue queries while connected to an instance of the impalad daemon. You can specify the connection information:

  • Through command-line options when you run the impala-shell command.
  • Through a configuration file that is read when you run the impala-shell command.
  • During an impala-shell session, by issuing a CONNECT command.

Note: You cannot connect to the 3.2 or earlier versions of Impala using the 'hs2' or 'hs2-http' protocol (--protocol option).

See impala-shell Configuration Options for the command-line and configuration file options you can use.

You can connect to any Impala daemon (impalad), and that daemon coordinates the execution of all queries sent to it.

For simplicity during development, you might always connect to the same host, perhaps running impala-shell on the same host as impalad and specifying the hostname as localhost.

In a production environment, you might enable load balancing, in which you connect to specific host/port combination but queries are forwarded to arbitrary hosts. This technique spreads the overhead of acting as the coordinator node among all the Impala daemons in the cluster. See Using Impala through a Proxy for High Availability for details.

To connect to an Impala during shell startup:

  1. Locate the hostname that is running an instance of the impalad daemon. If that impalad uses a non-default port (something other than port 21000) for impala-shell connections, find out the port number also.

  2. Use the -i option to the impala-shell interpreter to specify the connection information for that instance of impalad:

    1. # When you are connecting to an impalad running on the same machine.
    2. # The prompt will reflect the current hostname.
    3. $ impala-shell
    5. # When you are connecting to an impalad running on a remote machine, and impalad is listening
    6. # on a non-default port over the HTTP HiveServer2 protocol.
    7. $ impala-shell -i some.other.hostname:port_number --protocol='hs2-http'
    8. # When you are connecting to an impalad running on a remote machine, and impalad is listening
    9. # on a non-default port.
    10. $ impala-shell -i some.other.hostname:port_number

To connect to an Impala in theimpala-shell session:

  1. Start the Impala shell with no connection:

    1. $ impala-shell

  2. Locate the hostname that is running the impalad daemon. If that impalad uses a non-default port (something other than port 21000) for impala-shell connections, find out the port number also.

  3. Use the connect command to connect to an Impala instance. Enter a command of the form:

    1. [Not connected] > connect impalad-host

    Note: Replace impalad-host with the hostname you have configured to run Impala in your environment. The changed prompt indicates a successful connection.

To start impala-shell in a specific database:

You can use all the same connection options as in previous examples. For simplicity, these examples assume that you are logged into one of the Impala daemons.

  1. Find the name of the database containing the relevant tables, views, and so on that you want to operate on.

  2. Use the -d option to the impala-shell interpreter to connect and immediately switch to the specified database, without the need for a USE statement or fully qualified names:

    1. # Subsequent queries with unqualified names operate on
    2. # tables, views, and so on inside the database named 'staging'.
    3. $ impala-shell -i localhost -d staging
    5. # It is common during development, ETL, benchmarking, and so on
    6. # to have different databases containing the same table names
    7. # but with different contents or layouts.
    8. $ impala-shell -i localhost -d parquet_snappy_compression
    9. $ impala-shell -i localhost -d parquet_gzip_compression

To run one or several statements in non-interactive mode:

You can use all the same connection options as in previous examples. For simplicity, these examples assume that you are logged into one of the Impala daemons.

  1. Construct a statement, or a file containing a sequence of statements, that you want to run in an automated way, without typing or copying and pasting each time.

  2. Invoke impala-shell with the -q option to run a single statement, or the -f option to run a sequence of statements from a file. The impala-shell command returns immediately, without going into the interactive interpreter.

    1. # A utility command that you might run while developing shell scripts
    2. # to manipulate HDFS files.
    3. $ impala-shell -i localhost -d database_of_interest -q 'show tables'
    5. # A sequence of CREATE TABLE, CREATE VIEW, and similar DDL statements
    6. # can go into a file to make the setup process repeatable.
    7. $ impala-shell -i localhost -d database_of_interest -f recreate_tables.sql

Parent topic: Using the Impala Shell (impala-shell Command)