Adding Apache® Impala Query Metrics (Parcel)

Supported Versions of Impala: Versions supported by any Cloudera CDH/CDP distribution that Pepperdata supports; see Pepperdata-Platform Support

Cluster administrators often need precise resource usage information, even down to the query level of detail, in order to create accurate chargeback reports. If you’re using Apache Impala, you can enable Pepperdata to collect Impala query metrics for CPU and memory usage. When the queries are finished, Pepperdata reads the Impala query profiles to calculate the resource usage.

For more information about Apache® Impala query metrics, see Monitoring Apache® Impala Query Metrics.

Pepperdata surfaces Apache® Impala query metrics for two distinct purposes: Charts and Query Spotlight.

• When you enable Impala query monitoring, the metrics appear in the Charts page.

• When you enable Impala query monitoring and Query Spotlight, the metrics appear in both the Charts and in Query Spotlight.

• If you do not enable Impala query monitoring, but you do enable Query Spotlight, Impala metrics do not appear in Query Spotlight, but Query Spotlight will monitor any other query types that you’ve configured Pepperdata to gather.

Prerequisites

Before you enable Impala query monitoring, ensure that your system meets the required prerequisites.

Pepperdata PepAgent (pepagentd) and PepCollector (pepcollectd) must be installed and running on all hosts on which the Impala impalad daemon is running.

Procedure

If you already added Impala query metrics for Query Spotlight, you do not need to add them again, and you should skip this procedure.

Add the Impala query properties to the Pepperdata configuration to enable query monitoring and to configure where to read the query profiles from.
1. Locate the Pepperdata > Configuration > Pepperdata (Service-Wide) > Enable Impala query monitoring parameter, and select it.
2. (Optional) To configure a non-default location—any location other than /var/log/impalad/profiles/—for where to read the query profiles, add the following snippet to the Pepperdata > Service Wide > PepAgent Advanced Configuration Snippet (Safety Valve) for conf/pepperdata-site.xml template, as an XML block.
  
  Be sure to substitute your location of your query profiles for the your-impalad-profiles-location placeholder.
```
<property>
  <name>pepperdata.impala.query.queryLogDir</name>
  <value>your-impalad-profiles-location</value>
</property>
```
(HTTPS impalad daemon endpoints) If your impalad daemon is configured for HTTPS instead of HTTP, add the pepperdata.agent.genericJsonFetch.impala.httpsEnabled property so that the fetcher for information about Impala queries in flight uses the HTTPS endpoint instead of the default HTTP endpoint (http://LOCALHOST:25000/queries?json).

Add the following snippet to the Pepperdata > Service Wide > PepAgent Advanced Configuration Snippet (Safety Valve) for conf/pepperdata-site.xml template, as an XML block.
```
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.httpsEnabled</name>
  <value>true</value>
</property>
```
(Digest authentication for the Impala Web UI for debugging) If the impalad daemon for your Impala Web UI for debugging is secured by digest authentication, add the authentication credentials.

Note: This step is for using digest auth to secure the impalad daemon of the Impala Web UI for debugging , not for securing the Impala core services with Kerberos or LDAP.

Add the following snippet to the Pepperdata > Service Wide > PepAgent Advanced Configuration Snippet (Safety Valve) for conf/pepperdata-site.xml template, as an XML block. Be sure to substitute your username and password for the your-username and your-password placeholders in the following code snippet.
```
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.http.authentication.type</name>
  <value>digest</value>
</property>
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.auth.username</name>
  <value>your-username</value>
</property>
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.auth.password</name>
  <value>your-password</value>
</property>
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.httpsEnabled</name>
  <value>true</value>
</property>
```
(Kerberos for the Impala Web UI for debugging) If the impalad daemon for your Impala Web UI for debugging is Kerberized, add the authentication credentials.

Note: This step is for using Kerberos to secure the impalad daemon of the Impala Web UI for debugging , not for securing the Impala core services with Kerberos or LDAP.

Add the following snippet to the Pepperdata > Service Wide > PepAgent Advanced Configuration Snippet (Safety Valve) for conf/pepperdata-site.xml template, as an XML block.

Be sure to substitute your Kerberos principal and the path of the corresponding keytab file for the your-kerberos-principal and your-kerberos-keytab-pathname placeholders in the following code snippet.

If you already configured the cluster for Kerberos during the installation process (that is, you you selected Enable Access to Kerberized Cluster Components in Task 2: Add Pepperdata Service to Cloudera Manager), do not assign the fetcher properties except to override the cluster-level assignments.

The fetcher properties (pepperdata.agent.genericJsonFetch.impala.kerberos.principal and pepperdata.agent.genericJsonFetch.impala.keytab.location) are inherited from the cluster-level properties that were automatically assigned when you configured the cluster.
```
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.http.authentication.type</name>
  <value>kerberos</value>
</property>
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.kerberos.principal</name>
  <value>your-kerberos-principal</value>
</property>
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.keytab.location</name>
  <value>your-kerberos-keytab-pathname</value>
</property>
<property>
  <name>pepperdata.agent.genericJsonFetch.impala.httpsEnabled</name>
  <value>true</value>
</property>
```
(Re)start the PepAgent.
- If the Pepperdata services are not yet running, in Cloudera Manager, select the Start action for the Pepperdata service.
- Otherwise, in Cloudera Manager, select the Restart action for the Pepperdata service.
Contact Pepperdata Support to request that Impala query metrics be activated for your Pepperdata dashboard.