Apache Hadoop with YARN: Configuring HBase Monitoring (RPM/DEB)

Supported versions: See the Apache Hadoop entries for Pepperdata 8.0.x in the table of Supported Platforms by Pepperdata Version

To configure HBase, activate Pepperdata in the HBase RegionServers, and add the required properties to the Pepperdata site configuration to enable collecting the HBase metrics. If there are different authentication types for the HBase Web UI and the other cluster services, override the Pepperdata-default setting (pepperdata.agent.hbase.http.authentication.type) for HBase. If the HTTP service policy for HBase is different than the HTTP service policy for YARN, override the Pepperdata-default setting (pepperdata.agent.hbase.http.policy) for HBase.

On This Page

Prerequisites
Task 1: Configure Pepperdata to Collect HBase Filesystem Metrics
Task 2: Turn On HBase Monitoring and Configure Ports and Polling
Task 3: (Different Kerberization for HBase Web UI and Other Cluster Services) Override the Pepperdata-Default Authentication Type for HBase
Task 4: (Different HTTP Service Policy for HBase YARN) Override the Pepperdata-Default HTTP Service Policy for HBase

Prerequisites

Before you perform any of this procedure’s tasks (Task 1–Task 4), ensure that you know the HBase URL endpoint’s HTTP service policy (HTTPS_ONLY or HTTP_ONLY). You’ll need to know the HTTP service policy when you:

Replace the {your-protocol} placeholder in the cURL command in Task 3: Override the Pepperdata-Default Authentication Type for HBase.
Replace the your-hbase-http-service-policy placeholder in the XML snippet in Task 4: Override the Pepperdata-Default HTTP Service Policy for HBase.

Task 1: Configure Pepperdata to Collect HBase Filesystem Metrics

To activate Pepperdata—inject the necessary instrumentation—for HBase in a manually configured cluster, add the activation script to each Hadoop service’s environment shell executable file.

Procedure

Beginning with any host, add the following snippet to the HBase RegionServer configuration file, hbase-env.sh, typically located in /etc/hbase/conf.

Important: Add the snippet to the end of the script.

This ensures that the activation script’s variable appends (YARN_NODEMANAGER_OPTS, YARN_RESOURCEMANAGER_OPTS, HBASE_REGIONSERVER_OPTS, and SPARK_SUBMIT_OPTS) are not overwritten by other assignments in the script.
```
PEPPERDATA_ACTIVATE_SCRIPT_PATH="/opt/pepperdata/supervisor/lib/pepperdata-activate.sh"
if [ -e $PEPPERDATA_ACTIVATE_SCRIPT_PATH ]; then
 . $PEPPERDATA_ACTIVATE_SCRIPT_PATH
fi
```
Repeat step 1 on every host in the cluster.
On every host in the cluster, restart the HBase RegionServer daemon.

Task 2: Turn On HBase Monitoring and Configure Ports and Polling

To configure Pepperdata to collect HBase metrics, set the pepperdata.agent.genericJsonFetch.hbase.enabled property in the Pepperdata site configuration to true, and optionally add properties to override the default ports and polling intervals for the RegionServer and Master Server.

Procedure

Beginning with any host, enable HBase metrics collection.
1. Open the host’s Pepperdata site file, pepperdata-site.xml, for editing.
  
  By default, the Pepperdata site file, pepperdata-site.xml, is located in /etc/pepperdata. If you customized the location, the file is specified by the PD_CONF_DIR environment variable. See Change the Location of pepperdata-site.xml for details.
2. Add the following snippet.
```

<property>
    <name>pepperdata.agent.genericJsonFetch.hbase.enabled</name>
    <value>true</value>
</property>
```
  Malformed XML files can cause operational errors that can be difficult to debug. To prevent such errors, we recommend that you use a linter, such as xmllint, after you edit any .xml configuration file.

(Optional) To use ports and settings other than the defaults, configure any/all of the override properties.

The table shows the default values.

Setting	Default Value
RegionServer port	16030
RegionServer polling interval	60000 milliseconds (1 minute)
HBase Master Server port	16010
HBase Master Server polling interval	60000 milliseconds (1 minute)

To change the RegionServer port, add the pepperdata.agent.hbase.jmx.regionserver.port property.

<!-- For an HBase Region Server port other than the default Hadoop port (16030) -->
<property>
  <name>pepperdata.agent.hbase.jmx.regionserver.port</name>
  <value>your-region-server-portnum</value>
</property>

To change the RegionServer polling interval, add the pepperdata.agent.hbase.web.regionservermetrics.monitor.interval property.

<!-- For an HBase Region Server polling interval other than the default (60000 milliseconds) -->
<property>
  <name>pepperdata.agent.hbase.web.regionservermetrics.monitor.interval</name>
  <value>your-region-server-polling-interval-in-milliseconds</value>
</property>

To change the Master Server port, add the pepperdata.agent.hbase.jmx.master.port property.

<!-- For an HBase Master Server port other than the default Hadoop port (16010) -->
<property>
  <name>pepperdata.agent.hbase.jmx.master.port</name>
  <value>your-master-server-portnum</value>
</property>

To change the Master Server polling interval, add the pepperdata.agent.hbase.web.mastermetrics.monitor.interval property.

<!-- For an HBase Master Server polling interval other than the default (60000 milliseconds) -->
<property>
  <name>pepperdata.agent.hbase.web.mastermetrics.monitor.interval</name>
  <value>your-master-server-polling-interval-in-milliseconds</value>
</property>

Malformed XML files can cause operational errors that can be difficult to debug. To prevent such errors, we recommend that you use a linter, such as xmllint, after you edit any .xml configuration file.

Save your changes and close the file.
Repeat steps 1–3 on every host in the cluster.
Restart the HBase RegionServer daemons on every host in the cluster.

Task 3: (Different Kerberization for HBase Web UI and Other Cluster Services) Override the Pepperdata-Default Authentication Type for HBase

Although the HBase Web UI and the remaining cluster services typically use the same authentication mechanism—and therefore by default, Pepperdata assumes that HBase Web UI is Kerberized or not according to whether the YARN services are Kerberized—they can be different. That is, a cluster can be Kerberized for other services, but not for the HBase Web UI, or vice versa (Kerberized HBase Web UI but nothing else Kerberized in the cluster).

Important: If the HBase Web UI uses the same authentication—Kerberized or not—as the rest of the cluster, skip this task.

Procedure

Determine/verify the authentication type for the HBase endpoint by running the following cURL command, where {your-protocol} is http or https:

curl --tlsv1.2 -kIs {your-protocol}://YOUR_HBASE_HOST:YOUR_PORT/jmx | grep WWW-Authenticate
- If the returned response is WWW-Authenticate: Negotiate, the authentication type (your-hbase-auth-type) is kerberos.
- Otherwise nothing is returned, and the authentication type (your-hbase-auth-type) is simple.
Beginning with any host, override its authentication settings for the HBase Web UI by assigning the authentication type you found in step 1 (your-base-auth-type) to the pepperdata.agent.hbase.http.authentication.type property.
1. Open the host’s Pepperdata site file, pepperdata-site.xml, for editing.
  
  By default, the Pepperdata site file, pepperdata-site.xml, is located in /etc/pepperdata. If you customized the location, the file is specified by the PD_CONF_DIR environment variable. See Change the Location of pepperdata-site.xml for details.
2. Add the following snippet.
  
  Be sure to substitute the authentication type for the HBase Web UI (simple or kerberos, as indicated by the cURL command’s output in step 1) for the your-hbase-auth-type placeholder.
```

<property>
    <name>pepperdata.agent.hbase.http.authentication.type</name>
    <value>your-hbase-auth-type</value>
</property>
```
  Malformed XML files can cause operational errors that can be difficult to debug. To prevent such errors, we recommend that you use a linter, such as xmllint, after you edit any .xml configuration file.
3. Save your changes and close the file.
Repeat step 2 on every host in the cluster.
On every host in the cluster, restart the PepAgent.

You can use either the service (if provided by your OS) or systemctl command:
- sudo service pepagentd restart
- sudo systemctl restart pepagentd

Task 4: (Different HTTP Service Policy for HBase YARN) Override the Pepperdata-Default HTTP Service Policy for HBase

Although HBase and YARN typically use the same HTTP service policy—and therefore by default, Pepperdata assumes that the service policy for HBase is HTTPS_ONLY or HTTP_ONLY according to the policy for YARN—they can be different. That is, HBase might be configured for HTTPS_ONLY when the HTTP service policy for YARN is HTTP_ONLY, or vice versa (HTTP_ONLY for HBase, but HTTPS_ONLY for YARN).

Important: If the HTTP service policy for HBase is the same as for YARN—both are HTTPS_ONLY or both are HTTP_ONLY—skip this task.

Procedure

Beginning with any host, override its HTTP service policy for HBase.
1. Open the host’s Pepperdata site file, pepperdata-site.xml, for editing.
  
  By default, the Pepperdata site file, pepperdata-site.xml, is located in /etc/pepperdata. If you customized the location, the file is specified by the PD_CONF_DIR environment variable. See Change the Location of pepperdata-site.xml for details.
2. Add the following snippet.
  
  Be sure to substitute the HTTP service policy for HBase (HTTPS_ONLY or HTTP_ONLY) for the your-hbase-http-service-policy placeholder.
```

<property>
  <name>pepperdata.agent.hbase.http.policy</name>
  <value>your-hbase-http-service-policy</value>
</property>
```
  Malformed XML files can cause operational errors that can be difficult to debug. To prevent such errors, we recommend that you use a linter, such as xmllint, after you edit any .xml configuration file.
3. Save your changes and close the file.
Repeat step 1 on every host in the cluster.
On every host in the cluster, restart the PepAgent.

You can use either the service (if provided by your OS) or systemctl command:
- sudo service pepagentd restart
- sudo systemctl restart pepagentd