Disabling Components/Uninstalling Pepperdata (Cloud)

Some Pepperdata products can be individually disabled, while other products are always on as long as Pepperdata is running. This section includes procedures for all the products that can be individually disabled, and explains their interdependencies.

Uninstall All Pepperdata Products

To uninstall all Pepperdata products and functions, perform the configuration and installation procedures in reverse.

Use the distribution-specific configuration and installation documentation as a guide. More explicit instructions cannot be provided because the actual configuration you implemented during the installation and configuration steps is unique to your environment. If you need help, contact Pepperdata Support.

Disable Capacity Optimizer

Capacity Optimizer operates (is enabled) on a percentage of the hosts in the cluster as defined by the pepperdata.supervisor.splitTest.percent property value. To disable Capacity Optimizer, reset this property value to zero, and load the configuration change by refreshing the YARN ResourceManager configuration.

Procedure

1. In your cloud environment (such as GDP or AWS), reset the pepperdata.supervisor.splitTest.percent property in the Pepperdata configuration.

   1. From the environment’s cluster configuration folder (in the cloud), download the Pepperdata site file, /etc/pepperdata/pepperdata-site.xml, to a location where you can edit it.

   2. Open the file for editing, find the pepperdata.supervisor.splitTest.percent property, and change its value to 0.

      <property>
        <name>pepperdata.supervisor.splitTest.percent</name>
        <value>0</value>
      </property>
      

      Malformed XML files can cause operational errors that can be difficult to debug. To prevent such errors, we recommend that you use a linter, such as xmllint, after you edit any .xml configuration file.

   3. Save your changes and close the file.

   4. Upload the revised file to overwrite the original pepperdata-site.xml file.

   If there are no already-running hosts with Pepperdata, you are done with this procedure. Do not perform the remaining steps.

2. Open a command shell (terminal session) and log in to any already-running host as a user with sudo privileges.

   Important: You can begin with any host on which Pepperdata is running, but be sure to repeat the login (this step), copying the bootstrap file (next step), and loading the revised Pepperdata configuration (the following step) on every already-running host.

3. From the command line, copy the Pepperdata bootstrap script that you extracted from the Pepperdata package from its local location to any location; in this procedure’s steps, we’ve copied it to /tmp.

   • For Amazon EMR clusters:

     aws s3 cp s3://<pd-bootstrap-script-from-install-packages> /tmp/bootstrap

   • For Google Dataproc clusters:

     sudo gsutil cp gs://<pd-bootstrap-script-from-install-packages> /tmp/bootstrap

4. Load the revised configuration by running the Pepperdata bootstrap script.

   • For EMR clusters:

     • You can use the --long-options form of the --bucket, --upload-realm, and --is-running arguments as shown or their -short-option equivalents, -b, -u, and -r.

     • The --is-running (-r) option is required for bootstrapping an already-running host prior to Supervisor version 7.0.13.

     • Optionally, you can specify a proxy server for the AWS Command Line Interface (CLI) and Pepperdata-enabled cluster hosts.

       Include the --proxy-address (or --emr-proxy-address for Supervisor version 8.0.24 or later) argument when running the Pepperdata bootstrap script, specifying its value as a fully-qualified host address that uses https protocol.

     • If you’re using a non-default EMR API endpoint (by using the --endpoint-url argument), include the --emr-api-endpoint argument when running the Pepperdata bootstrap script. Its value must be a fully-qualified host address. (It can use http or https protocol.)

     • If you are using a script from an earlier Supervisor version that has the --cluster or -c arguments instead of the --upload-realm or -u arguments (which were introduced in Supervisor v6.5), respectively, you can continue using the script and its old arguments. They are backward compatible.

     • Optionally, you can override the default exponential backoff and jitter retry logic for the describe-cluster command that the Pepperdata bootstrapping uses to retrieve the cluster’s metadata.

       Specify either or both of the following options in the bootstrap’s Optional arguments. Be sure to substitute your values for the <my-retries> and <my-timeout> placeholders that are shown in the command.

       • max-retry-attempts—(default=10) Maximum number of retry attempts to make after the initial describe-cluster call.

       • max-timeout—(default=60) Maximum number of seconds to wait before the next retry call to describe-cluster. The actual wait time for a given retry is assigned as a random number, 1–calculated timeout (inclusive), which introduces the desired jitter.

   # For Supervisor versions before 7.0.13:
   sudo bash /tmp/bootstrap --bucket <bucket-name> --upload-realm <realm-name> --is-running [--proxy-address <proxy-url:proxy-port>] [--emr-api-endpoint <endpoint-url:endpoint-port>] [--max-retry-attempts <my-retries>] [--max-timeout <my-timeout>]
      
   # For Supervisor versions 7.0.13 to 8.0.23:
   sudo bash /tmp/bootstrap --bucket <bucket-name> --upload-realm <realm-name> [--proxy-address <proxy-url:proxy-port>] [--emr-api-endpoint <endpoint-url:endpoint-port>] [--max-retry-attempts <my-retries>] [--max-timeout <my-timeout>]
      
   # For Supervisor versions 8.0.24 and later:
   sudo bash /tmp/bootstrap --bucket <bucket-name> --upload-realm <realm-name> [--emr-proxy-address <proxy-url:proxy-port>] [--emr-api-endpoint <endpoint-url:endpoint-port>] [--max-retry-attempts <my-retries>] [--max-timeout <my-timeout>]
   

   • For Dataproc clusters:

     sudo bash /tmp/bootstrap <bucket-name> <realm-name>

   The script finishes with a Pepperdata installation succeeded message.

5. Repeat steps 2–4 on every already-running host in your cluster.

Disable Application Profiler

1. In your cloud environment (such as GDP or AWS), change the PD_JOBHISTORY_MONITOR_ENABLED setting in the Pepperdata configuration.

   1. From the environment’s cluster configuration folder (in the cloud), download the Pepperdata configuration file, /etc/pepperdata/pepperdata-config.sh, to a location where you can edit it.

   2. Open the file for editing, find the PD_JOBHISTORY_MONITOR_ENABLED property, and change its value to 0.

      export PD_JOBHISTORY_MONITOR_ENABLED=0
      

   3. Save your changes and close the file.

   4. Upload the revised file to overwrite the original pepperdata-config.sh file.

   If there are no already-running hosts with Pepperdata, you are done with this procedure. Do not perform the remaining steps.

2. Open a command shell (terminal session) and log in to any already-running host as a user with sudo privileges.

   Important: You can begin with any host on which Pepperdata is running, but be sure to repeat the login (this step), copying the bootstrap file (next step), and loading the revised Pepperdata configuration (the following step) on every already-running host.

3. From the command line, copy the Pepperdata bootstrap script that you extracted from the Pepperdata package from its local location to any location; in this procedure’s steps, we’ve copied it to /tmp.

   • For Amazon EMR clusters:

     aws s3 cp s3://<pd-bootstrap-script-from-install-packages> /tmp/bootstrap

   • For Google Dataproc clusters:

     sudo gsutil cp gs://<pd-bootstrap-script-from-install-packages> /tmp/bootstrap

4. Load the revised configuration by running the Pepperdata bootstrap script.

   • For EMR clusters:

     • You can use the --long-options form of the --bucket, --upload-realm, and --is-running arguments as shown or their -short-option equivalents, -b, -u, and -r.

     • The --is-running (-r) option is required for bootstrapping an already-running host prior to Supervisor version 7.0.13.

     • Optionally, you can specify a proxy server for the AWS Command Line Interface (CLI) and Pepperdata-enabled cluster hosts.

       Include the --proxy-address (or --emr-proxy-address for Supervisor version 8.0.24 or later) argument when running the Pepperdata bootstrap script, specifying its value as a fully-qualified host address that uses https protocol.

     • If you’re using a non-default EMR API endpoint (by using the --endpoint-url argument), include the --emr-api-endpoint argument when running the Pepperdata bootstrap script. Its value must be a fully-qualified host address. (It can use http or https protocol.)

     • If you are using a script from an earlier Supervisor version that has the --cluster or -c arguments instead of the --upload-realm or -u arguments (which were introduced in Supervisor v6.5), respectively, you can continue using the script and its old arguments. They are backward compatible.

     • Optionally, you can override the default exponential backoff and jitter retry logic for the describe-cluster command that the Pepperdata bootstrapping uses to retrieve the cluster’s metadata.

       Specify either or both of the following options in the bootstrap’s Optional arguments. Be sure to substitute your values for the <my-retries> and <my-timeout> placeholders that are shown in the command.

       • max-retry-attempts—(default=10) Maximum number of retry attempts to make after the initial describe-cluster call.

       • max-timeout—(default=60) Maximum number of seconds to wait before the next retry call to describe-cluster. The actual wait time for a given retry is assigned as a random number, 1–calculated timeout (inclusive), which introduces the desired jitter.

   # For Supervisor versions before 7.0.13:
   sudo bash /tmp/bootstrap --bucket <bucket-name> --upload-realm <realm-name> --is-running [--proxy-address <proxy-url:proxy-port>] [--emr-api-endpoint <endpoint-url:endpoint-port>] [--max-retry-attempts <my-retries>] [--max-timeout <my-timeout>]
      
   # For Supervisor versions 7.0.13 to 8.0.23:
   sudo bash /tmp/bootstrap --bucket <bucket-name> --upload-realm <realm-name> [--proxy-address <proxy-url:proxy-port>] [--emr-api-endpoint <endpoint-url:endpoint-port>] [--max-retry-attempts <my-retries>] [--max-timeout <my-timeout>]
      
   # For Supervisor versions 8.0.24 and later:
   sudo bash /tmp/bootstrap --bucket <bucket-name> --upload-realm <realm-name> [--emr-proxy-address <proxy-url:proxy-port>] [--emr-api-endpoint <endpoint-url:endpoint-port>] [--max-retry-attempts <my-retries>] [--max-timeout <my-timeout>]
   

   • For Dataproc clusters:

     sudo bash /tmp/bootstrap <bucket-name> <realm-name>

   The script finishes with a Pepperdata installation succeeded message.

5. Repeat steps 2–4 on every already-running host in your cluster.