Cloudera Manager with CDH/CDP Private Cloud Base - YARN: Configuring Pepperdata Activation (RPM/DEB)

Supported versions: See the CDH and CDP Private Cloud Base entries for Pepperdata 8.0.x in the table of Supported Platforms by Pepperdata Version

To activate Pepperdata—inject the necessary instrumentation—across a Cloudera Manager configured cluster, add Pepperdata activation environment variables or Java options to each Hadoop service’s configuration options.

Procedure

Use Cloudera Manager to add the snippets described in this procedure’s steps.

  1. Instrument the ResourceManager.

    Add the following snippet to the YARN (MR2 Included) > Configuration > ResourceManager > Java Configuration Options for ResourceManager template.

    -Dpepperdata.class.transformer=com.pepperdata.supervisor.classpatcherlib.ClassTransformerForYarn3Daemon -Dpepperdata.class.patching.enabled=true -javaagent:/opt/pepperdata/lib/PepperdataSupervisor.jar
    
    The snippets are different for instrumenting the ResourceManager and NodeManager.
  2. Instrument the NodeManager.

    Add the following snippet to the YARN (MR2 Included) > Configuration > NodeManager > Java Configuration Options for NodeManager template.

    -Dpepperdata.nodemanager.connector.enabled=true -Dpepperdata.class.transformer=com.pepperdata.supervisor.classpatcherlib.ClassTransformerForYarn3Daemon -Dpepperdata.class.patching.enabled=true -javaagent:/opt/pepperdata/lib/PepperdataSupervisor.jar
    
    The snippets are different for instrumenting the NodeManager and ResourceManager.
  3. Instrument the Spark service.

    Add the following snippet to the Spark > Configuration > Gateway > Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-env.sh template.

    PEPPERDATA_ACTIVATE_SCRIPT_PATH="/opt/pepperdata/supervisor/lib/pepperdata-activate.sh"
    if [ -e $PEPPERDATA_ACTIVATE_SCRIPT_PATH ]; then
     . $PEPPERDATA_ACTIVATE_SCRIPT_PATH
    fi
    
  4. Instrument the Spark 2 service.

    Add the following snippet to the Spark 2 > Configuration > Gateway > Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-env.sh template.

    PEPPERDATA_ACTIVATE_SCRIPT_PATH="/opt/pepperdata/supervisor/lib/pepperdata-activate.sh"
    if [ -e $PEPPERDATA_ACTIVATE_SCRIPT_PATH ]; then
     . $PEPPERDATA_ACTIVATE_SCRIPT_PATH
    fi
    
  5. Restart the following application daemons:

    • YARN ResourceManager
    • YARN NodeManagers (all in the cluster)