Pepperdata Release Notes

On This Page

The release notes for a given version of Pepperdata Supervisor and Pepperdata CSDs (custom service descriptors for Cloudera Manager) summarize the bug fixes and new features for that version.

  • Unless identified as a Migration CSD, CSDs are Supervisor CSDs.
  • Dates are in yyyy/mm/dd format.

Supervisor v7.1.13 (2022/11/02)

Enhancements

  • Supports newer k8s version 1.22, 1.23, 1.24.

    Pepperdata now supports the following additional Kubernetes versions: 1.22, 1.23, and 1.24. See the support matrix.

  • Supports Red Hat Enterprise Linux (RHEL) version 8 for Cloudera Data Platform (CDP) installations.

Bug Fixes

  • Optimizer pod doesn’t restart on a license file change.

    Previously the optimizer pod failed to restart after a license file change. New installation/upgrade instructions include a modified helm upgrade command, which works only for this version and later. After you execute that command, your optimizer pod will restart properly after a license file change.

  • Reduced network bandwidth.

    Previously the pepcollectd daemon uploading data to Pepperdata servers might leave connections open when errors occurred, reducing effective network bandwidth. This no longer occurs.

Supervisor v7.0.14 (2022/11/02)

Bug Fixes

  • Negative value warnings.

    Previously Capacity Optimizer unnecessarily logged warnings when pending resources had negative values. Now it does not.

  • Failed installation.

    Previously installing Pepperdata on Kubernetes failed if the PD_UPLOAD_HOST config was missing. Now the installation proceeds regardless of the presence of that config.

  • Reduced network bandwidth.

    Previously the pepcollectd daemon uploading data to Pepperdata servers might leave connections open when errors occurred, reducing effective network bandwidth. This no longer occurs.

Supervisor v7.1.12 (2022/10/19)

Enhancement

  • Accurately calculate Pepperdata resources when Capacity Optimizer is enabled on Kubernetes clusters.

    Previously, Pepperdata counted only resources that were launched after Capacity Optimizer was enabled. Now it gives a more accurate picture.

Bug Fixes

  • License expiration or an invalid license disables autoscaling optimization on Kubernetes clusters.

    Previously, Pepperdata continued to optimize autoscaling after the required license expired. Now it stops optimizing autoscaling if the license expires or is invalid.

  • Supports all 3.x Helm versions.

    In Supervisor version 7.1.11, Pepperdata did not work properly with all Helm 3.x versions. Now it does.

  • Correctly mutates pods.

    Previously, the Pepperdata mutating webhook did not mutate pods correctly if the pod spec did not contain a limit field. Now it does.

CSD v3.1.5 (2022/09/30)

Bug Fix

In Pepperdata CSD v3.1.2 and later, when Pepperdata Capacity Optimizer configuration values are modified using the Cloudera Manager user interface, Pepperdata software fails to receive the updates automatically.

Now Pepperdata receives such updates automatically.

Supervisor v7.1.11 (2022/09/21)

Enhancements

  • Simplify port use by PepAgents.

    Previously PepAgents on Resource Manager hosts listened on both port 50510 and port 50505, which sometimes led to port conflicts with other processes.

    Services that used to listen on either of those ports now listen only on port 50505, removing the possibility of conflicts on port 50510.

  • Pepperdata now supports optimizing autoscaling on Microsoft Azure AKS clusters.

Bug Fix

  • Previously when Capacity Optimizer was disabled by a change in configuration parameters, all autoscaling changes remained in place.

    Now Capacity Optimizer resets autoscaling parameters and disables autoscaling optimization when this happens.

Supervisor v7.1.10 (2022/09/07)

Enhancements

  • Supervisor version 7.0.13 fixed a bug in YARN environments. When Capacity Optimizer started on a new RM node – either for the first time or because of failover – a configurable delay before autoscaling prevented suboptimal scaling. This feature was disabled by default.

    Now Capacity Optimizer sets the delay automatically as soon as it distinguishes the cluster start from an HA transition to active.

  • A new generic JSON fetcher collects additional cloud instance metadata for EMR.

Supervisor v7.1.9 (2022/08/24)

Enhancement

  • Previously Capacity Optimizer sometimes reported that a node was not fully utilized, even though the node had reached the maximum number of pods allowed by Kubernetes.

    Now it considers a node fully utilized if it reaches the maximum allowed number of pods, regardless of other considerations.

Bug Fixes

  • Previously Capacity Optimizer unnecessarily logged warnings when pending resources had negative values.

    Now it does not.

  • Previously Capacity Optimizer reported CPU usage in the wrong units on Kubernetes systems.

    Now it reports CPU values correctly.

  • Previously installing Pepperdata on Kubernetes failed if the PD_UPLOAD_HOST config was missing.

    Now the installation proceeds regardless of the presence of that config.

Streaming Spotlight (Deprecated)

Streaming Spotlight is deprecated as of August 5, 2022.

Supervisor v7.1.8 (2022/08/05)

This release introduces minor Supervisor version 7.1.8

Enhancements to Capacity Optimizer

Bug Fix

Previously, some Pepperdata services continued to work after the Pepperdata license expired. This is no longer the case.

Dropped Support

Query Spotlight no longer supports monitoring queries submitted via Hive CLI. Queries submitted via Beeline CLI or other JDBC clients remain unaffected.

Supervisor v7.0.13 (2022/07/15)

Enhancements

  • Previously, the EMR bootstrap script required users to specify whether the cluster is running.

    The script now determines this automatically.

  • Previously, the Capacity Optimizer in an EMR environment with automatic scaling with a custom policy optimized only upscaling, but not downscaling.

    Now it optimizes downscaling as well. This feature is disabled by default.

Bug Fixes

  • Previously, when the Pepperdata license expired, Pepperdata autoscaling optimization continued with a potential to destabilize the clusters.

    Now autoscaling is disabled when the license expires.

  • Previously, in YARN environments, when Capacity Optimizer started on a new RM node – either for the first time or because of failover – it began autoscaling optimization before the cluster had time to stabilize, resulting in suboptimal scaling.

    Now it waits a configurable time before beginning autoscaling. This feature is disabled by default.

Supervisor v7.0.12 (2022/06/17)

Enhancements

  • (Capacity Optimizer in cloud clusters) Changes the default for pessimistic mode from enabled to disabled. This ensures that by default, Capacity Optimizer does not throttle the maximum memory allocation, and can perform its normal operations.

    For more about pessimistic mode operation, see the Pessimistic Mode description in the Configure Capacity Optimizer procedure for your cloud environment: EMR: Pessimistic Mode or Dataproc: Pessimistic Mode (or corresponding pages for Pepperdata Supervisor versions newer than v7.0).

  • (Impala query monitoring in Query Spotlight) Adds support for Kerberized (Kerberos-secured) impalad daemons for the Impala Web UI for debugging. For the configuration procedure, see Configure Query Spotlight.

Bug Fix

(Capacity Optimizer in cloud clusters) Fixes an edge-case bug that caused Pepperdata autoscaling optimization to trigger pessimistic mode operation—throttling the maximum memory allocation to 100% of the baseline value defined by yarn.nodemanager.resource.memory-mb, regardless of the configured value of pepperdata.supervisor.allocation.maxMemory.percent—before confirming that more than one node was in the pessimistic state—over-allocated (by normal Capacity Optimizer operation).

Now Capacity Optimizer correctly triggers pessimistic mode operation only when both of the following are true:

  • Pessimistic mode is enabled (pepperdata.supervisor.pessimisticMode.enabled=true)
  • More than one node is in the pessimistic state (over-allocated, by normal Capacity Optimizer operation)

Supervisor v7.0.11 (2022/05/13)

Enhancements

  • (Amazon EMR clusters) Adds log data from failed pepagentd and pepcollectd daemons to the bootstrap log, to provide additional debugging information.

  • (Amazon EMR clusters) Improves usability and output messages of the check_IAM_role_permissions utility script for verifying that a cluster’s IAM role provides the necessary permissions for the Pepperdata bootstrap script’s actions. For details, see Verify that IAM Role Permissions are Sufficient (or corresponding page for Pepperdata Supervisor versions newer than v7.0).

Bug Fixes

  • (Capacity Optimizer in Amazon EMR clusters) Fixes an edge case race condition that on rare occasions caused Capacity Optimizer’s autoscaling optimization to stop its autoscaling operations.

  • (Capacity Optimizer on Kubernetes) Fixes an issue that prevented Capacity Optimizer from operating after a restart of the optimizer Pod—the Pod where Capacity Optimizer performs its calculations.

  • (Capacity Optimizer on Kubernetes) Fixes an edge case issue that caused Capacity Optimizer to become disabled on a small percentage of cluster nodes. Now when Capacity Optimizer is enabled, it always remains enabled on all cluster nodes.

  • (Query Spotlight) Fixes a case-sensitivity issue that prevented the pepperdata.agent.genericJsonFetch.presto.http.authentication.type value from being honored when it was specified as lowercase simple or basic. You can now use any case for the values; for example, basic, BASIC, and Basic.

  • (Platform Spotlight, with custom Certificate of Authority) For environments that use a custom Certificate of Authority (CA), but where the REQUESTS_CA_BUNDLE and SSL_CERT_FILE environment variables are not explicitly set to point to the custom CA file, fixes an issue that prevented the PepCollector (pepcollectd daemon) from uploading data to the Pepperdata dashboard. Now, when the custom CA file is located in any of the Linux vendor-provided default locations, pepcollectd finds the custom CA file, and successfully uploads the collected data.

    This bug affected only Supervisor v7.0.2 to v7.0.9 releases. If you are using any Supervisor v6.5.x or earlier release, you do not need to upgrade for this issue.

Supervisor v7.0.9 (2022/04/20)

Enhancements

  • (Capacity Optimizer on Kubernetes) You can now enable Capacity Optimizer to honor the Kubernetes Guaranteed Quality of Service (QoS) class information when deciding which Pods to over-allocate and preempt: if the request and the limit of a resource (CPU or memory) are the same, Capacity Optimizer will neither over-allocate nor preempt those Pods.

    To enable this capability for CPU and/or memory, configure the PD_K8S_CAPACITY_OPTIMIZER_HONOR_CPU_QOS and/or PD_K8S_CAPACITY_OPTIMIZER_HONOR_MEMORY_QOS properties, respectively; see Advanced Tuning: Honor the Kubernetes Guaranteed Quality of Service (QoS) Class (or corresponding page for Pepperdata Supervisor versions newer than v7.0).

  • (autoscaling optimization) By default, effectively disables the maximum calculation interval (pepperdata.managedAutoscaling.inactionTimeout) for performing scaling calculations in the absence of resource-related triggers (memory or CPU). When the interval is assigned a value that is too low (so too frequent calculations), it can cause unnecessary scaling behavior. Therefore it is best to keep it disabled until specific use cases show a need for it.

Bug Fix

(Capacity Optimizer on Kubernetes) Fixes an issue that prevented configuration changes from being picked up on a helm upgrade command on the PepAgent Helm chart.

Supervisor v7.0.8 (2022/04/06)

  • Platform and System Support

    • Adds support for Apache Spark 3.2.x.
    • Adds support for Amazon EMR 6.5.x.
    • Adds support for Amazon EMR 5.34.x.
  • Enhancements

    • (Capacity Optimizer in Kubernetes clusters) Adds Capacity Optimizer Configs metrics to the Pepperdata dashboard. To view the metrics, navigate to the Charts page, and search for the Capacity Optimizer Configs group.

    • (Google Dataproc clusters) Adds log data from failed pepagentd and pepcollectd daemons to the bootstrap log, to provide additional debugging information.

  • Bug Fix

    (Amazon EMR clusters with autoscaling) Fixes a race condition that on rare occasions prevented Pepperdata from correctly determining the cluster’s autoscaling policy type—EMR-managed scaling or custom automatic scaling policy.

Supervisor v7.0.7 (2022/03/25)

Enhancement

  • (Amazon EMR clusters) Adds exponential backoff and jitter retry logic to the Pepperdata bootstrapping. This is particularly helpful for creating transient clusters where API rate limits result in throttling describe-cluster commands.

    • By default, max-retry-attempts is 10, and max-timeout—the wait time before the next retry—is 60 seconds.
    • The actual wait time for a given retry is assigned as a random number, 1–calculated timeout (inclusive), which introduces the desired jitter.

    To modify the retry behavior, use the optional bootstrapping arguments max-retry-attempts and max-timeout, as explained in the procedures for creating clusters; for example, Create New Cluster with Pepperdata: v7.0 (or corresponding page for Pepperdata Supervisor versions newer than v7.0).

Bug Fixes

  • (autoscaling optimization, for EMR) Fixed an issue that caused Pepperdata to overlook nodes when their fully-qualified hostnames are different in the ResourceManager and the AWS API server.

  • Fixes a Supervisor v7.0.5-introduced issue that caused PepAgents to crash in environments where all the following are true:

Supervisor v6.5.29 (2022/03/23)

Enhancement

  • (Amazon EMR clusters) Adds exponential backoff and jitter retry logic to the Pepperdata bootstrapping. This is particularly helpful for creating transient clusters where API rate limits result in throttling describe-cluster commands.

    • By default, max-retry-attempts is 10, and max-timeout—the wait time before the next retry—is 60 seconds.
    • The actual wait time for a given retry is assigned as a random number, 1–calculated timeout (inclusive), which introduces the desired jitter.

    To modify the retry behavior, use the optional bootstrapping arguments max-retry-attempts and max-timeout, as explained in the procedures for creating clusters; for example, Create New Cluster with Pepperdata: v6.5 (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

Bug Fix

(autoscaling optimization, for EMR) Fixed an issue that caused Pepperdata to overlook nodes when their fully-qualified hostnames are different in the ResourceManager and the AWS API server.

Supervisor v7.0.5 (2022/02/23)

Enhancements

  • Adds explanatory error messages to the PepCollector (pepcollectd daemon) logfile when a failure of the upload host is detected.

  • (Capacity Optimizer in Kubernetes) Adds the ability to tune Capacity Optimizer by adding properties to the YAML configuration file. For details, see Tune Capacity Optimizer (or corresponding page for Pepperdata Supervisor versions newer than v7.0).

  • Migrates PepAgent logging from Log4j v1 to Log4j v2.17.1 (which fixed the Remote code injection in Log4j vulnerability identified in CVE-2021-44228). A side-effect of this enhancement is the removal of the supervisor.log and nodemanager.log logfiles.

    The version of Log4j v1 that Pepperdata used pre-migration was not a version that was affected by the Remote code injection in Log4j vulnerability identified in CVE-2021-44228, and therefore the Pepperdata software was never vulnerable.

    We’ve migrated to Log4j v2 because Log4j v1 reached its end of life (EOL), and is no longer officially supported.

Supervisor v6.5.28 (2022/02/23)

Enhancement

  • Removes Log4j v1 classes from Pepperdata instrumentation code. A side-effect of this enhancement is the removal of the supervisor.log and nodemanager.log logfiles.

    Pepperdata instrumentation code does not configure the Log4j v1 classes, but we are removing them altogether to prevent erroneous or malicious activation.

Bug Fixes

  • (Dataproc v2.0.29-debian10 and later) Fixes a Java path bug that caused the Pepperdata bootstrapping to fail.

  • (Kubernetes 1.21) Fixes a bug in versioning checks that prevented Pepperdata from working in clusters with Kubernetes 1.21.

Supervisor v7.0.2 (2022/02/09)

This release—the first for the new Supervisor v7.0 major release—includes General Availability for Capacity Optimizer on Kubernetes, changes in platform and system support, and rolls up the Supervisor v6.5 patch/maintenance enhancements and bug fixes from Supervisor v6.5.5–6.5.27.

  • Capacity Optimizer on Kubernetes

    Adds Capacity Optimizer on Kubernetes, as a GA (general availability) release.

  • Platform and System Support

    • Adds support for ARM64 on Amazon EMR for Graviton/Graviton 2 instances.

    • Drops support for distributions that are no longer supported by the vendor and for distributions based on Hadoop 2.8.x and earlier:

      • Hortonworks HDP®
      • Qubole® QDS
      • MapR
      • Cloudera’s Distribution of Hadoop (CDH) 6.1.x and earlier
      • Apache Hadoop 2.8.x and earlier
      • Amazon EMR 5.30.0 and earlier
    • Drops support for components associated with now-unsupported distributions:

      • Hive 1.x
      • Java 7
    • Migrates Pepperdata software to internally use Python 3 instead of Python 2. The sunset date for Python 2 was January 1, 2020, and it is no longer supported nor receiving security updates. The migration created significant changes, which motivated the update to a new major Supervisor release.

    • Drops support for the Pepperdata PepMetrics agent. Pepperdata now gathers HBase metrics via fetchers invoked by PepAgents.

  • Pepperdata in Kubernetes

    Changes the path for shipped Kubernetes files from */lib/experimental/kubernetes to */lib/kubernetes. The installation procedure is updated accordingly.

  • Rollup of Previous Enhancements and Bug Fixes

    Rolls up the Supervisor v6.5 patch/maintenance enhancements and bug fixes from Supervisor v6.5.5–6.5.27.

CSD v3.1.2 (2022/02/09)

Adds service configuration parameters for:

  • Securing (via HTTPS) the PepAgent and Supervisor web interfaces in clusters that are enabled for Auto-TLS.

  • Configuring JobHistory monitoring for Application Spotlight.

  • Enabling debug logging for PepAgent fetchers (Kafka, HBase, and so on). Debug messages are output to the PepAgent log file: /var/log/pepperdata/pepagent/pepagent.log (unless you have customized the location to be other than /var/log/pepperdata/).

Supervisor v6.5.27 (2022/01/26)

Enhancement

  • Removes the Log4j v1.2 JMSAppender class from the PepAgent (pepagentd) and PepCollector (pepcollectd) class paths.

    Pepperdata does not configure or use the JMSAppender class, but we are removing it altogether to prevent erroneous or malicious activation.

Supervisor v6.5.26 (2022/01/12)

Bug Fixes

  • Fixes an issue with anonymization/encryption that prevented some ResourceManager metrics from being uploaded.

  • Fixes an issue that disabled Pepperdata container instrumentation in the YARN NodeManager after the PepAgent stopped and then subsequently restarted.

Enhancements

  • (Tez applications in Application Spotlight) Adds support for the ENDED Tez application state.

  • Reduces the logging verbosity of the Pepperdata instrumentation code for YARN daemon authorization.

Supervisor v6.5.25 (2021/12/01)

Enhancements

Bug Fix

(Spark 3.x, EMR) Fixes an issue that prevented Spark Executor Metrics from appearing in the Pepperdata dashboard.

Supervisor v6.4.19 (2021/12/01)

Bug Fix

(Spark 3.x, EMR) Fixes an issue that prevented Spark Executor Metrics from appearing in the Pepperdata dashboard.

Supervisor v6.5.24 (2021/11/17)

Enhancement

(Platform Spotlight, HBase 2.x) Adds support for configuring independent HTTP service policies for HBase and YARN by configuring the new pepperdata.agent.hbase.http.policy property.

Although HBase and YARN typically use the same HTTP service policy—and therefore by default, Pepperdata assumes that the service policy for HBase is the same as the service policy for YARN—they can be different. That is, HBase might be configured for HTTPS_ONLY when the HTTP service policy for YARN is HTTP_ONLY, or vice versa (HTTP_ONLY for HBase, but HTTPS_ONLY for YARN).

To override the Pepperdata default HTTP service policy for HBase, see Configure HBase Monitoring (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

Supervisor v6.5.23 (2021/10/18)

Enhancements

  • (Application Spotlight, YARN) Adds Pepperdata recommendations for Streaming Spark applications:

    • Batch Processing is taking too long—Answers the question, “Is the system able to process batches as quickly as they’re generated?” If not (as indicated by a batch processing time longer than the batch interval), we recommend reducing the batch processing time.

    • Batch Queue Delay is too large—Answers the question, “Is the system able to process batches as quickly as they’re generated?” If not (as indicated by the queuing delay data), we recommend reducing the batch processing time.

    • Not enough Parallelism—Answers the question, “Is the streaming job utilizing the hardware resources enough?” If not (as indicated by the number of tasks being less than the number of cores/machine), we recommend reducing the block interval time, which increases the number of tasks for a given batch interval.

  • (Platform Spotlight) Adds a more graceful shutdown for ephemeral clusters in cloud environments. This ensures that there are no data gaps at the end of the life of a node in an ephemeral cluster.

  • (Platform Spotlight) For installations using OS native packages, adds automatic start-up of Pepperdata services after a server (on which Pepperdata is running) restarts.

Supervisor v6.5.22 (2021/10/04)

New Feature

  • (Technical Preview) Adds Platform Spotlight and Application Spotlight support for GPUs in Spark on Kubernetes applications.

    GPU-based additions to the Pepperdata dashboard (home page) are:

    • GPU tile in the Cluster Resources section
    • Top GPU-Allocating Apps and Top GPU-Wasting Apps tiles in the Applications section
    • GPU Power and Temperature section

    Additions to the Application Spotlight pages are:

    • Top GPU-Allocating Apps tile in the Applications Overview
    • GPU data columns in the Applications Overview table
    • App GPU Usage chart in the Resource Usage tab of the App Details page
    • Top-level tab for GPU in the App Details page

    As well, this release adds the following GPU-related recommendations:

    • Unused GPUs
    • Low usage of GPU resources

    For recommendation details—cause and recommendation message strings—see Spark Recommendations.

Enhancements

  • (Application Spotlight) Improves support for long-running Spark apps by reducing Pepperdata memory overhead for reporting data structures.

  • (Spark on Kubernetes) Adds a pepperdata-prometheus Helm chart that bundles all the required dependencies to the Pepperdata tarball, which eliminates the need for a help dependency update.

    For the updated procedures, see Installing or Upgrading Pepperdata (Kubernetes) (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

Supervisor v6.5.21 (2021/09/21)

Enhancements

  • Adds the capability for disabling the Pepperdata Status Views—near-real-time ResourceManager and NodeManager data collected by the PepAgent and Supervisor agents that is by default viewable by all your Pepperdata users (see Pepperdata Status Views via Web Servlets) (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

    To disable the views, see Disable Pepperdata Status Views (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

  • Consolidates the packaging for EMR and Qubole to be based on the distro version (not the particular Pepperdata features you want). See the Downloads page.

Supervisor v6.4.18 (2021/09/10)

Bug Fix

  • (Capacity Optimizer) For specific distros and environments, fixes an issue that can cause Capacity Optimizer to stop functioning after a NodeManager restart when tracking multiple NodeManager instances on the same host. The affected distros are:
    • All Hadoop 3-based distros
    • All Hadoop 2.9-based distros
    • Environments configured for autoscaling optimization, for EMR 5.x and Qubole QDS (Hadoop 2.6–2.8)

Supervisor v6.5.20 (2021/09/08)

Bug Fix

  • (Spark on Kubernetes) Fixes a templating issue that caused spurious entries to be written to an internal configuration file. Although functionality was not impacted, we recommend upgrading to ensure forward-compatibility.

Supervisor v6.5.19 (2021/09/01)

Enhancement

  • (Autoscaling optimization in Capacity Optimizer, for EMR-managed scaling) Adds support for non-uniform loads by considering only hosts from autoscaling groups.

    • To enable Pepperdata to correctly determine if the cluster is running out of resources, and therefore adjust the max capacity of the autoscaling policy, you must add the pepperdata.managedAutoscaling.partialInstanceGroupSupport.enabled property to the Pepperdata configuration (and set its value to true).

    • By default, Pepperdata considers the hosts in the TASK autoscaling group, but you can override it to use the CORE autoscaling group by adding the pepperdata.managedAutoscaling.partialAutoscalingGroup property and setting its value to CORE.

    For details, see Configure Capacity Optimizer: EMR (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

Supervisor v6.5.16 (2021/08/26)

Operational Change

  • (Capacity Optimizer) For cloud (HDaaS) environments, changes the interval for refreshing the list of nodes from two (2) minutes to 30 seconds.

Bug Fix

  • (Capacity Optimizer) For specific distros and environments, fixes an issue that can cause Capacity Optimizer to stop functioning after a NodeManager restart when tracking multiple NodeManager instances on the same host. The affected distros are:
    • All Hadoop 3-based distros
    • All Hadoop 2.9-based distros
    • Environments configured for autoscaling optimization, for EMR 5.x and Qubole QDS (Hadoop 2.6–2.8)

Supervisor v6.5.15 (2021/08/24)

This release provides the following enhancements for Pepperdata in Spark on Kubernetes environments:

  • Adds support for HPE Ezmeral Container Platform 5.2.x to Application Spotlight and Platform Spotlight.

  • Simplifies the configuration and installation process. For the updated procedures, see Installing or Upgrading Pepperdata (Kubernetes) (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

Supervisor v6.5.14 (2021/08/10)

Enhancement

  • (Autoscaling optimization in Capacity Optimizer, for EMR custom automatic scaling policy) Adds support for non-uniform loads by considering only hosts from autoscaling groups. To enable Pepperdata to correctly determine if the cluster is running out of resources, and therefore adjust the max capacity of the autoscaling policy, you must add the pepperdata.managedAutoscaling.partialInstanceGroupSupport.enabled property to the Pepperdata configuration; see Configure Capacity Optimizer: EMR (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

CSD v3.0.3 (2021/07/28)

(HDFS Data Temperature Report) Fixes a dependency bug that prevented HDFS Tiering operations from running on HDFS NameNodes.

Supervisor v6.5.13 (2021/07/28)

Enhancements:

  • (Platform Spotlight) Adds support for overriding the default scheduled read window (00:00–23:00 on Saturdays) and interval (once) of the FsImage files that provide the data for the HDFS Data Temperature Report.

    Although the schedule for attempting to generate the report is unchanged (and depends on how your cluster is configured in the Pepperdata backend), the time to complete the report generation, and whether it succeeds or fails, is influenced by the revised scheduled read window, as well as the size of the given FsImage files.

    For the procedure to override the default scheduled read window and interval, see Configure HDFS Data Temperature Report v6.5 (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

  • (Supervisor) Improves PepAgent logging:

    • Adds canonical host names when available.
    • Removes JSON output dump for GenericJSON fetchers when the logging level is DEBUG.
    • Adds messages for the GenericJSON fetchers to indicate when they’ve been switched on for a given host.

Bug Fixes:

  • (HDFS Data Temperature Report) Fixes an issue that caused incorrect interpretation of storage policy values in FsImage files when the policy’s age value >= 10 (ten).

  • Fixes an issue that prevented the PepAgents from starting when the HDFS Web UI is Kerberized but the Kerberos credentials are not provided because the YARN Web UI is not Kerberized.

Supervisor v6.5.10 (2021/07/16)

Enhancements:

  • (Platform Spotlight) Adds support for running the HDFS Data Temperature Report on a host other than the NameNodes.

  • (Platform Spotlight) For the HDFS Data Temperature Report, increases the default value of the number of inodes to read during every 15-second interval from 10000 (10K) to 100000 (100K).

    This increased value enables Pepperdata to read the inodes more quickly. The trade-off is that reading more inodes during every 15-second interval requires more memory. Therefore, you might need to increase the amount of memory to provide for PepAgent reporting’s processing of the inodes:

    • For Parcel installations, use the Java Heap Size of PepAgent parameter.
    • For RPM/DEB installations, use the PD_MEM_LIMIT environment variable.

    For detailed steps, see Configure HDFS Data Temperature Report v6.5 (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

  • (Capacity Optimizer) Reduces the number of configuration properties required for autoscaling optimization in Capacity Optimizer for EMR and Dataproc.

Bug Fixes:

  • (Query Spotlight) Fixes an issue that prevented showing information on the Databases Overview page if the PepAgent was unable to fetch metadata for any database or table (for example, if a table is misconfigured or there are dependency issues). Now the PepAgent skips such failed queries for metadata, continues as expected, and shows all databases and tables that are successfully identified.

  • (Hadoop 2.6–2.8; Platform Spotlight and Application Spotlight) Fixes an issue that caused inconsistent reporting of Job Current Queue Duration Seconds metric values on different dashboard pages; for example, the Charts page, the Bottlenecks tab of an App Details page, and the Queues Overview page.

Supervisor v6.5.9 (2021/06/30)

Enhancement:

  • (Application Spotlight) Adds collection of HDFS capacity monitoring metrics from NameNodes. To see these metrics in the Pepperdata dashboard, navigate to the NameNode metrics group.

Bug Fix:

  • (Application Spotlight) When a Spark job fails for a reason other than ExceptionFailure, the reason now correctly appears on the Application Spotlight overview and the applicable app’s App Detail Page (and no exceptions are thrown by Pepperdata).

CSD v3.0.2 (2021/06/07)

(Streaming Spotlight) Adds support for SASL_PLAINTEXT (Simple Authentication and Security Layer [SASL] protocol with no SSL encryption) secured Kafka clusters.

Supervisor v6.5.8 (2021/06/17)

This patch (“.dot”) release provides the following enhancements:

  • (Streaming Spotlight) Adds support for SASL_PLAINTEXT (Simple Authentication and Security Layer [SASL] protocol with no SSL encryption) secured Kafka clusters.

  • (Spark on Kubernetes) Improves response time for showing fresh data in the dashboard for Kubernetes clusters. The improvement comes from capping the maximum size of metric files, which reduces Pepperdata backend memory usage.

  • (Application Spotlight) Adds collection of JMX metrics for SSL-secured and Kerberos-secured DataNodes.

CSD v3.0.1 (2021/06/09)

  • Changes the default user for Pepperdata from root to pepperdata. (Only the CSD v1.x series assigns root as the default user for Pepperdata.)

  • Adds Pepperdata CSD parameters for configuring the HDFS Data Temperature Report and Streaming Spotlight; they appear as checkboxes in the Cloudera Manager interface.

    This means that instead of having to add XML code to the Pepperdata configuration, you use the Cloudera Manager UI to directly select and enter the necessary options and details.

Migration CSD v1.0 (2021/06/09)

This Migration CSD contains the PepperdataMigration service, which manages the directory ownership changes that are required for Pepperdata upgrades in which the default user for Pepperdata changes from root to any other user (typically, pepperdata).

If you are upgrading from Supervisor v6.4.x or earlier to Supervisor v6.5.5 or later, you must perform a migration-upgrade, as instructed in the applicable upgrade procedure.

Supervisor v6.5.5 (2021/06/09)

This new minor release—the first Supervisor v6.5.x release—includes GA for Kubernetes support, significant enhancements to existing products, many additions to the documentation, and rolls up the Supervisor v6.4 patches’ (“.dot releases’”) enhancements and bug fixes for v6.4.10–6.4.17.

  • Platform and System Support

    • Adds support for Kubernetes on Amazon Elastic Kubernetes (Amazon EKS). For installation procedures, see Installing or Upgrading Pepperdata (Kubernetes) (or corresponding page for Pepperdata Supervisor versions newer than v6.5).

    • Drops support for Ubuntu 16.04 LTS.

  • Capacity Optimizer

    Adds support for the custom automatic scaling policy in EMR.

  • Parcel Installations of Pepperdata

    • Changes the default user for Pepperdata from root to pepperdata.

    • Decouples (separates) the release packages for the Pepperdata Supervisor and the Parcel CSDs.

      • When you perform a Parcel installation or upgrade of Pepperdata, you’ll need to download the CSD file separately from the Supervisor Parcel. Details are given in the applicable procedures.

      • In general, Supervisor parcel and Supervisor CSDs can be independently upgraded. Supervisor parcel v6.2.x and newer can operate with Supervisor CSD v1.4 (default user root)/CSD v2.1 (non-root default user pepperdata) and later.

      • For reference, the table identifies the CSD versions that were packaged with earlier Supervisor versions (before the decoupling of the Pepperdata Supervisor and the Supervisor Parcel CSD). The CSD v1.x series assigns root as the default user for Pepperdata. The CSD v2.x series assigns pepperdata (non-root) as the default user for Pepperdata.

        Supervisor Version Packaged CSD Version
        6.4.11–6.4.19 1.5.0 and 2.2
        6.4.6–6.4.10 1.4 and 2.1
        6.3.21 1.5.0 and 2.2
        6.3.10–6.3.20 1.4 and 2.1
        6.2.20–6.2.38 1.2
  • Application Spotlight

    Adds collection/display of JVM Native Memory metrics: total reserved, total committed, heap reserved, heap committed, and so on.

    • The -XX:NativeMemoryTracking=summary or -XX:NativeMemoryTracking=details flag must be set for the JVM before the application runs.
    • You must be using Java 8 or later.
    • This feature is available only for YARN clusters.
    • Collection of the metrics is not retroactive. That is, the metrics appear only for jobs that run after the NativeMemoryTracking flag is set for a JVM.
  • Platform Spotlight

    • Adds collection of instance type for cloud (HDaaS) clusters—EMR, Dataproc, and Qubole.

    • Fixes an issue on Java 11 environments where the Task JVM NIO metrics were missing. The following metrics now correctly appear on all supported environments: Task JVM NIO memory capacity, Task JVM NIO reservations, Task JVM NIO memory used.

  • Platform Spotlight and Application Spotlight

    For specific distros and environments, fixes an issue that caused inconsistent reporting of Job Current Queue Duration Seconds metric values on different dashboard pages; for example, the Charts page, the Bottlenecks tab of an App Details page, and the Queues Overview page. The affected distros are:

    • All Hadoop 3-based distros
    • All Hadoop 2.9-based distros
    • Environments configured for autoscaling optimization, for EMR 5.x and Qubole QDS (Hadoop 2.6–2.8)
  • Documentation

  • Roll Up of v6.4.x Enhancements and Bug Fixes

    This release rolls up all the enhancements and bug fixes from the v6.4.x patches (“dot releases”) that were completed before the v6.5.5 release. For details, see the entries on this page for Supervisor releases v6.4.10–v6.4.17.

Supervisor v6.4.17 (2021/05/10)

Enhancements:

  • (Streaming Spotlight) Adds support for Kafka 2.5.x. For more information about Kafka support, see Stream Processors in the System Requirements.

  • (Streaming Spotlight) Adds SSL client authentication support for Kafka Admin monitoring, which is performed by the host that you choose and configure as your-kafka-admin-host when you enable Kafka monitoring (see Configure Streaming Spotlight (or corresponding page for Pepperdata Supervisor versions newer than v6.4)).

  • (Streaming Spotlight) Enhances security by using encryption for all Kafka-related passwords that are specified in the Pepperdata configuration (see Configure Streaming Spotlight (or corresponding page for Pepperdata Supervisor versions newer than v6.4)).

  • (Platform Spotlight, HBase 2.x) Pepperdata now supports independent authentication settings (Kerberos or not) for HBase and the remaining services in the cluster. This means that a cluster can be Kerberized for other services, but not for the HBase Web UI, or vice versa (Kerberized HBase Web UI but nothing else Kerberized in the cluster).

    By default, Pepperdata assumes that the HBase Web UI uses the same authentication as the rest of the cluster’s services (Kerberized or not). If the HBase Web UI uses different authentication, you need to configure an override for HBase.

    To override the Pepperdata default authentication for HBase, see Configure HBase Monitoring (or corresponding page for Pepperdata Supervisor versions newer than v6.4).

Supervisor v6.4.16 (2021/05/03)

Bug Fixes:

  • (Capacity Optimizer, all versions of EMR/Dataproc) Fixes an issue that caused the ResourceManager to report an incorrectly high value for total memory, which could negatively affect autoscaling functionality.

    This issue is present in non-EMR/non-Dataproc deployments, but the only effect is that an incorrect value appears in the ResourceManager UI for Total Memory. There is no effect on functionality. If you want, you can rectify the erroneous display value when you configure Capacity Optimizer or configure autoscaling optimization.
  • (EMR 6.1 and later) Fixes an issue that prevented the JobHistory Monitor from starting, which in turn caused PepAgent to repeatedly restart.

Supervisor v6.4.15 (2021/04/30)

Enhancement:

  • (Platform Spotlight) Adds collection of HBase 2.0 metrics, and Kerberos support for HBase 2.0.

Bug Fixes:

  • (Hadoop 2.6–2.8) Fixed an issue that caused missing queue metrics from children of non-root queues.

  • For specific distros and environments, fixes an issue that prevented the collection of the reservedContainers and YARN container priority (t_ypri in the dashboard) metrics. The affected distros are:

    • All Hadoop 3-based distros
    • All Hadoop 2.9-based distros
    • Environments configured for autoscaling optimization, for EMR 5.x and Qubole QDS (Hadoop 2.6–2.8)

Supervisor v6.4.13 (2021/03/22)

Enhancement:

  • Adds an EMR CloudFormation template for installations via the Amazon AWS Marketplace.

Supervisor v6.4.12 (2021/03/08)

Bug Fixes:

  • (Hadoop 2.9 and later; all versions of EMR, Dataproc, and Qubole) Capacity optimizer now protects (does not preempt) the first five (5) containers of an application, regardless of the distro.

  • (autoscaling optimization) Corrects the calculations that determine how many instances to add. Before this fix, autoscaling optimization was operating too conservatively at times, and not adding instances even when resources were available.

Supervisor v6.4.11 (2021/02/12)

Bug Fix:

  • Fixes an issue where the file names of rolled-over Pepperdata daemon logs included the rollover timestamp in UTC, although the actual log messages in the files were timestamped in the local time. With this fix, the timestamps in the file names are now in local time so that they match the timestamps in the messages themselves.

Enhancements:

  • For Streaming Spotlight, adds password authentication support for Kafka server JMX connections. For details, see Configure Streaming Spotlight (or corresponding page for Pepperdata Supervisor versions newer than v6.4).

  • For Application Spotlight, adds SQL support for Spark 2.0.

    When you display the App Details page for a Spark 2.0 job that includes SQL statements, you can see the first 100 SQL statements/queries that are encountered in the code, as well as the Spark-generated query plans for those statements/queries.

    Screenshot of Application Details with the Spark SQL tab

    Screenshot of Application Details with the Spark Query Plan

Supervisor v6.4.10 (2021/01/21)

Bug Fix:

  • Fixes an issue with the HDFS Data Temperature Report that caused the HDFS fetcher in the PepAgent to crash if the FsImage file contained details about a directory that contained no files, but that still contained inode references to previously-deleted files.

Enhancements:

  • Reduces the verbosity of error logs:
    • Lowers the logging level for client socket errors in YARN containers.
    • Removes a spurious WARN message from the output of SparkSQL query sessions, WARN spark.SparkSessionEnhancer: Queue is full to its capacity of 150. Unable to add more sparkSqlDetails.
    • Removes unnecessary stack traces from info-level log entries.
  • Changes the capitalization of the name of the Capacity Optimizer property for the CPU-specific maximum for YARN containers so that it is consistent with the names of the other resource-specific maximums.

    If you’ve configured this advanced configuration property, you must change it from pepperdata.supervisor.allocation.maxCpu.Percent to pepperdata.supervisor.allocation.maxCpu.percent. For more information, see Advanced Configuration: Resource-Specific Maximums for YARN Containers (or corresponding page for Pepperdata Supervisor versions newer than v6.4).

Supervisor v6.4.6 (2021/01/04)

This release includes significant enhancements to existing products, and rolls up the Supervisor v6.3 patches’ (“dot-releases”) enhancements and bug fixes (for v6.3.10–v6.3.20).

  • Capacity Optimizer

    • Adds support for autoscaling optimization for Amazon EMR-managed clusters.

      For detailed configuration steps, see Configure Autoscaling Optimization: EMR (or corresponding page for Pepperdata Supervisor versions newer than v6.4).

    • For autoscaling optimization, for Hadoop as a Service (Cloud) installations on Google Dataproc and Qubole QDS, greatly simplifies the configuration process:

      • Removes many of the Pepperdata properties.

      • Eliminates the need to manually configure most of the values in the Dataproc and Qubole autoscaling policies, and to exactly replicate them in the Pepperdata configuration.

  • Streaming Spotlight

    • Adds collection of Kafka JMX and broker young garbage collection (GC) metrics. In the metrics picker, look for the Kafka memory metrics group and Kafka young GC metrics group, respectively.Screenshot of Kafka memory metrics group in the metrics pickerScreenshot of Kafka young GC group in the metrics picker

    • Adds collection of the metric for internal topic consumer_offset, which fixes a discrepancy between the total partition count that is reported and the total obtained by manually summing the partition counts for all the individual topics.

    • (Parcel installations) Adds support and easy configuration for Streaming Spotlight on clusters without Hadoop, such as Kafka-only clusters, via a single Pepperdata CSD parameter, Run Pepperdata in Non-Hadoop Environment, in an updated Pepperdata custom service descriptor (CSD): for the recommended non-root user installation, use PEPPERDATA-2.1.jar; for root user installations, use PEPPERDATA-1.4.jar. For detailed configuration steps, see Configure Streaming Spotlight (Parcel) (or corresponding page for Pepperdata Supervisor versions newer than v6.4).

  • Query Spotlight

    • Removes many of the manual steps from the Configure Query Spotlight procedure, and removes the requirement for restarting the PepMetrics agent.

    • Adds Kerberos support.

    For detailed configuration steps, see Configure Query Spotlight (or corresponding page for Pepperdata Supervisor versions newer than v6.4).

  • Application Spotlight

    Enhances anonymization/encryption.

    In addition to the data that was already anonymized when you enable encryption, this release adds anonymization for sensitive data that is collected from the application’s history server:

    • MapReduce Job History Server for MapReduce apps
    • Spark History Server for Spark apps
    • YARN Timeline Server for Tez apps

    Newly-anonymized data includes the configuration properties of the job, such as host addresses, user names, and the like; and all sensitive data in Spark event logs, such as all the nodes of a JSON object tree (or even all nodes except a given one, via blocklisting).

    Data that is not sensitive, such as JAR file names that are collected as part of the job configuration, remains unencrypted.

    For detailed configuration steps, see Configure Encryption.

  • Roll Up of v6.3.x Enhancements and Bug Fixes

    This release rolls up all the enhancements and bug fixes from the v6.3.x patches (“dot releases”) that were completed before the v6.4.6 release. For details, see the entries on this page for Supervisor releases v6.3.10–v6.3.20.

Supervisor v6.3.21 (2021/04/14)

Bug Fix:

  • (Hadoop 2.6–2.8) Fixed an issue that caused missing queue metrics from children of non-root queues.

Supervisor v6.3.20 (2020/12/14)

Enhancements:

  • For HDFS Data Temperature reports, adds support for a low value of the dfs.namenode.num.checkpoints.retained property—the number of FsImage files that will be retained by the NameNode and Secondary NameNode in their storage directories.

    This ensures that data is successfully obtained from FsImage files that would normally be deleted while Pepperdata is still reading from them.

  • For HDFS Data Temperature reports on Kerberized clusters, adds support for non-renewable Kerberos principals.

Supervisor v6.3.19 (2020/12/04)

Bug Fix:

  • (EMR-based installations) Fixes an issue that prevented the cluster bootstrap script from correctly assigning the cluster (realm) name—as specified by the -c or --cluster argument—to new ephemeral clusters.

Supervisor v6.3.18 (2020/11/23)

Bug Fix:

  • (Capacity Optimizer) Fixes an edge case race condition that occasionally prevented new NodeManager instances from registering with the ResourceManager.

Supervisor v6.3.17 (2020/11/13)

Enhancements:

  • For ephemeral clusters in cloud environments, adds capability to filter the Overview pages, charts, and tables in the Pepperdata dashboard by Ephemeral Cluster ID and/or Ephemeral Cluster Name.

    As with other filters, you can use a comma-separated list, a regular expression (regex), or Pepperdata advanced breakdown filters; see Filtering Mechanisms and Regular Expressions (Regex).

    Likewise, you can breakdown tables and charts by Ephemeral Cluster ID and/or Ephemeral Cluster Name.

  • Provides support for using a single license for all of of your Pepperdata-enabled clusters.

  • (Parcel installations for Cloudera Manager) Updates text in the CSD to more clearly identify that it’s Capacity Optimizer enabled (not Dynamic capacity optimizer enabled) for the pepperdata.supervisor.control.enabled value.

Supervisor v6.3.16 (2020/10/21)

Enhancements:

Supervisor v6.3.15 (2020/10/09)

Enhancement:

  • (autoscaling optimization for cloud clusters) Adds consideration of a cluster’s configured max instances value when determining whether it’s appropriate to trigger autoscaling. This avoids over-allocating a cluster when the cluster is configured at its max instance level.

Bug Fix:

  • In Streaming Spotlight, the message count for topics now correctly shows the actual count instead of using calculations based on the message rate (which can be incorrect due to sampling rate and rounding errors).

    The affected data appears in the:

    • Messages In column of the Topics Overview table.
    • Total Messages value in the Summary section of the Topic Details page.

Supervisor v6.3.14 (2020/10/05)

Bug Fix:

Supervisor v6.3.13 (2020/09/28)

Enhancements:

  • Adds support for autoscaling optimization for Qubole-managed clusters.

  • (YARN Adds support for retrieving metrics from Resource Managers where the yarn.resourcemanager.hostname is defined but the yarn.resourcemanager.address is not.

Bug Fix:

  • (YARN 3, YARN 2.9) For PepAgents on Kerberized NodeManager hosts, fixes a bug that caused the PepAgent to require the presence of a valid keytab file when fetching ResourceManager scheduler pages, which it shouldn’t.

    Although PepAgents issue HTTP requests to ResourceManager hosts (and therefore require the keytab file), PepAgents do not issue HTTP requests to NodeManager hosts, and so the keytab file should not be required.

Supervisor v6.3.12 (2020/09/23)

Enhancement:

  • Adds support for monitoring container-launched Spark jobs where the Spark driver and the PepAgent are on different hosts. See Advanced Spark Job Monitoring (or corresponding page for Pepperdata Supervisor versions newer than v6.3).

Bug Fixes:

  • Eliminated potentially conflicting dependencies to prevent application failures where version incompatibilities caused Java “class definition not found” errors.

  • Fixed Query Spotlight support for all combinations of query engines to prevent crashes of HiveServer2 in environments without Tez.

    We recommend upgrading all Query Spotlight v6.3 deployments to this release.

Supervisor v6.3.10 (2020/09/11)

This release includes significant enhancements to existing products, and rolls up the Supervisor v6.2 “dot-release” enhancements and bug fixes.

Enhancements:

  • Adds Capacity Optimizer support for autoscaling optimization in Google Dataproc environments.

    For details about the system requirements and configuration procedures, see Configure Autoscaling Optimization.

  • Adds support for Spark 3 to Application Spotlight. For the complete list of supported tech stack components—JDK, Hadoop as a Service (HDaaS) clouds, stream processors (for Streaming Spotlight), HBase, and so on—see System Requirements.

  • Kafka consumer group metrics and support for Simple Authentication and Security Layer (SASL)-secured Kafka clusters are added to Streaming Spotlight.

    To enable and configure Streaming Spotlight in Pepperdata v6.3, see Configure Streaming Spotlight (or corresponding page for Pepperdata Supervisor versions newer than v6.3).

  • For Parcel installations of Pepperdata (CSD 1.4/CSD 2.1), removes the YARN dependency.

  • For Parcel installations of Pepperdata, you can now choose whether to create the default user (PD_USER) as a non-root user (CSD 2.1) or a root user with root-level privileges (CSD 1.4).

    • We strongly recommend the non-root user option (CSD 2.1), which complies with good security practices of running everything under the principle of least privilege (PoLP).

    • If you choose the root user option, you should be aware that running applications as a root user is a violation of typical security policies in enterprise-level organizations.

    If you are upgrading from an earlier version of Pepperdata, which created the PD_USER as a root user, be sure to carefully follow the migration steps given in the upgrade procedure; see the Upgrade Guide (Parcel).

  • Adds the HDFS Data Temperature report.

    The HDFS Data Temperature report shows the data temperature across a cluster’s entire Hadoop Distributed File System (HDFS) space—how many hot, warm, and cold files are on the HDFS file system, how many files (and the total size of all such files) haven’t been accessed in more than 90 days—which classifies them as cold files—and what the configured storage policies are for those cold files. Using this data, you can optimize your storage use by changing the storage policies for cold files to cold. Before you can access the HDFS Data Temperature report from the Pepperdata dashboard, you must configure your Pepperdata installation to collect the necessary metrics and to upload them to the Pepperdata backend data storage. See Configure HDFS Data Temperature Report v6.3 (or corresponding page for Pepperdata Supervisor versions newer than v6.3).

  • Significantly reduces PepAgent memory usage by more quickly clearing YARN container report heap objects.

  • New Pepperdata release packages are added to support Java 11 in YARN 2 clusters.

    This addresses the edge case of running a Spark 3 job, which requires Java 11, on a YARN 2 (Hadoop 2) cluster. To obtain the new packages, see the Downloads page.

Roll Up of v6.2.x Enhancements and Bug Fixes

This release rolls up all the enhancements and bug fixes from the v6.2.x “dot releases”. For details, see the entries on this page for Supervisor releases v6.2.20–v6.2.32.

Supervisor v6.2.38 (2020/10/08)

Bug Fix:

  • In Streaming Spotlight, the message count for topics now correctly shows the actual count instead of using calculations based on the message rate (which can be incorrect due to sampling rate and rounding errors).

    The affected data appears in the:

    • Messages In column of the Topics Overview table.
    • Total Messages value in the Summary section of the Topic Details page.

Supervisor v6.2.37 (2020/10/05)

Bug Fix:

Supervisor v6.2.36 (2020/09/28)

Enhancement:

  • (YARN Adds support for retrieving metrics from Resource Managers where the yarn.resourcemanager.hostname is defined but the yarn.resourcemanager.address is not.

Bug Fix:

  • (YARN 3, YARN 2.9) For PepAgents on Kerberized NodeManager hosts, fixes a bug that caused the PepAgent to require the presence of a valid keytab file when fetching ResourceManager scheduler pages, which it shouldn’t.

    Although PepAgents issue HTTP requests to ResourceManager hosts (and therefore require the keytab file), PepAgents do not issue HTTP requests to NodeManager hosts, and so the keytab file should not be required.

Supervisor v6.2.35 (2020/09/15)

Bug Fixes:

  • Fixed Query Spotlight support for all combinations of query engines to prevent crashes of HiveServer2 in environments without Tez.

    We recommend upgrading all Query Spotlight v6.2 deployments to this release.

  • Eliminated potentially conflicting dependencies to prevent application failures where version incompatibilities caused Java “class definition not found” errors.

Supervisor v6.2.34 (2020/09/14)

Enhancement:

  • Adds support for monitoring container-launched Spark jobs where the Spark driver and the PepAgent are on different hosts. To enable such monitoring for a container-launched Spark job, include the following Pepperdata configuration override in the launch command: --conf spark.force.data.toRemoteHost=true.

Bug Fixes:

  • Reduces the logging frequency for transient client socket issues in YARN containers.
  • Disables the PepAgent startup option ExitOnOutOfMemoryError by default.
  • Drops support for the legacy Classic Hadoop configuration option PD_HADOOP_FRAMEWORK_NAME.

Supervisor v6.2.32 (2020/08/28)

Enhancement:

  • (Query Spotlight) Revises the caching of Hive-on-Tez queries to better manage stale status from incomplete queries and avoid Hive performance degradation.

Supervisor v6.2.30 (2020/08/26)

Enhancements:

  • (YARN 3) Adds support for more than two ResourceManager HA hosts in the auxiliary pre-check utility, YarnHttpAccessTester. (This was already supported for YARN 2.)

  • Reduces PepAgent memory use by removing extraneous debug information.

Bug Fixes:

  • Corrects the logging severity from error to debug for transient client socket errors in YARN containers.

  • Recognizes that the Hadoop proxy user HTTP filter supports Kerberos authentication.

Supervisor v6.2.29 (2020/08/11)

Enhancement:

  • Adds support for running a different version of Hadoop filesystem client code from the cluster’s Hadoop version. For example, you can run Hadoop 3.x filesystem client code on Hadoop 2.x clusters. Likewise, you can run Hadoop 2.x filesystem client code on Hadoop 3.x clusters.

Bug Fix:

  • (MapReduce and Tez) Changes the registration mechanism for the Pepperdata URL handler to use a Java system property. This change eliminates the potential failure of YARN containers, which sometimes happened when they registered their own URL handler factory.

Supervisor v6.2.27 (2020/08/04)

Enhancement:

  • (Hadoop 2.8 and earlier/YARN 2) Adds support to Capacity Optimizer for honoring YARN queue preemption settings.

    • By default, Capacity Optimizer won’t preempt containers from non-preemptable queues. You can enable Capacity Optimizer to preempt such containers by setting the pepperdata.supervisor.allocation.shouldProtectNonPreemptableQueues property’s value to false.

    • By default, Capacity Optimizer won’t preempt the first five (5) containers of an application. You can change the number of containers that won’t be preempted to any value from 3–9 inclusive by setting the pepperdata.supervisor.allocation.numFirstContainersToAvoidKilling property’s value to a number from 3–9.

    • Capacity Optimizer supports queue-level preemption for distros based on Hadoop versions 2.8 and earlier.

      Additionally, support requires that the distro itself supports queue-level preemption for the scheduler in use—Fair or Capacity.

      • Typically, Capacity scheduler queue-level preemption is supported by distros based on Hadoop 2.7 and later.

      • Fair scheduler queue-level preemption is supported by distros based on Hadoop 2.9 and later, which generally means that it is not supported by Capacity Optimizer (because as stated earlier, Capacity Optimizer supports queue-level preemption for Hadoop 2.8 and earlier distros).

      • However, some vendors have backported queue-level preemption for the Fair scheduler to their distro versions that are based on Hadoop 2.8 and earlier; for example, Cloudera’s Distribution of Hadoop (CDH) 5.7.x and later. In such cases, Capacity Optimizer also supports queue-level preemption for the Fair scheduler.

      In summary, if a distro based on Hadoop v2.8 or earlier supports queue-level preemption for the scheduler in use, Capacity Optimizer also supports it.

    For details about configuring these properties, see Configure Capacity Optimizer v6.2 (or corresponding page for Pepperdata Supervisor versions newer than v6.2).

Supervisor v6.2.25 (2020/07/10)

Enhancement:

  • Adds error handling ot the HTTPS uploader component of the Pepperdata Collector (the pepcollected daemon) to support the following new Spark memory recommendations:

    • Excessive executor memory wasted
    • Excessive driver memory wasted

Supervisor v6.2.24 (2020/06/12)

Bug Fix:

  • (YARN 3, YARN 2.9) For PepAgents on Kerberized NodeManager hosts, fixes a bug that caused the PepAgent to require the presence of a valid keytab file when it shouldn’t.

    Although PepAgents issue HTTP requests to ResourceManager hosts (and therefore require the keytab file), PepAgents do not issue HTTP requests to NodeManager hosts, and so the keytab file should not be required.

Supervisor v6.2.22 (2020/05/20)

Enhancement:

  • Adds support for JKS (Java KeyStore) certificates for securing the ports that Pepperdata uses for listening (port 50510 for the Supervisor, and port 50505 for PepAgents).

    If the certificate name extension is .jks, Pepperdata assumes that it is a JKS certificate. For any other extension, Pepperdata assumes that it is a PKCS #12 certificate.

    For instructions, see Configure SSL Near Real-Time Monitoring on Ports 50510 and 50505 (or corresponding page for Pepperdata Supervisor versions newer than v6.2).

Supervisor v6.2.21 (2020/05/19)

Enhancements:

  • Adds support for monitoring apps that change queues after they’ve launched.

  • Adds a startup check to verify that the version of YARN on the cluster (YARN 2 or YARN 3) matches the version of YARN that the Pepperdata installation package is for. This ensures that the correct Pepperdata package has been installed.

Bug Fix:

  • For Cloudera Manager Parcels installations, fixes a timeout bug that caused the curl command to remain active (on port 50510) after the Pepperdata services were stopped. This caused the PepAgent to fail because it could not bind to the still-locked port, which it uses as the listener port on the ResourceManager host.

Supervisor v6.2.20 (2020/05/12)

This release has new products, enhancements to existing products, and rolls up the Supervisor v6.1 “dot-release” enhancements and bug fixes.

New products/GA (general availability) is announced for:

  • Streaming Spotlight—Get detailed, near near real-time visibility into Kafka cluster metrics, broker health, topics, and partitions. See:

  • Query Spotlight—Tune, debug, and optimize query workloads for better performance and reduced costs. See:

  • Pepperdata in Hadoop-as-a-Service (cloud) environments—Run more apps, track your spend, and manage costs with Pepperdata, just like you do with your on-premises big data analytics stack. See:

Enhancements:

  • For Impala query monitoring:
    • Adds support for collecting Impala query metrics in Hadoop 3-based distros.
    • Now supports digest auth for the impalad daemon for the Impala Web UI for debugging; see Adding Apache® Impala Query Metrics (or corresponding page for Pepperdata Supervisor versions newer than v6.2).
  • Adds support for Java 11—the current long-term support (LTS) version of Java.

  • Adds support to Query Spotlight for:
    • Encrypting query text in the dashboard displays.
    • Using the hive sentry subject name (hive.sentry.subject.name) as the application owner.
  • Improved performance by reducing the network traffic among PepAgents (by obtaining information that’s available from the configuration instead of querying other PepAgents).

  • Improved management of the Pepperdata configuration. To pick up these changes during your upgrade, you’ll need to merge changes from the configuration template file to your configuration file. Be sure to see the detailed steps in whichever upgrade procedure you perform: Quick Start for Upgrading Pepperdata, Upgrade Pepperdata, or Upgrade Hadoop Distribution and Pepperdata (or corresponding pages for Pepperdata Supervisor versions newer than v6.2).

Roll Up of v6.1.x Enhancements and Bug Fixes

This release rolls up all the enhancements and bug fixes from the v6.1.x “dot releases”. For details, see the entries on this page for Supervisor releases v6.1.0–v6.1.20.

Supervisor v6.1.22 (2020/05/20)

Enhancement:

  • Adds support for JKS (Java KeyStore) certificates for securing the ports that Pepperdata uses for listening (port 50510 for the Supervisor, and port 50505 for PepAgents).

    If the certificate name extension is .jks, Pepperdata assumes that it is a JKS certificate. For any other extension, Pepperdata assumes that it is a PKCS #12 certificate.

    For instructions, see Configure SSL Near Real-Time Monitoring on Ports 50510 and 50505 for your Supervisor version.

Supervisor v6.1.21 (2020/05/19)

Enhancements:

  • Improved performance by reducing the network traffic among PepAgents (by obtaining information that’s available from the configuration instead of querying other PepAgents).

  • Adds a startup check to verify that the version of YARN on the cluster (YARN 2 or YARN 3) matches the version of YARN that the Pepperdata installation package is for. This ensures that the correct Pepperdata package has been installed.

  • Adds support for collecting Impala query metrics in YARN 3-based environments.

Bug Fixes:

  • For Cloudera Manager Parcels installations, fixes a timeout bug that caused the curl command to remain active (on port 50510) after the Pepperdata services were stopped. This caused the PepAgent to fail because it could not bind to the still-locked port, which it uses as the listener port on the ResourceManager host.

  • For Capacity Optimizer running in a Cloudera Manager Hadoop 3 environment, fixes a bug that caused Cloudera Manager to not recognize a configuration change made via the Pepperdata site file, pepperdata-site.xml, when the PapAgent restarted.

  • Fixes a log redirect from YARN 3 daemons to retry if it fails due to a missing base log directory (/var/log/pepperdata), which happens unless the PepAgent starts first.

Supervisor v6.1.20 (2020/04/13)

Enhancement:

  • Adds a configurable parameter, PD_JOBHISTORY_SPARK_CONNECTION_TIMEOUT_SEC, for the connection timeout to use when making a REST request to fetch Spark app data from the Spark History Server. This parameter’s default value corresponds to the original behavior: five (5) seconds. For information about changing this value, see Configure Connection Timeout for Spark History Server for your Supervisor version.

Supervisor v6.1.19 (2020/04/02)

Enhancement:

  • Adds a second phase for fetching history data for applications if there is no data available during the original first phase, which by default includes up to three retries after an initial failure. The second phase adds an additional try and by default up to five retries, with the interval between retries increased by a factor of five every time.

    For environments with significant network latency or frequent connectivity issues, this enhancement ensures that Pepperdata can successfully fetch the history from the applicable component: the MapReduce Job History Server (for MapReduce apps), the Spark History Server (for Spark apps), or the YARN Timeline Server (for Tez apps).

    You can customize the number of tries for each phase; see Configure History Fetcher Retries for your Supervisor version.

Supervisor v6.1.18 (2020/03/26)

Enhancement:

  • (Technical Preview) Adds support to Application Profiler for Tez applications in Hortonworks Data Platform (HDP®) 3.x; see Task 5: (Tez Applications on HDP® 3.x) Configure Tez to use the YARN Timeline Server for your Supervisor version.

Supervisor v6.1.17 (2020/03/18)

Bug Fixes:

  • The encrypt_password.sh utility script now works for all supported Java versions.

  • Roll up of bug fixes from releases v6.1.3–v6.1.16 (inclusive):

    • (v6.1.15) For environments with significant network latency or frequent connectivity issues, fixes a timeout issue that caused applications to hang (instead of successfully terminating) when they were stopped.

    • (v6.1.14) Fixed an issue that caused missing Spark driver information when the launch command wasn’t in the final 10 lines of the container launch script.

    • (v6.1.9) Fixed an issue that caused missing Spark driver information when newline characters were included in a multi-line container launch command.

    • (v6.1.8) Fixed an issue that, for Supervisor v6.1.x (not v6.0.x), caused HiveServer2 service errors due to irrelevant JARs in the Pepperdata Supervisor release package.

    • (v6.1.4) For deployments on OSes running Linux kernel v4.19 or later, fixed an issue that prevented Pepperdata from collecting disk I/O stats.

Enhancements—Roll up of enhancements from releases v6.1.3–v6.1.16 (inclusive):

  • (v6.1.16) Added the encrypt_password.sh utility script for encrypting the password used to secure (via SSL) the ports that Pepperdata uses for listening (port 50510 for the Supervisor, and port 50505 for PepAgents). When you run the script to encrypt your password, the script outputs the encrypted password to the command line. You can then copy it and paste it into the Pepperdata configuration as described in Configure SSL Near Real-Time Monitoring on Ports 50510 and 50505.

  • (v6.1.15) Masks passwords and other sensitive information in Pepperdata logs.

  • (v6.1.12) Surfaces stack trace data for non-zero exit status from all containers (not just Spark executors).

    To see the stack trace data, navigate to the App Details for the associated application, click the Errors tab, and click Stack Trace.Screenshot of Errors tab for non-zero exit status from containers

  • (v6.1.10) Adds support to the JobHistory Monitor for Spark History Server 302 redirects.

  • (v6.1.6) Adds support to Pepperdata Capacity Optimizer for cluster hosts that are decommissioned during cloud autoscaling.

  • (v6.1.5) Adds support for Hadoop 2.9 on EMR.

  • (v6.1.3) Adds support for Hadoop 2.9.

  • (v6.1.3) Adds SSL support for Pepperdata near real-time monitoring views via Web servlets.

    For details, see Monitoring Pepperdata for your Supervisor version. To enable SSL support, contact Pepperdata Support.

Supervisor v6.1.1 (2019/10/23)

Bug Fix:

  • (Hadoop 3.x) Many more ResourceManager metrics are now collected and available in the dashboard. (Some metrics were missing from the earlier releases with Hadoop 3 support.)

Supervisor v6.1.0 (2019/10/04)

Enhancement:

  • (Technical Preview) Adds Hadoop 3.x support for Pepperdata Capacity Optimizer.

Bug Fixes:

  • (Hadoop 3.x) Authentication now works when fetching data from ResourceManager and NodeManager REST endpoints.

  • Rolls up the Supervisor v6.0 “dot-release” bug fixes:

    • (Non-standard Spark environments in which unreleased features from the Spark master branch have been backported to a Spark version that is otherwise supported by Pepperdata) If the Pepperdata Code Analyzer for Apache Spark crashes due to a non-standard Spark environment, your Spark apps now remain unaffected (instead of also crashing), and error messages are printed to the log files.

    • (Non-standard Spark environments in which unreleased features from the Spark master branch have been backported to a Spark version that is otherwise supported by Pepperdata) You can now successfully disable the Pepperdata Code Analyzer for Apache Spark via the Spark configuration file.

    • If the Pepperdata Code Analyzer for Apache Spark crashes, your Spark apps now remain unaffected (instead of also crashing), and error messages are printed to the log files.

    • Capacity Optimizer now runs when the ResourceManager uses the IP address (instead of the hostname) of the NodeManager.

    • For Qubole-managed clusters, fixed an issue that prevented Pepperdata from collecting job history data.

    • For Pepperdata deployments that are installed into non-standard home directories or with unexpected permissions, fixed an issue that prevented Pepperdata from starting due to missing write permissions for some directories.

    • For Spark versions later than 2.4.x, fixed an issue that prevented the collection of Spark stage metrics.

Supervisor v6.0.13 (2019/09/19)

Bug Fixes:

  • (Non-standard Spark environments in which unreleased features from the Spark master branch have been backported to a Spark version that is otherwise supported by Pepperdata) If the Pepperdata Code Analyzer for Apache Spark crashes due to a non-standard Spark environment, your Spark apps now remain unaffected (instead of also crashing), and error messages are printed to the log files.

  • (Non-standard Spark environments in which unreleased features from the Spark master branch have been backported to a Spark version that is otherwise supported by Pepperdata) You can now successfully disable the Pepperdata Code Analyzer for Apache Spark via the Spark configuration file.

Enhancement:

  • (Pepperdata Use Only) Adds additional data collection for information about failed tasks in Spark apps, which we’re using to improve the Spark heuristics tuning suggestions and guidance. After we obtain sufficient data from enough users, we can improve the heuristics evaluations and recommendations. Therefore, we’re asking you to upgrade to Pepperdata v6.0.13 even if you were unaffected by the bugs that were fixed.

Supervisor v6.0.11 (2019/09/13)

Bug Fix:

  • If the Pepperdata Code Analyzer for Apache Spark crashes, your Spark apps now remain unaffected (instead of also crashing), and error messages are printed to the log files.

Supervisor v6.0.10 (2019/08/30)

Bug Fix:

  • Capacity Optimizer now runs when the ResourceManager uses the IP address (instead of the hostname) of the NodeManager.

Supervisor v6.0.9 (2019/07/26)

Bug Fixes:

  • For Qubole-managed clusters, fixed an issue that prevented Pepperdata from collecting job history data.

  • For Pepperdata deployments that are installed into non-standard home directories or with unexpected permissions, fixed an issue that prevented Pepperdata from starting due to missing write permissions for some directories.

  • For Spark versions later than 2.4.x, fixed an issue that prevented the collection of Spark stage metrics.

Supervisor v6.0.8 (2019/07/09)

Enhancements:

  • Adds support for Hadoop 3.x and distros built on Hadoop 3.x.

    Although coming soon, Pepperdata Capacity Optimizer is not yet supported for Hadoop 3.x-based distributions.

  • Introduces Universal Packages. Instead of separate packages based on vendor distros and versions, one set of packages (RPM, DEB, and Parcels) supports any vendor’s distro that’s built on Hadoop 2.6–2.8, while a second set of packages supports vendor distros built on Hadoop 3.0.x and later.

  • Increases the heap size for PepAgent from 64 MB to 128 MB because Hadoop 3 support requires a larger heap. This increase is small compared to the typical amount of memory available to hosts, but you can overwrite this value if needed. (If you need assistance, contact Pepperdata Support.)

  • Adds sensible configuration settings for Java 8 memory options for PepAgent. These settings are similar to those used for Java 6 and Java 7, and provide a better operating environment than the system defaults for Java 8 memory options.

  • Adds monitoring, by default, of additional Hadoop daemons: ResourceManager, NodeManager, NameNode, and DataNode. Correspondingly, the monitored domain breakdown now includes the yarn-daemon. For the program matching rules file, /opt/pepperdata/supervisor/lib/pepagent-program-monitor-config-default.yaml, see Preconfigured Custom Program Monitoring for your Supervisor version.

  • No longer creates empty metric files from the generic JSON fetch module when there is no fetch output.

  • (Technical Preview) Detects and captures disk partition labels—customer-assigned symbolic names for disk partitions.

  • Dropped support for YARN 2.5.x and earlier.

  • Rolls up the Pepperdata Supervisor v5.7 “dot-release” enhancements:

    • Added support for MapR 6.1. For detailed Hadoop distro support information, see Pepperdata-Platform Support.

    • Added support for self-signed HTTPS certificates on upload hosts in customer-hosted Dashboard installs.

    • The HTTPS uploader component of the Pepperdata Collector (the pepcollectd daemon) now conducts exponential backoff in the event of a failed upload host. This enables quicker discovery when a failed upload host returns to a healthy status, and improves load balancing.

    • Added a new configuration property for customizing how much CPU must be available in order for Capacity Optimizer to operate.

      To preserve the behavior of earlier Capacity Optimizer versions, which did not consider CPU availability when deciding whether or not to add containers, leave the free CPU percentage floor property (pepperdata.supervisor.allocation.create.idleCpuPercent) set to its default value, zero. For more information, see Free CPU Percentage Floor for your Supervisor version.

    • Added a new configuration property for customizing how much disk time must be available—time when the disk is not busy performing seeks, rotations, or data transfer—in order for Capacity Optimizer to operate.

      To preserve the behavior of earlier Capacity Optimizer versions, which did not consider disk time availability when deciding whether or not to add containers, leave the idle disk time percentage floor property (pepperdata.supervisor.allocation.create.idleDiskPercent) set to its default value, zero. For more information, see Idle Disk Time Percentage Floor for your Supervisor version.

    • Added collection of the used resources metric for Fair Scheduler.

      The new metric variables are rmq_used_mem and rmq_used_nvc; the searchable descriptions are Queue Info Used Memory Size and Queue Info Used Num Cores, respectively; and the metrics are in the Fair Scheduler Queue Capacity group of metrics.

      To chart these metrics, use the dashboard navigation bar to navigate to Charts & Tables > One Metric per Row or Charts & Tables > Compact Layout, enter the metrics group name into the search, scroll to and select the metric you want to show, and click Go.

    • Improved the accuracy of the queue duration metric for applications.

      The metric name is currentQueueDuration, its searchable description is job current queue duration seconds, and it’s in the ResourceManager group of metrics.

      To chart this metric, use the dashboard navigation bar to navigate to Charts & Tables > One Metric per Row or Charts & Tables > Compact Layout, enter the metrics group name into the search, scroll to and select the metric you want to show, and click Go.

    • Tez support and monitoring added to Application Profiler. Recommendations for monitored Tez applications appear in the Application Detail. (YARN Timeline Server must be running.) Regardless of whether you’ve previously enabled Application Profiler on your cluster, you must contact Pepperdata Support to enable Tez monitoring and recommendations.

    • Added ignore-match rule for custom program monitoring to provide more granular string matching rules. For details, see YAML Sections: Program Matching Rules for your Supervisor version.

Supervisor v5.7.18 (2019/06/06)

Enhancements:

  • Added support for MapR 6.1. For detailed Hadoop distro support information, see Pepperdata-Platform Support.

  • Added support for self-signed HTTPS certificates on upload hosts in customer-hosted Dashboard installs.

  • The HTTPS uploader component of the Pepperdata Collector (the pepcollectd daemon) now conducts exponential backoff in the event of a failed upload host. This enables quicker discovery when a failed upload host returns to a healthy status, and improves load balancing.

Bug Fix:

  • Fixed an issue that prevented stdout logging for Spark applications launched using Spring Boot.

Supervisor v5.7.15 (2019/05/08)

Enhancement:

  • Added a new configuration property for customizing how much disk time must be available—time when the disk is not busy performing seeks, rotations, or data transfer—in order for Capacity Optimizer to operate.

    To preserve the behavior of earlier Capacity Optimizer versions, which did not consider disk time availability when deciding whether or not to add containers, leave the idle disk time percentage floor property (pepperdata.supervisor.allocation.create.idleDiskPercent) set to its default value, zero.

Supervisor v5.7.14 (2019/04/16)

Bug Fix:

  • Fixed an issue that in environments running under JDK 7 or later caused a JMX RMI port to be opened.

Supervisor v5.7.13 (2019/04/12)

Bug Fix:

  • Fixed an issue where the pepcollectd HTTPS uploader did not honor a valid, custom Certificate of Authority (CA) located at the default system-specific CA bundle location.

Supervisor v5.7.12 (2019/04/01)

Enhancements:

  • Added collection of the used resources metric for Fair Scheduler.

    The new metric variables are rmq_used_mem and rmq_used_nvc; the searchable descriptions are Queue Info Used Memory Size and Queue Info Used Num Cores, respectively; and the metrics are in the Fair Scheduler Queue Capacity group of metrics.

  • Improved the accuracy of the queue duration metric for applications.

    The metric name is currentQueueDuration, its searchable description is job current queue duration seconds, and it’s in the ResourceManager group of metrics.

To chart these metrics, use the dashboard navigation bar to navigate to Charts & Tables > One Metric per Row or Charts & Tables > Compact Layout, enter the metrics group name into the search, scroll to and select the metric you want to show, and click Go.

Supervisor v5.7.9 (2019/03/04)

Bug Fix:

  • Fixed an issue that prevented Impala metrics files from being processed when anonymization is enabled for the Pepperdata Collector (the pepcollectd daemon).

Supervisor v5.7.7 (2019/02/26)

Bug Fix:

  • Fixed an issue that prevented the JobHistory Monitor from fetching Tez data when the YARN Timeline server is Kerberized, but the MapReduce Job History Server and ResourceManager are not Kerberized.

Supervisor v5.7.6 (2019/01/24)

Enhancement:

  • Tez support and monitoring added to Application Profiler. Recommendations for monitored Tez applications appear in the Application Detail. (YARN Timeline Server must be running.) Regardless of whether you’ve previously enabled Application Profiler on your cluster, you must contact Pepperdata Support to enable Tez monitoring and recommendations.

Bug Fix:

  • Fixed an issue that caused the JobHistory Monitor to fail when the MapReduce Job History Server and the Spark History Server are running on the same host and are both Kerberized.

Supervisor v5.7.5 (2019/01/15)

Bug Fix:

  • Fixed an issue that prevented the JobHistory Monitor from communicating with a Kerberized Spark History Server that returns empty Hadoop authentication token cookies.

Supervisor v5.7.4 (2018/12/21)

Bug Fix:

  • Fixed an issue that under rare, edge-case conditions caused the JobHistory Monitor to fail when YARN or Spark services restarted.

Supervisor v5.7.3 (2018/12/18)

Performance Enhancement:

  • The JobHistory Monitor makes fewer requests to the Kerberos server to fetch Spark data, which improves the server’s overall responsiveness.

Supervisor v5.7.2 (2018/12/12)

Enhancements:

  • Added ignore-match rule for custom program monitoring to provide more granular string matching rules.

  • Added a new configuration property for customizing how much CPU must be available in order for Capacity Optimizer to operate.

    To preserve the behavior of earlier Capacity Optimizer versions, which did not consider CPU availability when deciding whether or not to add containers, leave the free CPU percentage floor property (pepperdata.supervisor.allocation.create.idleCpuPercent) set to its default value, zero.

Bug Fix:

  • Roll up the fix from v5.6.17: removed a library JAR that, depending on your Hadoop classpath, could interfere with other PepAgent JARs and cause the PepAgent to crash.

Supervisor v5.6.19 (2018/12/21)

Bug Fix:

  • Fixed an issue that under rare, edge-case conditions caused the JobHistory Monitor to fail when YARN or Spark services restarted.

Supervisor v5.6.18 (2018/12/18)

Performance Enhancement:

  • The JobHistory Monitor makes fewer requests to the Kerberos server to fetch Spark data, which improves the server’s overall responsiveness.

Supervisor v5.6.17 (2018/12/12)

Bug Fix:

  • Removed a library JAR that, depending on your Hadoop classpath, could interfere with other PepAgent JARs and cause the PepAgent to crash.

Supervisor v5.6.16 (2018/11/30)

Bug Fix:

  • The JobHistory Monitor now successfully fetches data from all HTTPS REST endpoints.

Supervisor v5.6.15 (2018/11/27)

Enhancement:

  • Adds collection of Spark metrics for Spark jobs that you deploy in client mode on hosts without Pepperdata services.

Supervisor v5.6.14 (2018/11/02)

Enhancement:

  • Adds HTTPS support for Impala queries in flight.

Supervisor v5.6.13 (2018/10/18)

Enhancements:

  • Adds metrics collection for Impala queries in flight, which provides information about the query’s state (CREATED, INITIALIZED, COMPILED, RUNNING, FINISHED, and EXCEPTION).

    These metrics enable you to create alarms and alerts, such as queries in the RUNNING state for more than a given amount of time and too many queries are in the EXCEPTION state for the last 10 minutes. For information about creating alarms from the applicable metrics’ charts, see Create Alarms From a Chart View.

    Metrics collection for Impala queries in flight is disabled by default. To enable it, contact Pepperdata Support.

  • (Pepperdata Use Only) Adds additional data collection from the MapReduce Job History Server, which we’re using to improve the Spark heuristics recommendations. After we obtain sufficient data from enough users, we can improve the heuristics evaluations and recommendations. Therefore, we’re asking you to upgrade to Pepperdata v5.6.13 even if you do not need the metrics for Impala queries in flight.

Supervisor v.5.6.11 (2018/08/21)

  • Bug Fix

    • Fixes an issue that prevented Pepperdata Supervisor release 5.6.10 from producing package files for Apache Hadoop 2.7.2–2.8.0. Pepperdata Supervisor v5.6.11 provides correct packages for them.

Supervisor v.5.6.10 (2018/08/10)

  • Bug Fix

    • Fixes an issue that prevented Pepperdata Supervisor release v5.6.8 from producing package files for MapR 5.1+. Pepperdata Supervisor v5.6.10 provides correct packages for MapR 5.1+.

Supervisor v.5.6.8 (2018/08/08)

  • Bug Fix

    • Fixes an issue that occurred if the PepAgent was not configured to fetch data from the Spark History Server. The PepAgent no longer crashes under such conditions.

Supervisor v.5.6.7 (2018/08/03)

  • Enhancements

    • Easier, “sticky” configuration for Application Profiler.

      Instead of adding elements to .xml files, which can be overwritten when you upgrade Pepperdata, you now add configuration keys to pepperdata-config.sh. For detailed instructions, see Configure Application Profiler (version v7.1).

    • Dropped support for YARN 2.3. This means that if you are using Pepperdata Supervisor v5.5.x with the following distros, you must upgrade your distro when you upgrade to Pepperdata Supervisor v5.6.x:

      • Apache Hadoop 2.3.x YARN
      • Cloudera CDH 5.0.x YARN
      • Cloudera CDH 5.1.x YARN

    • During PepAgent startup, the presence of the Spark fetcher for the Job History monitor is confirmed. If the validation fails, an explanatory error message appears, and the PepAgent does not start.

    • Added support for _HOST macro expansion for Kerberos principal strings.

  • Bug Fix

    • Improved handling of file system stats with missing scheme names, which caused spurious exception messages that did not affect applications.

Supervisor v.5.5.17 (2018/07/19)

Bug Fix

  • Fixes an issue with Debian-based packages. If you’re using Debian, do not use Pepperdata v5.5.16.

Supervisor v.5.5.16 (2018/05/17)

  • Bug Fixes
    • Fixes an issue with the Pepperdata Custom Service Descriptor (CSD) for Cloudera Manager that incorrectly warned about missing log directory locations.
    • Fixes an authentication bug that prevented the PepAgent from starting for Apache Hadoop 2.8 and distros based on older Hadoop versions that have been backported with some newer Hadoop features. If you are successfully running Pepperdata, you were not affected by this bug.
  • Enhancements
    • Added support for Spark History Server basic access (BA) authentication in the JobHistory Monitor.
    • Improved debugging message when PepAgent fails to start due to errors in the Hadoop configuration parsing process.

Supervisor v.5.5.15 (2018/05/15)

  • Bug Fix
    • Fixes an issue that allowed multiple copies of pepcollectd or pepmetricsd daemons to run simultaneously on the same host. If you try to run multiple copies of either daemon, an error is shown on the console, and the additional daemon is prevented from starting.

Supervisor v.5.5.14 (2018/04/03)

  • Bug Fixes
    • Roll up fixes from v5.5.13 and v5.5.12.
    • Fixes an issue that could prevent PepAgent from starting up when a configured ResourceManager in an HA pair is down.

Deprecated. v.5.5.13 (2018/03/30)

  • Bug Fixes
    • Fixes an issue that could prevent Pepperdata ResourceManager metrics from being reported.
    • Fixes an overflow-induced error that could cause PepAgent to crash on hosts that have been up for a very long time.
    • Fixes an issue where data sampling in Application Profiler was incorrectly disabled, leading to performance degradation while fetching MapReduce task data from JobHistoryServer.
    • Fixes Application Profiler to support clusters that enable Kerberos in only one of ResourceManager and JobHistoryServer REST endpoints.

Deprecated. v.5.5.12 (2018/03/06)

  • Enhancements
    • Fetching the job history data for multiple apps is now performed in parallel, not serially. (Job history monitoring must be enabled.)
    • Support added for fetching job history data from an SSL-enabled Spark History Server. (Job history monitoring must be enabled.)
    • Validation checks added during pepagentd startup to ensure that the ResourceManager and JobHistoryServer authentication policies match the policies that pepagentd retrieves from the cluster’s Hadoop configuration, /etc/hadoop/conf by default. (Job history monitoring must be enabled.)
  • Bug Fixes
    • Fixes an issue that could prevent Pepperdata from fetching job history data from Hadoop daemons that are SSL-enabled. (Job history monitoring must be enabled.)

Supervisor v.5.5.10 (2018/02/23)

  • Bug Fixes
    • Fixes an issue introduced in 5.5.7 that can prevent PepAgent from functioning a week after it starts.

Supervisor v.5.5.9 (2018/02/20)

  • Bug Fixes
    • Fixes an issue with the Pepperdata Custom Service Descriptor (CSD) for Cloudera Manager that prevented replication of pepperdata-site.xml to all the ResourceManager hosts.

Supervisor v.5.5.8 (2018/02/16)

  • Bug Fixes
    • Fixes an issue with CDH Parcel packages that prevented the Pepperdata services from starting up.
    • Fixes an issue with the Pepperdata Custom Service Descriptor (CSD) for Cloudera Manager that prevented the PepAgent process from restarting automatically.

Supervisor v.5.5.7 (2018/02/02)

  • Enhancements
    • Added support for Kerberos authentication for Application Profiler fetches of job history data.
    • A new required control key was added for Policy Enforcer. If not using Policy Enforcer, no change is required. For details, see http://help.pepperdata.com/v5-5/install-guide/configuration/yenable-policy-enforcer/.
    • Collects disk resource metrics for MapR clusters.
    • Collects Spark related application specific config for Code Analyzer For Apache Spark.

Supervisor v.5.4.1 (2017/12/01)

  • Enhancements
    • Removes support for Classic Hadoop across all distros.
    • Adds an optional feature for collecting filesystem statistics without instrumenting the I/O path.
  • Roll up of all current 5.3 dot-release additions
    • Bug fixes
      • Fixes an issue that prevented some CDH package from being available for download.
      • Fixes a bug that could, under rare circumstances that employed multiple class loading schemes with different mechanisms (parallel-ready vs. non-parallel-ready class loading), cause a deadlock while attempting to load a Pepperdata-instrumented Hadoop class.

Supervisor v.5.3.11 (2017/11/21)

  • Bug Fix
    • Fixes an issue that prevented some CDH package from being available for download.

Supervisor v.5.3.10 (2017/11/16)

  • Bug Fix
    • Fixes a bug that could, under rare circumstances that employed multiple class loading schemes with different mechanisms (parallel-ready vs. non-parallel-ready class loading), cause a deadlock while attempting to load a Pepperdata-instrumented Hadoop class.

Supervisor v.5.3.8 (2017/10/25)

  • Enhancements
    • Collects metrics for the entire duration of large and/or long-running Spark applications that, in prior releases, would have resulted in dropped metrics beyond a fixed number jobs, stages, or executors.
    • Collects additional failure-related information for failed tasks in a Spark application.
    • Ensures that the cluster name specified in the configuration files matches the cluster name in license file on pepcollectd startup.
  • Bug Fixes
    • Fixes a bug that prevented Capacity Scheduler queue capacity metrics from being collected.

Supervisor v.5.2.4 (2017/09/01)

  • Enhancements
    • Application Profiler now includes support for Spark version 2.1, in addition to earlier supported versions.
    • Allows for better network utilization while processing Spark event logs for all supported Spark versions.
  • This release rolls up all current 5.1 dot-release additions.

Supervisor v.5.1.8 (2017/07/25)

  • Enhancements
    • Improved reliability of Spark driver metrics collection.

Supervisor v.5.1.7 (2017/07/14)

  • Enhancements
    • Improved collection of JVM metrics.

Supervisor v.5.1.6 (2017/07/12)

  • Enhancements
    • Reduced Pepperdata logging in Spark Driver and Executors to suppress irrelevant, internal-only messages.

Supervisor v.5.1.5 (2017/06/26)

  • Features
    • Application Profiler has graduated from early access and is generally available to customers. Please contact Support for details.
    • Code Analyzer for Apache Spark is available for Early Access. Please contact Support if you would like to participate in the Early Access Program.
    • Removed ability to upload via ssh/rsync protocol. Only https is now supported.
    • Removed ability to upload via signed-realm method. Only license key method is now supported.
    • HTTPS support for JobHistory Monitor.
  • Roll up of all current 5.0 dot-release additions
    • Bug fixes
      • Removes Play and JobHistory Monitor dependencies from PepperdataSupervisor jar.
      • Reduces error logging frequency in Pepperdata Spark support for benign errors.
      • Fixes PepCollector upload failures when anonymization is turned on.
      • Fixes a jersey package version conflict issue.
      • Fixes a bug where spark shuffle metrics are not reported for Spark 2.
      • Fixes a bug that could cause task failures under certain launch conditions.
      • Fixes a bug where some metrics from the ResourceManager were not anonymized before upload (if using optional anonymization).
      • Fixes a bug where Pepperdata logging might fail when limited implementations of log4j are provided.
      • Removes Pepperdata instrumentation for HBase Master when using Java Agent based install mechanism.
      • Fixes an issue that prevented pepcollectd from uploading for Cloudera CSD/Parcel based installs.
      • Fixes an issue that might cause a Hive server to abort on startup due to a logging library version conflict.
      • Fixes an issue that might cause a Spark notebook to fail due to a Play library version conflict.
      • Fixes typos in the job history monitor 50505 web page.
      • Fixed an issue to make PepAgent do nothing instead of resetting task renice values when the job group control is disabled.
      • Updated default program monitoring rule for Spark history server.

Supervisor v.5.0.22 (2017/06/20)

  • Bug fixes
    • Removes Play and JobHistory Monitor dependencies from PepperdataSupervisor jar.

Supervisor v.5.0.21 (2017/06/05)

  • Bug fixes
    • Reduces error logging frequency in Pepperdata Spark support for benign errors.
    • Fixes PepCollector upload failures when anonymization is turned on.

Supervisor v.5.0.20 (2017/05/14)

  • New feature
    • Added support for Spark applications in Application Profiler (beta).
  • Bug fixes
    • Fixes a jersey package version conflict issue.

Supervisor v.5.0.17 (2017/05/09)

  • Bug fixes
    • Fixes a bug where spark shuffle metrics are not reported for Spark 2.

Supervisor v.5.0.16 (2017/05/04)

  • New feature
    • Enhances Pepperdata debugging capabilities.
  • Bug fixes
    • Fixes a bug that could cause task failures under certain launch conditions.
    • Fixes a bug where some metrics from the ResourceManager were not anonymized before upload (if using optional anonymization).

Supervisor v.5.0.13 (2017/04/13)

  • Bug fixes
    • Fixes a bug where Pepperdata logging might fail when limited implementations of log4j are provided.
    • Removes Pepperdata instrumentation for HBase Master when using Java Agent based install mechanism.
    • Fixes an issue that prevented pepcollectd from uploading for Cloudera CSD/Parcel based installs.

Supervisor v.5.0.11 (2017/04/07)

  • Bug fixes
    • Fixes an issue that might cause a Hive server to abort on startup due to a logging library version conflict.
    • Fixes an issue that might cause a Spark notebook to fail due to a Play library version conflict.
    • Fixes typos in the job history monitor 50505 web page.

Supervisor v.5.0.10 (2017/03/28)

  • Bug fixes
    • Fixed an issue to make PepAgent do nothing instead of resetting task renice values when the job group control is disabled.
    • Updated default program monitoring rule for Spark history server.

Supervisor v.5.0.7 (2017/03/23)

  • New features
    • Added support for Application Profile (beta). Disabled by default and can be turned on through configuration.
    • Added default program monitoring rules for Spark Job History and Hadoop Job History servers.
    • Made the default uploading methodology to the Pepperdata-hosted Dashboard https-based, with a destination host of https://upload-main.pepperdata.com
  • Bug fixes
    • Improved robustness of pepcollectd uploads to customer-hosted Dashboard installs.
    • Added Pepperdata workflow id support for Spark Applications (only Java Agent).
    • Fixed an issue that prevented Pepperdata instrumentation on Debian installations when using Java Agent based install mechanism.
  • Roll up of all current 4.10 dot-release additions
    • New features
      • Adds alpha support for collecting Apache Impala query-level metrics (for details, please contact Pepperdata Support).
      • Added additional security headers to built-in servlet pages (on port 50505 and 50510 by default) of Supervisor and PepAgent.
    • Bug fixes
      • Fixes an issue that could cause jobs to fail due to unhandled null-pointer exceptions.
      • Fixes an issue that caused under-reporting of Impala CPU metrics.
      • Fixes an issue that prevented the correct launching of tasks in a combined CDH/Oozie/Spark environment, when Pepperdata was installed using the Java Agent based mechanism.
      • Fixes a bug that did not account for the host timezone when collecting Impala query metrics.
      • Fixes a bug that could cause failures in https uploads through a proxy.
      • Fixes an XML syntax error in /etc/pepperdata/pepperdata-site.xml-template.
      • Fixes an issue, for clusters using Java Agent based install mechanism, that created log directories in /tmp/to-pepperdata for every task launched even when there was nothing to log.
      • Makes it easier to explicitly specify the hosts that should or should not participate in dynamic allocation.
      • Fixes an issue that can crash ResourceManager during startup when DEBUG logging is enabled.

Supervisor v.4.10.16 (2017/04/07)

  • Bug fix
    • Fixes an issue that might cause a Hive server to abort on startup due to a logging library version conflict.

Supervisor v.4.10.15 (2017/03/06)

  • Bug fixes
    • Fixes an issue that could cause jobs to fail due to unhandled null-pointer exceptions.

Supervisor v.4.10.14 (2017/02/23)

  • Bug fixes
    • Fixes an issue that caused under-reporting of Impala CPU metrics.
    • Fixes an issue that prevented the correct launching of tasks in a combined CDH/Oozie/Spark environment, when Pepperdata was installed using the Java Agent based mechanism.

Supervisor v.4.10.12 (2017/02/17)

  • Bug fixes
    • Fixes a bug that did not account for the host timezone when collecting Impala query metrics.

Supervisor v.4.10.11 (2017/02/15)

  • New features
    • Adds alpha support for collecting Apache Impala query-level metrics (for details, please contact Pepperdata Support).

Supervisor v.4.10.10 (2017/02/02)

  • Bug fix
    • Fixes a bug that could cause failures in https uploads through a proxy.

Supervisor v.4.10.9 (2017/01/30)

  • Bug fix
    • Fixes an XML syntax error in /etc/pepperdata/pepperdata-site.xml-template.

Supervisor v.4.10.8 (2017/01/18)

  • Bug fix
    • Fixes an issue, for clusters using Java Agent based install mechanism, that created log directories in /tmp/to-pepperdata for every task launched even when there was nothing to log.

Supervisor v.4.10.7 (2017/01/12)

  • New features
    • Added additional security headers to built-in servlet pages (on port 50505 and 50510 by default) of Supervisor and PepAgent.
  • Bug fixes
    • Makes it easier to explicitly specify the hosts that should or should not participate in dynamic allocation.
    • Fixes an issue that can crash ResourceManager during startup when DEBUG logging is enabled.

Supervisor v.4.10.5 (2016/12/30)

  • New features
    • Adds support for an alternate method to configure and install Pepperdata via Java Agent (beta).
    • Ensures PepAgent reserved port (default 50505) is free when pepagentd service is started.
  • Bug fixes
    • Fixes an issue that can cause Spark 2.0 job failures when Pepperdata monitoring is enabled.
    • Improves identification of spark driver/launch tasks outside of the cluster.

Supervisor v.4.9.3 (2016/11/07)

  • New features
    • Support for HDP 2.5
    • Support for MapR 5.2
    • Support for Hadoop 2.7.3
    • Support for CDH 5.7.3
    • Provides a command line utility to extract Pepperdata license info from license
    • Support for custom workflow ids
    • Creates /etc/pepperdata automatically on package installation
    • Adds configuration template for some commonly used properties and vars in /etc/pepperdata
    • Allows Pepperdata daemon environment variables to be specified in /etc/pepperdata/pepperdata-config.sh
  • Bug fixes
    • Fixes a bug that reported zero values for cpu time and thread count if a task or daemon to monitor has very large CPU time values.

Supervisor v.4.8.11 (2016/10/03)

  • New features
    • Support for CDH 5.8
    • Support for Amazon EMR
    • Support for Apache Hadoop 2.7.1 and 2.7.2
    • Pepperdata daemon log files are always compressed upon rotation (some were rotated uncompressed before)
    • Support for using https for delivery of metrics to Pepperdata-hosted Dashboard
    • Support for running Pepperdata daemons as a configurable, non-privileged user.
  • Bug fixes
    • Fixes an issue with task launching when using LinuxContainerExecutor on Ambari managed HDP clusters
  • Rollup of 4.7 dot-release additions:
    • Support for HDP 2.4.2 and HDP 2.4.3
    • Support for SUSE Linux
    • Fixes a bug that could cause a deadlock when using CapacityScheduler with preemption enabled.
    • Fixes a bug that could cause non-Java jobs to fail when LinuxContainerExecutor is in use and PepAgent is down

Supervisor v.4.7.8 (2016/09/30)

  • New Features:
    • Support for HDP 2.4.3
  • Bug fixes
    • Fixes an issue that could cause non-Java jobs to fail when PepAgent is down.
    • Fixes a regression in Impala monitoring rules.
    • Fixes a bug that could cause a deadlock when using HDP 2.3 and above with CapacityScheduler and preemption enabled.

Supervisor v.4.7.6 (2016/09/16)

  • Bug fixes
    • Fixes an issue that could cause the jobs to fail when LinuxContainerExecutor is in use and PepAgent is down.

Supervisor v.4.7.3 (2016/08/11)

  • New features
    • Makes PepperdataSupervisor RPMs compatible with SUSE Linux Enterprise and variants.

Supervisor v.4.7.2 (2016/07/29)

  • New features
    • Adds optional resource max percent support in job groups as a beta-test feature.
  • Bug fixes
    • Supports use of command path, not just a command base name, in configuration of Impala monitoring.
    • Recognizes and monitors Impala processes even if the associated pid files are created after PepAgent started.

Supervisor v.4.7.1 (2016/07/15)

  • New features
    • Supports optional anonymization of metrics records and daemon logs where sensitive data such as user and job names will be encrypted before delivery to the Pepperdata-hosted Dashboard (if in use).
    • Merges pepcollectd into the PepperdataSupervisor rpm.
    • Adds an optional facility to set Pepperdata-specific Node Manager configuration properties in a file named pepperdata-site.xml, located by default in /etc/pepperdata/.
    • Hourly rotates log files that contains messages emitted by Pepperdata components running inside Hadoop daemons.
    • Adds OS and Hadoop software version metrics.
    • Adds an optional mechanism of uploading metrics to HTTPS receiver.
  • Bug fixes
    • Suppresses log4j config warnings caused by Pepperdata components in tasks.
  • Rollup of 4.6 dot release additions
    • Bug fixes
      • Fixes a bug that could incorrectly report spikes in collected metric values.
      • Reduces severity of routine Collector log messages.

Supervisor v.4.6.9 (2016/09/30)

  • New features
    • HDP 2.4.2 support
    • HDP 2.4.3 support
  • Bug fixes
    • Fixes a bug that could cause a deadlock when using HDP 2.3 and above with CapacityScheduler and preemption enabled.

Supervisor v.4.6.6 (2016/07/05)

  • Bug fixes
    • Fixes a bug that could incorrectly report spikes in collected metric values.
    • Reduces severity of routine Collector log messages.

Supervisor v.4.6.4 (2016/06/17)

  • New features
    • Limits the total CPU used by a PepAgent process, restarting when usage becomes too high. Under normal conditions, a single PepAgent process should run for months before automatically restarting.
    • Makes the default permissions on /var/log/pepperdata such that non-root users can read them. This can be controlled via the PD_LOG_DIR_PERMS setting in /etc/sysconfig/pepperdata.
    • Adds an optional facility to set Pepperdata-specific Resource Manager and PepAgent configuration properties in a file named pepperdata-site.xml, located by default in /etc/pepperdata/.
    • Moves the default location of all Pepperdata process PID files from /var/run/ to /var/run/pepperdata/.
    • Adds beta support for managing Pepperdata services, configuration, and package installation via Cloudera Manager (for details, please contact Pepperdata Support).
    • Adds alternatives and scripts to control PepMetrics plugins version independently of the Supervisor version.
    • Renames pepmetricsd tcollector PID file to allow standalone tcollector to run in parallel.
    • Allows PepCollector upload command to be configurable (for details, please contact Pepperdata Support).
  • Rollup of 4.5 dot release additions
    • Bug fixes
      • Fixes a bug that could omit collection of dynamic allocation metrics upon NodeManager restart.
      • Fixes a bug that caused a partial backoff of dynamic capacity allocation every six minutes, making it more conservative than necessary.
      • Fixes a bug introduced in the 4.5.12 pepcollectd that could prevent delivery of metrics to on-site Dashboards.
      • Fixes a bug where, in rare cases, Pepperdata’s monitoring of task configuration changes could cause an intermittent CPU busy-waiting condition.
      • Makes the option to prevent the preemption of containers in non-elastic queues (introduced in 4.5.13) more robust in the face of absolute queue capacities being expressed as floating point values.
      • Fixes a critical bug introduced in 4.5.12 where uploading to the Pepperdata-hosted Dashboard can fail after a week of running pepcollectd.
      • Provides an option to prevent capacity scheduler preemption of containers in non-elastic queues.

Supervisor v.4.5.20 (2016/09/30)

  • New features
    • Adds support for HDP 2.4.3

Supervisor v.4.5.18 (2016/09/21)

  • New features
    • Adds support for HDP 2.4.2.
  • Bug fixes
    • Fixes a bug that could cause a deadlock when using CapacityScheduler with preemption enabled.

Supervisor v.4.5.16 (2016/06/17)

  • Bug fixes
    • Fixes a bug that could omit collection of dynamic allocation metrics upon NodeManager restart.

Supervisor v.4.5.15 (2016/05/25)

  • Bug fixes
    • Fixes a bug that caused a partial backoff of dynamic capacity allocation every six minutes, making it more conservative than necessary.

Supervisor v.4.5.14 (2016/05/24)

  • Bug fixes
    • Fixes a bug introduced in the 4.5.12 pepcollectd that could prevent delivery of metrics to on-site Dashboards.
    • Fixes a bug where, in rare cases, Pepperdata’s monitoring of task configuration changes could cause an intermittent CPU busy-waiting condition.
    • Makes the option to prevent the preemption of containers in non-elastic queues (introduced in 4.5.13) more robust in the face of absolute queue capacities being expressed as floating point values.

Supervisor v.4.5.13 (2016/05/12)

  • Bug fixes
    • Fixes a critical bug introduced in 4.5.12 where uploading to the Pepperdata-hosted Dashboard can fail after a week of runningpepcollectd.
    • Provides an option to prevent capacity scheduler preemption of containers in non-elastic queues.

Supervisor v.4.5.12 (2016/05/04)

  • New features
    • Turns on monitoring for Impala job metrics by default (was so far a technology preview feature).
    • Turns on Tez I/O monitoring by default (was so far a technology preview feature).
    • Adds beta support for HBase table and HDFS metrics monitoring (for details, please contact Pepperdata Support).
    • Adds scripts to control which version of Supervisor/PepCollector is active, if multiple versions are installed on a node at the same time.
  • Bug fixes
    • Limits the maximum memory used by long-running PepAgent processes.
    • Reduces the possibility of deadlocks in Supervisor and PepAgents by adopting a new inter-thread communication protocol.

Supervisor v.4.4.7 (2016/05/26)

  • Bug fixes
    • Fixes a bug that caused a partial backoff of dynamic capacity allocation every six minutes, making it more conservative than necessary.

Supervisor v.4.4.3 (2016/03/25)

  • New features
    • Add optional monitoring for Impala job metrics.
    • Reduces the frequency of metrics file roll-over, which improves the speed of metrics appearing in dashboard.
    • Alleviate swapping by removing dynamic allocations when excessive paging is detected.
    • Improves dynamic allocation’s ability to handle large container sizes.
    • Gathers metrics about workflows, scheduler queues, and node labels.
    • Beta-test version of CSD for Cloudera Manager integration.
    • Reduce the maximum memory consumed by PepAgents.
  • Bug fixes
    • Fixes task launching for kerberized HDP clusters using Ambari.
  • Rollup of 4.3 dot release additions
    • Bug fixes
      • Prevent sending negative resource limits to Application Masters when dynamic allocation removes resources from a node.
      • Fixes a bug that could cause the Resource Manager to crash immediately after a refreshQueues command.
      • Fixes a synchronization issue that may slow down HBase queries.
      • Fixes dynamic allocation bug when the ResourceManager tracks multiple NodeManagers per host.
      • Fixes an issue where restarting pepagents can adversely affect supervisor monitoring and job scheduling.

Supervisor v.4.3.15 (2016/05/26)

  • Bug fixes
    • Fixes a bug that caused a partial backoff of dynamic capacity allocation every six minutes, making it more conservative than necessary.

Supervisor v.4.3.14 (2016/03/25)

  • Bug fixes
    • Prevent sending negative resource limits to Application Masters when dynamic allocation removes resources from a node.

Supervisor v.4.3.12 (2016/03/23)

  • Bug fixes
    • Fixes a bug that could cause the Resource Manager to crash immediately after a refreshQueues command.

Supervisor v.4.3.11 (2016/03/21)

  • Bug fixes
    • Fixes a synchronization issue that may slow down HBase queries.
    • Fixes dynamic allocation bug when the ResourceManager tracks multiple NodeManagers per host.

Supervisor v.4.3.10 (2016/02/24)

  • Bug fixes
    • Fixes an issue where restarting pepagents can adversely affect supervisor monitoring and job scheduling.

Supervisor v.4.3.7 (2016/02/05)

  • New features
    • Parcel support allows distribution and installation of Pepperdata from Cloudera Manager (Beta version).
    • Support for MapR 5.0, in both YARN and Classic mode.
    • Support for CDH 5.3.9 and 5.4.9.
    • Support for Apache Hadoop 2.6.3.
    • More application and node metrics to Pepperdata Dashboard.
    • More tests available to determine the correctness of Pepperdata install (Beta version).

Supervisor v.4.2.9 (2016/02/24)

  • Bug fixes
    • Fixes an issue where restarting PepAgents can adversely affect supervisor monitoring and job scheduling.

Supervisor v.4.2.7 (2016/01/27)

  • Bug fixes
    • Fix a problem that PepAgent fails to start when Hadoop is not in the path.

Supervisor v.4.2.6 (2016/01/21)

  • Bug fixes
    • Restore base resource capability to a node when it is deactivated.
    • Remove harmless log warnings about unknown scheduler events.

Supervisor v.4.2.5 (2016/01/14)

  • New features
    • Finer grained control of dynamic allocation algorithm.

Supervisor v.4.2.4 (2015/12/23)

  • Bug fixes
    • Fix an issue with experimental custom Spark-client monitoring.

Supervisor v.4.2.3 (2015/12/22)

  • New features
    • Reduction of upload bandwidth to the Pepperdata Dashboard.
    • Allow explicit enabling/disabling of nodes for split testing.
    • More metrics on Application Master resource requests.
    • Beta support for Apache Tez I/O monitoring and throttling.
    • Spark 1.5 support.
    • CDH 5.5 support.
    • MapR 4.1 MRv1 support.
  • Bug fixes
    • ResourceManager could display incorrect information on startup when using split testing.

Supervisor v.4.1.7 (2015/12/11)

  • Bug fixes
    • Refresh license information on web page immediately upon refresh queues.

Supervisor v.4.1.6 (2015/12/04)

  • New features
    • Report Spark Shuffle, DAG, BlockManager metrics for Spark jobs in client mode.
    • Additional metrics for task start time, CPU info, and Tez input tasks.
    • Standalone script to check Pepperdata license status (print_license_status).
  • Bug fixes
    • Fix a problem that caused metrics collection to be skipped on Hive and Oozie based jobs.
    • Fix an issue that could cause dynamic allocation to stay active even if it has been explicitly disabled.

Supervisor v.4.0.13 (2015/11/05)

  • Bug fixes
    • Fix an issue that could cause Tez or MapR-classic tasks to fail during filesystem access.
    • Improved efficiency of Pepperdata Dashboard storage for YARN tasks when work-preserving restarts are used.

Supervisor v.4.0.12 (2015/10/30)

  • Bug fixes
    • Disabled Apache Tez I/O monitoring and throttling support by default.

Supervisor v.4.0.10 (2015/10/22)

  • New features
    • Installation
      • A Pepperdata License Key is now required. Please contact Pepperdata Support for upgrades.
      • /usr/lib/hadoop/lib/PepperdataSupervisor.jar only created if /usr/lib/hadoop exists. /opt/pepperdata/lib/PepperdataSupervisor.jar is always created.
      • PepAgent does not start if /var/log/pepperdata does not have enough free space. The default free space requirement is 5GB and can be changed by modifying PD_MAX_LOG_DIR_SIZE.
      • The default value of PD_LOG_PROC_MAX_AGE_DAYS has been decreased from 92 days to 7 days. Files older than PD_LOG_PROC_MAX_AGE_DAYS will not be processed by pepcollectd and may be deleted.
    • Core functionality
      • Default job group is implicitly created if it is not specified
      • Launched tasks/containers automatically get the PepperdataSupervisor jar added to their classpath.
      • Supports Hadoop 2.7.x.
      • Supports MapR 4.1.x.
      • Supports HBase 1.1 monitoring and resource prioritization.
      • Monitors Spark shuffle metrics for Spark 1.3.x and later.
      • Beta support for Apache Tez I/O monitoring and throttling
      • Default job group is implicitly created if it is not specified
      • Launched tasks/containers automatically get the PepperdataSupervisor jar added to their classpath.
  • Rollup of 3.4 dot release additions
    • Bug fixes
      • Cleans up unnecessary log messages.
      • Allows users to set minimum allocation sizes in yarn to 0
    • New features
      • Core functionality
        • Recognizes classic Hadoop fair scheduler pool names in Dashboard and resource prioritization.

Supervisor v.3.4.24 (2016/03/02)

  • New features
    • Support for Apache Hadoop 2.6.2.
    • Support for HDP 2.2.9.

Supervisor v.3.4.22 (2016/01/21)

  • Bug fixes
    • Restore base resource capability to a node when it is deactivated.

Supervisor v.3.4.21 (2015/11/05)

  • Bug fixes
    • Fix an issue that could cause Tez or MapR-classic tasks to fail during filesystem access.
    • Improved efficiency of Pepperdata Dashboard storage for YARN tasks when work-preserving restarts are used.

Supervisor v.3.4.19 (2015/10/29)

  • Bug fixes
    • Keeps dynamic node states during a refresh command.

Supervisor v.3.4.18 (2015/09/24)

  • New features
    • Core functionality
      • Recognizes classic Hadoop fair scheduler pool names in Dashboard and resource prioritization.

Supervisor v.3.4.17 (2015/09/17)

  • Bug fixes
    • Cleans up unnecessary log messages.

Supervisor v.3.4.16 (2015/09/01)

  • Bug fixes
    • Allows users to set minimum allocation sizes in yarn to 0.

Supervisor v.3.4.13 (2015/09/14)

  • New features
    • Core functionality
      • In dynamic allocation, preempt mapreduce tasks before spark jobs.
      • Improved dynamic slots and allocation
    • Monitoring
      • Deprecates duplicate jvm memory stats
      • Collects more jvm metrics
      • Added per node environment metrics, includes version information for JRE, Hadoop, Pepperdata, Linux kernel, and OS.
      • Added more spark metrics
      • Detect swap events (experimental)
    • Logging
      • Logs errors if the default container executor is used against secure setup instead of the linux container executor
      • Puts Pepperdata related task tracker logs in a separate file
  • Bug fixes
    • Core Functionality
      • Improve the way resource manager tracks node resource changes.
      • Improve the way dynamic allocator handles the case where it requires more resource than currently free
    • Logging
      • Clean up logging

Supervisor v.3.3.14 (2015/09/07)

  • Collector only release, no supervisor changes.

Supervisor v.3.3.13 (2015/07/21)

  • Bug fixes
    • Fixes a bug so that node manager JARs for the localization protocol is not accessed unnecessarily.

Supervisor v.3.3.7 (2015/06/24)

  • Bug fixes
    • Fix a communication bug that can disrupt communication between the Supervisor and the PepAgents when many nodes become disconnected.
    • Prevent a PepAgent stack overflow when walking process trees for stale processes.

Supervisor v.3.3.3 (2015/06/08)

  • Bug fixes
    • Fix a problem where containers launched with some custom Java options would not be monitored with Pepperdata.
    • Fix a problem that prevented the Job Group config linter from running.

Supervisor v.3.3.2 (2015/05/06)

  • New features
    • Installation & running
      • Simplify configuration by using the same YARN scheduler names, irrespective of version of Hadoop.
      • Support HBase 0.98.
  • Bug fixes
    • Fix a problem where dynamic allocation could cause too much resource allocation in YARN.
    • Fix a problem introduced in 3.2.11 that could cause a failure to remove containers in YARN.
    • Fix a problem where dynamic allocation could allocate too few resources when using YARN & fair scheduler.

Supervisor v.3.2.11 (2015/04/20)

  • New features
    • Core functionality
      • In DynamicAllocation allow independent configuration of backoff timers as well as addition timers. (Ask PD support for using these advanced options.)
      • Allow separate values for configuration overrides of disk read & write bandwidth values.
    • Monitoring
      • Capture more resource manager metrics.
      • Capture more swap metrics.
      • Track YARN container exit status.
      • Capture Hadoop-daemon specific debugging information into the Pepperdata logging area.
    • Installation & running
      • Support Apache Hadoop 2.6 & distributions based upon it.
      • Support MapR 3.0.3.
      • pepagentd properly restarts after a machine reboot.
      • Add a utility to look at jmx metrics files and print out the bean names found.
  • Bug fixes
    • Monitoring
      • Eliminated an edge case that could show spurious very large CPU spikes.
    • Logging
      • Decreased some excess logging
  • Rollup of 3.1 dot release additions
    • New features
      • Core functionality
        • More aggressively reduce allocated resources when nodes are out of memory.
    • Bug fixes
      • Core functionality
        • Increase API compatibility with more Hadoop versions and configurations. (Including an important Apache 2.x Kerberos fix)
        • Ensure compatibility of YARN tasks launched with rewritten java options.
      • Logging
        • Decrease verbosity of dynamic allocation logging.

Supervisor v.3.1.15 (2015/03/16)

  • New features
    • Core functionality
      • More aggressively reduce allocated resources when nodes are out of memory.
  • Bug fixes
    • Core functionality
      • Increase API compatibility with more Hadoop versions and configurations. (Including an important Apache 2.x Kerberos fix)
      • Ensure compatibility of YARN tasks launched with rewritten java options.
    • Logging
      • Decrease verbosity of dynamic allocation logging.

Supervisor v.3.1.11 (2015/02/02)

  • New features
    • Core functionality
      • Dynamic capacity management for YARN based clusters (Similar to dynamic slot management for MRv1)
      • Beta support of Spark-on-YARN
      • Node state information sent from PepAgents to Supervisor
      • Separate create and remove delays in dynamic capacity/slot management
    • Monitoring
      • Additional metrics surrounding slots & containers
      • Additional metrics surrounding node memory usage
      • Additional JVM metrics collected from JobTracker/ResourceManager/PepAgent
      • Collect JMX metrics from NodeManager
    • Logging
      • Use local time zone and show UTC offset in text log files
  • Bug fixes
    • Installation/running
      • Improved shutdown code of PepAgent when potentially hung
  • Rollup of 3.0 dot release additions
    • New features
      • Core functionality
        • Improvements to dynamic slot management when using Fair Scheduler
        • Improvements to dynamic slot management with certain workloads
        • Reduced memory usage in JobTracker/ResourceManager
        • Support Dynamic Slot Creation on MapR v. 3.1.x
      • Monitoring
        • Additional metrics surrounding killed tasks
    • Bug fixes
      • Core functionality
        • Fixed a bug (introduced in 3.0.14) that could have caused overly aggressive dynamic slot creation on certain workloads.
      • Logging
        • Fixed a bug when enabling debug-level logging of NodeManager.

Supervisor v.3.0.15

  • Bug fixes
    • Core functionality
      • Fixed a bug (introduced in 3.0.14) that could have caused overly aggressive dynamic slot creation on certain workloads.

Supervisor v.3.0.14

  • New features:
    • Core functionality
      • Improvements to dynamic slot management when using Fair Scheduler
      • Improvements to dynamic slot management with certain workloads
    • Monitoring
      • Additional metrics surrounding killed tasks
  • Bug fixes
    • Core functionality
      • Reduced memory usage in JobTracker/ResourceManager

Supervisor v.3.0.10

  • Collector-only release - Supervisor 3.0.10 is equivalent to 3.0.9.

Supervisor v.3.0.9

  • New features
    • Core functionality
      • Support Dynamic Slot Creation on MapR v. 3.1.x
  • Bug fixes
    • Logging
    • Fixed a bug when enabling debug-level logging of NodeManager.

Supervisor v.3.0.7

  • New features
    • Core functionality
      • YARN support
  • Bug fixes
    • Monitoring
      • Improved metric reporting at task start time
  • Rollup of 1.11 dot release additions
    • New features
      • Core functionality
        • Support MapR v. 3.1.x.
      • Monitoring
        • Correctly monitor usage metrics on tasks in reused JVMs.
    • Bug fixes
      • Core functionality
        • Improved disk capacity estimation
        • Fixes in support of MapR v. 3.1.x. MapRFS throttling
        • Fix for CDH5 MRv1, identifying HDFS locality
        • Prevent a potential JobTracker lockup when PepAgents are shut down.

Supervisor v.1.11.22

  • Core functionality
    • Prevent a potential JobTracker lockup when PepAgents are shut down.

Supervisor v.1.11.21

  • Core functionality
    • Improved disk capacity estimation.
  • Monitoring
    • Correctly monitor usage metrics on tasks in reused JVMs.

Supervisor v.1.11.20

  • Core functionality
    • Fixes in support of MapR v. 3.1.x. MapRFS throttling.
    • Fix for CDH5 MRv1, identifying HDFS locality.

Supervisor v.1.11.15

  • Installation & running
    • Support MapR v. 3.1.x.

Supervisor v.1.11.13

  • Core functionality
    • Improved JobGroup controls for CPU and RAM
    • Improved disk capacity estimation
    • Explicit configuration of unlimited JobGroups
    • HBase minimum resource protection via JobGroups
    • Add an explicit default JobGroup capability
    • Automatic A/B testing of dynamic slot management capacity improvements
    • Improve dynamic slot management on mixed workloads
    • Turning off global controls disables dynamic slot management
    • Improve monitoring and controlling of separate physical resources
    • Installation & running
    • Deprecate old dynamic slot configuration in favor of percentage based configuration
    • Remove Zookeeper dependence
  • Monitoring
    • Collect more socket, CPU usage statistics
    • Collect disk space statistics
    • Improved metric collection around task starting & ending
    • Collect JMX statistics from various Hadoop daemons
  • Logging
    • Move text logs from Pepperdata daemons into dedicated directories

Supervisor v.1.10.22

  • Core functionality
    • Ensure that the FILE_* job counters are accurate.

Supervisor v.1.10.21

  • Core functionality
    • Improved the dynamic slot creation for certain kinds of workloads.

Supervisor v.1.10.18

  • Core functionality
    • Improved the performance of Pepperdata instrumentation.
    • Disable instrumentation when pepagent is not running.
  • Installation & running
    • Added an option to specify the mapred user name instead of the mapred group name.

Supervisor v.1.10.11

  • Core functionality
    • Ensured that pepagentd continues to run in the face of several recoverable error conditions.
  • Logging
    • Added a utility program to print proto metrics files in a easily readable way.

Supervisor v.1.10.5

  • Core functionality
    • Multiple software layers that access the same underlying hardware can be tracked (and controlled) as a combined unit, as well as the old separate behavior.
    • Introduces a throttler for IOPS. (Disabled by default)
    • Disk capacity estimation does not depend on lsblk.
    • Improved slot management for fast-running tasks. NOTE: The phantom slot is moved from a reducer to a mapper.
  • Monitoring
    • Additional node and task network stats are tracked.
    • Additional node and task I/O stats are tracked.
    • Web servlets are self-contained and make no internet requests (for e.g. for utility javascript libraries like YUI)
    • Additional node-level swap stats are tracked.
    • PepAgent monitors its own usage.
  • Logging
    • Shrunk log size by using protobuf binary format. Text format is available as an option.

Supervisor v.1.9.12

  • Core functionality
    • Introduces new configuration options for specifying the minimum number of slots as a percentage of the total available.
    • Allows the underlying Hadoop installation to be upgraded without restarting pepagentd.
  • Monitoring
    • Added a near real-time cluster status monitoring page at http://jobtracker:50510/Cluster
    • Added a near real-time job groups monitoring page at http://jobtracker:50510/JobGroups
    • In monitoring webpages, sort lists of hosts alphabetically by default.
  • Logging
    • Improved logging of some error conditions.

Supervisor v.1.9.1

  • Core functionality
    • Most communication between the Supervisor and the PepAgents now uses a tree-based TCP connection. Zookeeper usage is still present, but minimized.
    • Memory protection against task (including forked children) consuming too much virtual or physical RAM.
  • Monitoring
    • Improved information available via Supervisor and PepAgent built-in status web pages.
  • Logging
    • Introduced flag to enable JobTracker node metrics logging.

Supervisor v.1.8.27

  • Core functionality
    • Fix a bug that can cause throttling of tasks that should not be throttled due to an integer overrun.

Supervisor v.1.8.21

  • Core functionality
    • Fix a bug where the Supervisor could miss the termination of some tasks - now all tasks in a job are considered terminated when their job completes.

Supervisor v.1.8.15

  • Core functionality
    • Fix a bug that caused JobTrackerMetrics to be missing from the JobTracker’s JMX report on some Hadoop versions.
  • Logging
    • Reduce frequency and severity of messages in pepagent log.
    • Roll pepagent log hourly.

Supervisor v.1.8.9

  • Core functionality
    • Ensure that HBase daemon monitoring consistently reports HDFS metrics. (Bugfix)
    • Various bug fixes.
  • Installation & running
    • Option to read from JMX port directly instead of HTTP port to retrieve datanode JMX beans.
  • Logging
    • Add bandwidth data (in addition to absolute counters) for task/hbase filesystem metrics.

Supervisor v.1.8.4

  • Core functionality
    • Monitoring HBase daemon resource usage.
    • Various bug fixes.
  • Installation & running
    • Renamed /etc/sysconfig/pepagentd configuration to /etc/sysconfig/pepperdata configuration. The old name, /etc/sysconfig/pepagentd, will still work but is depreciated. The daemon’s name is still pepagentd.
  • Logging
    • Collect additional node, JobTracker, and job metric data.
    • Fix permissions on logging directory paths so that the Supervisor (running in the JobTracker) can log data there.

Supervisor v1.7.11

  • Core functionality
    • Various bug fixes.
    • Increase visibility of dynamic RAM management operations.
    • Compatibility with Debian Squeeze (6.0)
  • Logging
    • Reduced log output on TaskTracker.

Supervisor v1.7.3

  • Core functionality
    • Fixed a Supervisor bug that exhibits only on clean installs.

Supervisor v1.7.0

  • Core functionality
    • Add JobGroup functionality. (Link to docs to follow.)
    • Dynamic RAM management to increase throughput and node reliability. (Link to docs to follow.)
  • Logging
    • Convert several logs from .proto (protobuf binary) format to compressed (bzip2) text format.
    • Add disk-level I/O data and node-level memory data.

Supervisor v1.6.16

  • Core functionality
    • Greatly increase performance of assign_tasks by writing to Zookeeper in a background thread.
  • Logging
    • Reduce frequency of task assignment messages that get printed into the JobTracker’s log.
    • Fix a spurious log4j warning about “No appenders could be found for logger” that got printed into task logs.

Supervisor v1.6.13

  • Core functionality
    • Better support for custom Java installations.

Supervisor v1.6.10

  • Installation and running
    • Fixes PD_LOG_DIR in /etc/sysconfig/pepagentd to be respected.

Supervisor v1.6.8

  • Installation and running
    • Moved python into the /opt/pepperdata/supervisor-{version} directory. All files in the RPM are now underneath that unique path base.
  • Core functionality
    • Provided a configurable cap on Zookeeper writes per second.
  • Logging
    • Moved several log lines from INFO to DEBUG level.

Supervisor v1.6.6

  • Installation and running
    • Addition of single-run mode to /opt/pepperdata/supervisor/pepagentd when PD_SINGLE_RUN=1 is in the environment.
    • mkdir/rmdir (when empty) of expected paths containing symlinks created by alternatives. Helps when Hadoop is installed in “unusual” locations.
  • Core functionality
    • Provided a method to disable per-task metrics collection counters.
  • Logging
    • Improved error logging when PepAgent is down or unreachable.

Supervisor v1.6.3

  • Installation and dependencies
    • RPM targets architecture x86_64, so can be installed via yum on both amd64 and x86_64 systems.
    • Renamed /etc/sysconfig/pepagentd to /opt/pepperdata/supervisor-{version}/sysconfig/pepagentd-template, and use alternatives to make the template file in /etc/sysconfig/pepagentd-template.

Supervisor v1.6.0

  • Core functionality
    • Provide (and use by default) a new control mechanism for dynamically allocating local disk, HDFS, network, and shuffle resources among tasks to achieve minimums/maximums while using resources efficiently. New “resource specs” are used to configure the allocations.
    • Allow specification of minimum bandwidth allocation per task (not just maximum) for local disk, HDFS, network, and shuffle.
    • Optimize PepAgent performance.
    • Use an http server to show the current control settings and task metrics. Also add resource spec metrics to the Supervisor JMX.
    • Provide a fast method to turn off all Pepperdata controls on a cluster.
    • Add counters for per-task metrics collection.
  • Installation and dependencies
    • Support Apache Hadoop 0.20.2.
    • Use “alternatives” during RPM installation.
    • Install software in /opt/pepperdata/ instead of /usr/share/pepperdata/.
    • Remove use of cgroups for resource control.
    • Replace avro with protocol buffers for logging and communication.
    • Package a standard version of python in the RPM to be installed and used only by Pepperdata software.
  • Logging
    • Reduce volume of log messages that occur because pepagent is down.
    • Add consistent startup message to each component, for ease of troubleshooting.

Supervisor v1.5.:

  • Fix a HDFS throttling compatibility issue with cdh3u4.

Supervisor v1.5.3

  • PDTaskController is included in the package, simplifying configuration. One cluster-global config, including job clients, is now possible, without the Pepperdata jars being present on client nodes. Individual throttler classes are no longer specified in global configs, but instead added by the PDTaskController.
  • Data from various throttling classes is now logged in /var/log/pepperdata/metrics/ avro log files.
  • HDFS throttling can be specified in terms of local node HDFS, rack-local HDFS, and remote HDFS bandwidth, separately.
  • Shuffle phase bandwidth is monitored, logged, and throttle-able.

Supervisor v1.4.0

  • HDFS rate monitoring and throttling now available via the com.pepperdata.supervisor.agent.resource.DistributedFileSystemWrapper class.
  • /usr/share/pepperdata/pepagent_health_checker added, allowing PepAgent errors to be blacklist the node and propagate to the JobTracker JMX report.
  • Local filesystem rate throttler refuses to throttle so slowly that the task times out. (Throttling still happens, but with a higher limit than was requested.)

Supervisor v1.3.0

  • /usr/share/pepperdata/pepagent_health_checker is included. This allows PepAgent ERROR/FATAL conditions to temporarily blacklist the node and be seen in the JobTracker’s JMX data.
  • Process memory collection enabled on CentOS 5.
  • Datanode metric data collection enabled for configs of CDH3 running on CentOS. (As well as other combinations, this was a bugfix.)

Supervisor v1.2.1

  • Changed to three number versioning scheme: major.minor.bugfix
  • Error messages that are repeats will not flood reporting.
  • CentOS 5 compatibility - cgroups support is now an enhancement, not a requirement.
  • PepperdataSupervisor reports more errors and metrics to JMX.

Supervisor v1.1

  • Guava is packaged (with a shaded name of com.pepperdata.guava) inside PepperdataSupervisor*.jar. PepperdataSupervisor and PepAgent will run on hosts with or without a dedicated Guava install.
  • pepagentd (the runscript) is now configurable. /etc/sysconfig/pepagentd has the control options commented out but documented. Please look there. This will allow (among other things) site-specific commands like -Djava.library.path=… to be specified.
  • pepagentd (the runscript) supports a non-daemonizing debug mode, for debugging system configuration issues.
  • BPS as an I/O resource limit specification has been renamed bytesPerSec for clarity. (Of bytes vs bits per second.) Please see Running Jobs for more information on available resource limits.
  • Default jar install location has been moved from /usr/lib/hadoop to /usr/lib/hadoop/lib for compatibility with earlier versions of Hadoop.
  • NOTICE and LICENSE files added to /usr/share/pepperdata
  • Allow Zookeeper port specification as host:port in the zookeeper.host configuration directive on each host. (If multiple Zookeeper hosts are specified, they do not have to all have the same port.) Please see Property Definitions for more information on available configuration directives.
  • PepAgent now respects configuration directives specified in mapred-site.xml
  • PepperdataSupervisor RPM now correctly indicates dependency on cgroups and python.

Supervisor v1.0

Initial release.