Configure History Fetcher Retries (Parcel)

To ensure that application history is successfully fetched from the applicable component (MapReduce Job History Server for MapReduce apps, or Spark History Server for Spark apps), the Pepperdata Supervisor uses a two-phase approach. Phase 1 makes the initial attempt to fetch the history, and if it fails, makes up to three retries. Phase 2 adds an additional try and by default up to five retries, with the interval between retries increased by a factor of five every time. You can customize the number of retries for each phase, which might be required for environments with extreme network latency or frequent connectivity issues.

Related Topics

• Configure Connection Timeout for Spark History Server

Procedure

1. Add the environment variables for the number of history fetcher retries, for either or both fetching phases.

   Use Cloudera Manager to add the environment variables for the number of history fetcher retries to the Pepperdata > Configuration > Pepperdata Service Environment Advanced Configuration Snippet (Safety Valve) template.

   Add the environment variables in the following format. Be sure to replace the default number of retries for the first and second phases (3 and 5, respectively) with your custom values.

   PD_JOBHISTORY_MONITOR_FIRST_RETRY_COUNT=3
   PD_JOBHISTORY_MONITOR_SECOND_RETRY_COUNT=5
   

2. Restart the Pepperdata services.

   In Cloudera Manager, select the Restart action for the Pepperdata service.