/jobdetails

Calling arguments

To access job details resources, access the /jobdetails endpoint by sending an HTTP GET command with the desired arguments. All of the HTTP commands are GETs, so the simplest way is to construct a URL by adding arguments to the endpoint URL (for example, https://dashboard.pepperdata.com/pepperdata-demonstration/api/jobdetails?limit=2). By default, the response contains the most recent apps in a realm.

The table describes the calling arguments.

Argument Description Notes
s Start time of query Can be absolute (2020/07/20-05:54), relative (1h), or timestamp(1595163240000)
e End time of the query Defaults to current time
tzo TimeZone offset in hours Example: PDT: -7
j Filter by job IDs Comma-separated list. Example: 1679427987070_2188,1679427987070_2255
apptype Type of app Can be spark, mapreduce, or tez
finalstatus Final status of the app Possible values: SUCCEEDED, FAILED, KILLED, ENDED
resolvedstatus Resolved status of the app First available of FINAL_STATUS or APP_STATE; otherwise INVALID
appstate Current status of the app Possible values: NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED
sdelay Minimum time (ms) in queue Measured from submission to first container launch
limit Maximum number of apps to return Limited to 10,000
showalldetails App profiler details Set to 1. Returns configuration properties, recommendations, and counters
recommendationsonly Return only apps with recommendations Equivalent to recommendationsgte=low
recommendationsgte Filter by minimum severity level For example, recommendationsgte=moderate returns only apps with moderate or higher recommendation level
recommendationsmatching Filter by specified severity levels Comma-separated list. Possible values: critical, severe, moderate, low, none
fetchsparkdetails Spark driver metadata about executors and stages Set to 1. Requires app type is spark. Limited to most recent 100 jobs, 1000 stages, 24 hours
fetchtaskdetails Container error exit codes Set to 1. Limited to 50 containers with errors tracked and displayed
workflowid Filter by specified workflow ID  
filterjobidsafter Return only jobs running after the specified job ID Use this to paginate through a set of results that exceeds the maximum return size
namespace Filter by cluster namespace Kubernetes only
showmaybesettleddata Data that is asynchronously populated in later stages of the metrics processing pipeline Set to 1. For yarn this includes max memory used across all containers. For Kuberntes, it includes max memory for the pods and details about pods completion status of driver/master pods.

Pagination

Requests to the /jobdetails endpoint return a maximum of 10,000 apps. To see more, use the filterjobidsafter argument.

For convenience, the response header contains the appropriate value to use in the next call. For example, if the first batch of 10,000 results has

filterjobidsafter: 1649700263217_84251

among the response headers, use the following URL to request the next batch:

https://dashboard.pepperdata.com/pepperdata-demonstration/api/jobdetails?filterjobidsafter=1649700263217_84251