/m (Metrics)

The Pepperdata REST API resource for metrics, series, and filters is /your-realm-name/api/m.

Metrics API Query String Arguments

querystring key description
s Start time, in the format YYYY/MM/DD-HH:mm.
e End time, in the format YYYY/MM/DD-HH:mm.
tzo Time zone, in offset from UTC hours.
m Metric to access. For the list of metrics, refer to the Metric Definitions page on your dashboard (on the navigation bar, click Help > Metric Definitions); click or hover over the metric you want to see the metric name, which appears after the #m= part of the URL.
sample (required if targetpoints not specified; n/a otherwise) Sample interval (time between points returned), in milliseconds. WARNING: A small value over a long time period is expensive. The minimum value is 5000 (5 seconds). For a query that lasts a few hours, we recommend a value of 10000 (10 seconds).
targetpoints (required if sample not specified; n/a otherwise) Target number of points to return. The actual number of points returned can be greater or fewer depending on the sample size that is cached.
ms (optional, default=20) Maximum Series to return. If the number of series exceeds this value, we show the top n series and include the remaining series as aggregates: sum_of_the_others, max_of_the_others, average_of_all_matched_series.
nohostsreporting (optional, default=off) Include or omit the hostsreporting in the response. To omit hostsreporting, set to 1.
omitpoints (optional, default=off) Include or omit datapoints in the response. To omit all datapoints in the response, set to all.
downsampler (optional, default=off) Flag to split a single series into percentiles by time. To split a series, set to percentile. NOTE: The percentile downsampler applies only to queries that would otherwise return a single series.
format (optional, default=json) Format. For the fastest API performance, use the JSON format (lowercase json). To get the query in CSV format, set to (lowercase) csv.
omitaggregates (optional, default=off) Include or omit aggregate series—sum_of_the_others, max_of_the_others, and average_of_all_matched_series—in the response. To omit aggregate series, set to 1.
dataonly (optional, default=off) Include or omit progress updates and other performance instrumentation data in the response. To omit progress updates and instrumentation data, set to 1.

Metrics Breakdowns and Filters

You can add series breakdowns and/or filters to the metrics’ requests.

To get the breakdown of all the results, use * for the querystring value. To specify a filter, use the specific string. For example, to breakdown results by hosts, add the querystring &h=*. To filter by a given hostname, for example, “my-host”, add the querystring &h=my-host. To perform a regex match, add the querystring &h=/.*/.

You can specify multiple breakdowns. For example, for all jobs on a specific host, my-host, add the querystring, &h=my-host&j=*.

The table describes the breakdowns and filters options.

querystring key description
h host
u user
q queue
c control domain
j job sequence id
tt task type
t task

The only valid value is *. This querystring key cannot be used by itself. It must be used with another filter. For example, to get all the tasks for a job, use &j={JOBID}&t=*.
d device

Data Format of the Metrics API

The Pepperdata REST API metrics resource returns a single JSON object, defined as follows:

  • expectedCount: Expected number of progress chunks.
  • (omitted if dataonly=1) progress: Array of progress updates during the query runtime.
  • (omitted if dataonly=1) instrumentation: Debug information about the query, such as timing numbers.
  • data: Data payload.
    • allseries: Array of timeseries returned timeseries.
      • {timeseries}: timeseries object inside allseries.
        • dataPoints: Array of arrays of the form [[timestamp, value], [timestamp,value] ...]]
        • summaryStats: Object that contains the summaries for each series, such as area, average (mean), min, p05, max, and p95.
        • name: Object that contains name details about the series; varies by breakdown/filter.
        • seriesId: Unique name for this series.
    • sampleMillis: Distance between points, in milliseconds.
    • startTimeMillis: Start timestamp.
    • hostsReporting: Object with the number of hosts that have finished reporting; helps in determining whether data collection is complete.
      • expectedHosts: Number of hosts expected.
      • hostCounts: Array of arrays that contain the timestamp and number of hosts reported at that time.

Example (Python): Metrics API

The commented Python example shows how to make an HTTP request to the Pepperdata REST API resource for retrieving the Pepperdata dashboard metrics data.

# Pepperdata REST API Python example for retrieving metrics.
# Sections are denoted by '#' characters.

########################### IMPORTS ###########################

# Not required, but recommended for date and time manipulation.
# Required in this example.
import datetime

# To get this use the following: `pip install requests`
# If you do not have the pip Python package management system,
# install it from the python.org website:
# https://packaging.python.org/tutorials/installing-packages/
import requests
import os

# The authorization process (and therefore the required code imports) is
# different for recently-generated keys and legacy keys
# (keys generated before late August, 2017). Recently-generated
# keys are listed in the Active Keys section of the API Keys page
# (https://dashboard.pepperdata.com/account/apikeys); legacy keys are
# listed in the Legacy Keys section.
#
# **** IMPORTANT ****
#
# Be sure to use the correct commands (comment/uncomment the applicable
# lines) wherever you see "Active Key" or "Legacy Key" in the comments in
# this script.
#
# Legacy keys are DEPRECATED as of 2022 Oct 19. If you have one, generate a new 
# key at https://dashboard.pepperdata.com/account/apikeys and follow the 
# Active Key instructions. 
#

# *******************

# (Legacy Key) Required
#from requests.auth import HTTPBasicAuth


####################### DEFINE GLOBALS ########################

# Name of the your cluster/realm configured on Pepperdata
# **** IMPORTANT ****
#
# Before you run this Python example script, be sure to assign the correct
# realm name to the REALM_NAME environment variable by exporting an
# appropriate value for 'your-realm-name':
#
# export REALM_NAME='your-realm-name'
#
# *******************
REALM_NAME = os.environ['REALM_NAME']

# Base url where the Pepperdata dashboard REST API metrics
# resource for your realm is hosted
base_url = 'https://dashboard.pepperdata.com/{0}/api/m'.format(REALM_NAME)

# For Pepperdata-hosted dashboards (the default implementation),
# API access requires authentication via an API key.
# To obtain an API key, go to the following web page:
# https://dashboard.pepperdata.com/account/apikeys
#
# The "ID" is the portion of the API key before the ":"
# The password (secret) token is the portion of the API key after the ":"
# **** IMPORTANT ****
#
# Before you run this Python example script, be sure to assign the correct
# values to the API_KEY_ID and API_KEY_SECRET environment variables by
# exporting the correct values for 'your-api-key-id' and 'your-api-key-secret':
#
# export API_KEY_ID='your-api-key-id'
# export API_KEY_SECRET='your-api-key-secret'
#
# *******************
API_KEY_ID = os.environ['API_KEY_ID']
API_KEY_SECRET = os.environ['API_KEY_SECRET']

# The metrics API uses this format for time OR default Unix time.
pepperdata_time_format = "%Y/%m/%d-%H:%M"

# Although there is no maximum for the time range that you can specify,
# we recommend setting it to a maximum of 14 days.
# This example uses 1 day for the time range.
end_time = datetime.datetime.now()
start_time = end_time - datetime.timedelta(days=1)

# The distance between two data points in the returned time series in number of milliseconds.
sample_millis = 60000

# To determine the name of a metric on a chart, hover your mouse over the JSON
# export option link. Use the link's m={metric_url_name} parameter.
# The full list of metrics is on the Metrics Definitions page on the dashboard,
# http://dashboard.pepperdata.com/your-realm-name/metricdefinitions.
# This example use the "Active Tasks" metric, with m='tasks'.
metric_url_name = 'tasks'

# Time Zone, offset from UTC
time_zone_hour_offset = -7

# Output format: JSON or CSV. For fastest API performance, use JSON.
output_format = 'JSON'

# Maximum number of series (results) to return for the query
max_series = 20

# Assemble all the parameters to easily pass to the HTTP request portion.
# This example changes from the datetime format to the Pepperdata format
# by using strftime(). You could use Unix time instead.
pepperdata_params = {
  's': start_time.strftime(pepperdata_time_format),
  'e': end_time.strftime(pepperdata_time_format),
  'sample': sample_millis,
  'm': metric_url_name,
  'tzo': time_zone_hour_offset,
  'format': output_format,
  'ms': max_series
}

# To list the series by host/user/job/etc., add breakdowns to the parameters.
# This example breaks down the results by user ('u') and job ('j').
# For the complete list of parameters, refer to the following help page
# on your dashboard:
# https://dashboard.pepperdata.com/your-realm-name/metricdefinitions?print=1
pepperdata_params['u'] = '*'
pepperdata_params['j'] = '*'

# (Active Key) Required header
headers = {
  'Accept': 'application/json',
  'Authorization': 'PDAPI ' + API_KEY_ID + ":" + API_KEY_SECRET
}
# (Legacy Key) Required header
#headers = {'Accept': 'application/json'}


########### MAKE THE REQUEST AND PARSE THE RESPONSE ###########

# Send the request to the Pepperdata REST API metrics resource,
# passing the base url, all the parameters, the username and password,
# and the required header, which you defined previously.

# (Active Key) request:
try:
  response = requests.get(base_url, pepperdata_params,
                          headers=headers)
# (Legacy Key) request:
try:
#  response = requests.get(base_url, pepperdata_params, auth=HTTPBasicAuth(API_KEY_ID, API_KEY_SECRET),
#                          headers=headers)
except:
  print ("Error receiving response from server.")

# Parse the response as JSON, as specified by the output_format, above.
json_response = response.json()

# Store the data portion of the response. Everything else in the response
# is the progress updates as the database loaded the response.
json_data = json_response['data']

# Parse the data portion of the response, and store all the returned series
# in the all_series object.
all_series = json_data["allSeries"]

# Get the first point in the series, from the all_series object that
# stores the point elements as arrays of format [timestamp, data]. In this case,
# the point is [timestamp, active_tasks_at_timestamp]. The entirety of
# dataPoints is a list of these two element lists.
first_series_points = all_series[0]["dataPoints"]

# Get the object that contains the summary statistics of the series, including
# the following statistics:
# Area, Average, Minimum, Percentile 05, Maximum, and Percentile 95 over
# the entire series.
first_series_summary_stats = all_series[0]["summaryStats"]

# Get the unique series ID for this particular query.
first_series_id = all_series[0]["seriesId"]

# Get the map of name components for this series. If there are filters on
# this query, it will let you know that this series is a breakdown by a
# particular host and job, whose names are included here.
first_series_name_components = all_series[0]["name"]["seriesNameMap"]

# Print the output nicely.
for series in all_series:
    print "{0} {1}".format(series["summaryStats"]["avg"], series["seriesId"])