`trapperkeeper-metrics` Documentation

What This Library Does

The main purpose of this library is to provide some wrapper code around the Dropwizard Metrics Java Library, but there is some other functionality provided as well. Here are the major features available in trapperkeeper-metrics:

A Trapperkeeper service that handles life cycle management for objects from the Dropwizard Metrics library
Support for reading basic metrics configuration info from a Trapperkeeper config file, so that the metrics configuration syntax is consistent across all of your Trapperkeeper apps
Other utility functions for creating and interacting with metrics in your application

For more detail on these features, read on.

`MetricRegistry` Life Cycle

The main entry point into the Dropwizard Metrics API is a class called MetricRegistry.

This class requires some basic initialization, so trapperkeeper-metrics provides a Trapperkeeper service (MetricsService) that manages the life cycle of a MetricRegistry. The service includes a function, get-metrics-registry, so that all of your other Trapperkeeper services can access the registry and register new metrics with it.

For example:

(defservice my-service
  [[:MetricsService get-metrics-registry]]
  (init [this context]
    (let [metrics-registry (get-metrics-registry)
          my-metric-name (metrics/host-metric-name "localhost" "my-metric")
          my-timer (.timer metrics-registry my-metric-name)]
      (metrics/time! my-timer (do-some-work)))
    context))

See the source code for the sample app for a working example. See the utility functions in the trapperkeeper-metrics puppetlabs.metrics namespace for some helpers for constructing other kinds of metrics besides just Timer. See the Dropwizard Metrics docs for more info about all of the available metrics types and their features.

The get-metrics-registry also allows you to specify two additional fields, registry-key and domain which allow you to create other registries (besides the default given by (get-metrics-registry) and allow you to configure the namespace of the reporter for that registry.

(defservice my-service
  [[:MetricsService get-metrics-registry]]
  (init [this context]
    (let [default-metrics-registry (get-metrics-registry)
          my-metrics-registry (get-metrics-registry "my.metrics.domain")
          ;; This will create the metric
          ;; `my.metrics.domain:name=puppetlabs.localhost.my-metric`
          my-metric-name (metrics/host-metric-name "localhost" "my-metric")
          my-timer (.timer metrics-registry my-metric-name)]
      (assert (not= default-metrics-registry my-metrics-registry))
      (metrics/time! my-timer (do-some-work)))
    context)

  (start [this context]
    ;; We can retrieve the same metrics-registry later.
    (let [my-metrics-registry (get-metrics-registry "my.metrics.domain")]
      (do-some-other-work my-metrics-registry))
    context))

For more specific details on the service API, see the Metrics Service API docs.

Configuration & Reporters

The MetricsService also provides a configuration syntax that users can use to configure the metrics for a running TK app. This means that all TK services can provide a consistent interface for interacting with metrics.

For more specific details, see the MetricsService configuration docs.

Utility Functions

The puppetlabs.metrics namespace contains some utility functions for working with metrics. See the source code and docstrings for more detail. Here are a few bits of basic info:

time! is a macro that can be used to time some Clojure forms against an existing Timer. e.g.:

(let [my-timer (.timer (get-metrics-registry) "my.metric.name")]
    (time! my-timer
      (do-some-work!)
      (do-some-more-work!)))

host-metric-name can be used to provide a qualified, namespaced metric name. For best results, it is advisable to use this function to create a name for each of your application's metrics; this will ensure that the metrics are namespaced consistently across services. It will also ensure that metrics are namespaced by hostname, which is critical when consolidating metrics data from multiple hosts/services. You can use the server-id value from the MetricsService configuration to get the appropriate hostname for your metric.

(TODO: this part of the API needs to be fleshed out a bit further. We might want to have a more dynamic way to get the hostname/server-id rather than having to put it into the config file. We may want to tweak some other things as well. In the interim, it's probably a good idea to make it clear in your application/service documentation that the specific metric namespaces should not be considered part of a formal API and may change in subsequent releases.)

register can be used to add a metric to an existing MetricRegistry.

ratio, metered-ratio, and gauge can be used to construct other types of Metrics.

Low-level HTTP API

To enable the HTTP API for accessing individual metrics add the service to your bootstrap.cfg and configure the web-router-service accordingly:

web-router-service {
  "puppetlabs.trapperkeeper.services.metrics.metrics-service/metrics-webservice" : "/metrics"
}

Listing available metrics

Request format

To get a list of all available metric names:

Request /metrics/v1/mbeans.
Use a GET request.

Response format

Responses return a JSON object mapping a string to a string:

The key is the name of a valid MBean.
The value is a URI to use for requesting that MBean's attributes.

Retrieving multiple metrics

Request format

To get a the attributes for multiple metrics at the same time:

Request /metrics/v1/mbeans.
Use a POST request.
Use a request body which is either a JSON object whose values are metric names, JSON array of metric names or a JSON string of a metric name.

Response format

The response format, though always JSON, depends on the request format:

Requests with a JSON object will return the a JSON object where the values of the original object have been transformed into the Mbeans' attributes for the metric names.
Requests with a JSON array will return the a JSON array where the items of the original array have been transformed into the Mbeans' attributes for the metric names.
Requests with a JSON string will return the a JSON object of the Mbean's attributes for the given metric name.

Retrieving an specific metric

Request format

To get the attributes of a particular metric:

Request /metrics/v1/mbeans/<name>, where <name> is something that was returned in the list of available metrics specified above.
Use a GET request.

Response format

Responses return a JSON object mapping strings to (strings/numbers/Booleans).

For example, using curl from localhost:

curl 'https://localhost:8080/metrics/v1/mbeans/java.lang:type=Memory'
{
  "ObjectPendingFinalizationCount" : 0,
  "HeapMemoryUsage" : {
    "committed" : 807403520,
    "init" : 268435456,
    "max" : 3817865216,
    "used" : 129257096
  },
  "NonHeapMemoryUsage" : {
    "committed" : 85590016,
    "init" : 24576000,
    "max" : 184549376,
    "used" : 85364904
  },
  "Verbose" : false,
  "ObjectName" : "java.lang:type=Memory"
}

Alternatives

Since we support sending the metrics data to JMX, there are several existing tools and approaches that can be used to read the data for individual metrics via JMX. JVisualVM is one example; see pe-puppetserver-jruby-jmx-client for an example of how to do this from a JRuby script.

What This Library Does Not Do

Notes for Developers

For best results, use this library in combination with the comidi library, and then take advantage of the wrap-with-request-metrics Ring middleware to track metrics about all HTTP requests made to your application. See the comidi docs for more info.

In The Future There Will Be Robots

Some ideas for things we might want to add/change in the future:

metrics-clojure

There is an existing clojure library that wraps Dropwizard Metrics: metrics-clojure. At the time when we originally wrote our metrics code, this library was very out-of-date and didn't support some of the features we needed. It also seemed like a pretty thin facade around the Java library, and we decided that it wasn't worth adding an extra dependency.

Since then, it's been updated and should be much more compatible with the more recent versions of Dropwizard Metrics. At a glance, it seems like it would be possible for other TK services to use metrics-clojure in combination with trapperkeeper-metrics as-is; it mostly just provides utility functions that should work fine with the MetricRegistry object surfaced by trapperkeeper-metrics. So, if you feel like the abstractions it provides over the Java library are worthwhile, try it out and let us know if something doesn't work properly.

At some point in the future we may go ahead and add it as a direct dependency and refactor things in trapperkeeper-metrics to use it, but so far there hasn't been a hugely compelling reason to do so.

Additional Facilities for Exposing Metrics

There are a few other ideas floating around for how to make it easier for apps/services that are using trapperkeeper-metrics to expose the metrics info to end users. More utility functions for easier integration with status service, other tools for facilitating consumption / visualization / etc. Stay tuned.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!