Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,58 @@ visor:
type: dns
```

#### Policy attributes

A policy can have an `attributes` mapping, next to `kind`, `input`, `handlers` and `config`. Every series of every handler
of the policy carries these attributes, in the OpenTelemetry (OTLP) push and in the Prometheus output. The JSON output
does not carry them.

```yaml
visor:
policies:
mysocket:
kind: collection
attributes:
netbox.site.name: dc-west
netbox.tenant.name: acme
netbox_id: "007"
input:
tap: unix_dnstap
input_type: dnstap
handlers:
modules:
default_dns:
type: dns
```

* A value is a scalar, kept as the text written: `"007"` stays `007`. Quote a value whose exact text matters. A map, a list, a
null or an empty value is refused. The one exception is a backslash followed by three digits, which is rewritten as `\092`
and those digits (`C:\123` becomes `C:\092123`), as in every label value pktvisor exports.
* A key starts with an ASCII letter and is valid UTF-8; an empty key, or one that starts with a digit, an underscore or any
other character (`__name__` included), is refused. A null `attributes` is the same as none.
* OTLP carries each key as written. Prometheus writes each key with every character outside `[A-Za-z0-9_]` replaced by `_`,
one `_` per character, so `netbox.site.name` becomes `netbox_site_name` and `sité` becomes `sit_`.
* The policy does not load if two keys have the same Prometheus name (`netbox.site.name` and `netbox_site_name`).
* The policy does not load if a key is, or becomes after that conversion, a label pktvisor sets on the policy's series: `policy`,
`handler`, `tap`, `le`, `quantile`, `instance` when `--prom-instance` (or `prom_instance` in the configuration file) is set, or a label of one of the policy's handlers.
The labels of each handler are:

| Handler | Own labels |
|---|---|
| `net` v1 | `lat`, `lon`, `geo_loc`, `asn`, `ipv4`, `ipv6` |
| `net` v2 | `direction`, `lat`, `lon`, `geo_loc`, `asn`, `ipv4`, `ipv6` |
| `dns` v1 | `lat`, `lon`, `geo_loc`, `asn`, `ecs`, `qname`, `port`, `qtype`, `rcode` |
| `dns` v2 | `direction`, `lat`, `lon`, `geo_loc`, `asn`, `ecs`, `qname`, `port`, `qtype`, `rcode` |
| `dhcp` | `client`, `server` |
| `flow` | `device`, `device_interface`, `lat`, `lon`, `conversation`, `geo_loc`, `asn`, `ip`, `port`, `ip_port`, `dscp`, `ecn`, `interface` |
| `netprobe` | `target`, `status_code`, `rcode` |
| `pcap`, `bgp`, `input_resources` | none |

* The `netprobe` handler also takes attributes per target, see its [README](src/handlers/netprobe/README.md).
* A pktvisor version without this feature ignores the policy's own `attributes` and runs the policy without them. It reads
per-target `attributes` as input config: an all-digit value past 2^64 makes it refuse the policy, and policies that differ
only in their target attributes get separate inputs.

If running in a Docker container, you must mount the configuration file into the container. For example, if the configuration file
is on the host at `/local/pktvisor/agent.yaml`, you can mount it into the container and use it with this command:

Expand Down
22 changes: 22 additions & 0 deletions RFCs/2021-04-16-76-collection-policies.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,28 @@ explicit support for the handler type upstream to them, otherwise the policy wil

Policies require a `kind` to indicate the type of policy being applied.

A policy may also have an `attributes` mapping at the top level, next to `kind`, `input`, `handlers` and `config`. Each
entry is added as a label (an attribute, in OTLP) to every series of every handler of the policy, in the OTLP push and in
the Prometheus output:

```yaml
anycast_dns:
kind: collection
attributes:
netbox.site.name: dc-west
netbox_id: "007"
```

Values are scalars kept as the text written (a backslash followed by three digits is rewritten, as in every exported label
value). A map, list, null or empty value is refused. A key starts with an ASCII letter and is valid UTF-8; a null
`attributes` is none. OTLP uses the keys as written; Prometheus replaces every character outside `[A-Za-z0-9_]` in a key
with `_`, one per character. A policy does not load when two keys share a
Prometheus name, or when a key is, or becomes, a label pktvisor sets itself (`policy`, `handler`, `tap`, `le`, `quantile`,
`instance` with `--prom-instance` or `prom_instance`, and the labels of the policy's handlers). The `netprobe` handler also accepts
`attributes` per target under `input.config.targets.<name>.attributes`; see its README. A pktvisor that does not know
`attributes` ignores the policy's own, but reads per-target attributes as input config: an all-digit value past 2^64 makes it
refuse the policy, and policies that differ only in their target attributes get separate inputs.

NOTE: this is not yet a complete, formal specification, and not all of it is implemented. See
src/tests/test_policies.cpp for current, tested implementation.

Expand Down
9 changes: 9 additions & 0 deletions src/AbstractMetricsManager.h
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,7 @@ class AbstractMetricsManager
_metric_buckets[0]->configure_groups(_groups);
_metric_buckets[0]->set_start_tstamp(stamp);
_metric_buckets[0]->update_topn_metrics(_topn_count, _topn_percentile_threshold);
on_new_bucket(*_metric_buckets[0]);
if (_recorded_stream) {
_metric_buckets[0]->set_recorded_stream();
}
Expand Down Expand Up @@ -391,6 +392,14 @@ class AbstractMetricsManager
{
}

/**
* call back for the live bucket a period shift opens, made under the bucket lock before any event or export reaches
* it. The first bucket is made in the constructor, before a derived class exists, so this is not called for it.
*/
virtual void on_new_bucket([[maybe_unused]] MetricsBucketClass &bucket)
{
}

public:
AbstractMetricsManager(const Configurable *window_config)
: _metric_buckets{}
Expand Down
2 changes: 2 additions & 0 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ add_library(visor-core
CoreRegistry.cpp
Metrics.cpp
Policies.cpp
PolicyAttributes.cpp
ThreadName.cpp
IpPort.cpp
Taps.cpp)
Expand Down Expand Up @@ -89,6 +90,7 @@ add_executable(unit-tests-visor-core
tests/test_ipport.cpp
tests/test_taps.cpp
tests/test_policies.cpp
tests/test_policy_attributes.cpp
tests/test_handlers.cpp
tests/test_module_plugins.cpp
tests/test_prometheus_serializer.cpp
Expand Down
5 changes: 4 additions & 1 deletion src/CoreServer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -376,11 +376,14 @@ void get_default_prometheus(PolicyManager &policies, httplib::Response &res)
return;
}
auto logger = spdlog::get("visor");
// the legacy shape: the policy's attributes and policy, no handler label
auto labels = policy->attributes();
labels["policy"] = "default";
for (auto &mod : policy->modules()) {
auto hmod = dynamic_cast<StreamHandler *>(mod);
if (hmod) {
spdlog::stopwatch sw;
hmod->window_prometheus(ser, {{"policy", "default"}});
hmod->window_prometheus(ser, labels);
logger->debug("{} window_prometheus elapsed time: {}", hmod->name(), sw);
}
}
Expand Down
22 changes: 22 additions & 0 deletions src/MetricLabels.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,28 @@ bool is_utf8(std::string_view value);
bool label_value_needs_escape(std::string_view value);
std::string utf8_label_value(std::string_view value);

// The Prometheus label name of a label key: every character outside [A-Za-z0-9_] becomes _, one _ per
// code point (the bytes after the first of a UTF-8 sequence write nothing), as the OTLP to Prometheus
// translators do. append_prometheus_label_name writes it onto `out` without a temporary string.
inline void append_prometheus_label_name(std::string &out, std::string_view key)
{
for (unsigned char c : key) {
if ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '_') {
out.push_back(static_cast<char>(c));
} else if ((c & 0xC0) != 0x80) {
out.push_back('_');
}
}
}

inline std::string prometheus_label_name(std::string_view key)
{
std::string name;
name.reserve(key.size());
append_prometheus_label_name(name, key);
return name;
}

// Process-global static labels (e.g. instance). Storage defined in Metrics.cpp.
LabelMap &prometheus_static_labels_mutable();
inline const LabelMap &prometheus_static_labels() { return prometheus_static_labels_mutable(); }
Expand Down
34 changes: 28 additions & 6 deletions src/Policies.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include "CoreRegistry.h"
#include "HandlerManager.h"
#include "InputStreamManager.h"
#include "PolicyAttributes.h"
#include "PrometheusSerializer.h"
#include "Taps.h"
#include <algorithm>
Expand Down Expand Up @@ -55,8 +56,12 @@ std::vector<Policy *> PolicyManager::load(const YAML::Node &policy_yaml, bool si
std::unique_lock lock(_load_mutex);

auto policy_name = _get_policy_name(it);
// Input Section
auto input_node = it->second["input"];
// Input Section. The attributes of each netprobe target belong to the policy, not to the input: they are
// read from the YAML as written, like the policy's own, and the input is built and named from a copy
// without them, so they are never typed as config values and do not split one input in two
const auto raw_input_node = it->second["input"];
auto target_attributes = std::make_shared<const TargetAttributes>(parse_target_attributes(raw_input_node, policy_name));
auto input_node = without_target_attributes(raw_input_node);
auto [input_config, input_filter] = _registry->input_manager()->get_config_and_filter(input_node);
// Handler Default Section
auto handler_node = it->second["handlers"];
Expand All @@ -81,6 +86,10 @@ std::vector<Policy *> PolicyManager::load(const YAML::Node &policy_yaml, bool si
}
}

if (it->second["attributes"]) {
policy_ptr->set_attributes(parse_attributes(it->second["attributes"], fmt::format("policy '{}'", policy_name)));
}

std::vector<std::string> added_inputs;
std::vector<std::string> added_handlers;
// by name, like the resources policies below: the rollback looks each one up again
Expand Down Expand Up @@ -184,7 +193,20 @@ std::vector<Policy *> PolicyManager::load(const YAML::Node &policy_yaml, bool si
handler_modules.emplace_back(std::move(handler_module));
}

// the labels pktvisor sets itself, so an attribute cannot replace one: the policy's own, the static
// ones, and those of every handler it runs. A refusal rolls the load back like any other failure
std::vector<std::string> handler_labels;
for (const auto &m : handler_modules) {
auto names = m->own_label_names();
handler_labels.insert(handler_labels.end(), names.begin(), names.end());
}
auto reserved = reserved_label_names(handler_labels);
check_reserved(policy_ptr->attributes(), reserved, fmt::format("policy '{}'", policy_name));
check_target_attributes(*target_attributes, policy_ptr->attributes(), reserved, policy_name);

for (auto &m : handler_modules) {
// before the handler starts, and never changed after
m->set_target_attributes(target_attributes);
auto hname = m->name();
_registry->handler_manager()->module_add(std::move(m));
added_handlers.push_back(hname);
Expand Down Expand Up @@ -494,14 +516,14 @@ void Policy::prometheus_metrics(PrometheusSerializer &ser)
if (_merge_like_handlers) {
auto bucket_map = _get_merged_buckets();
for (auto &[bucket, hmod] : bucket_map) {
hmod->window_prometheus(ser, bucket.get(), {{"policy", name()}, {"handler", hmod->schema_key() + "_merged"}});
hmod->window_prometheus(ser, bucket.get(), series_labels(hmod->schema_key() + "_merged"));
}
} else {
for (auto &mod : modules()) {
auto hmod = dynamic_cast<StreamHandler *>(mod);
if (hmod) {
spdlog::stopwatch sw;
hmod->window_prometheus(ser, {{"policy", name()}, {"handler", hmod->name()}});
hmod->window_prometheus(ser, series_labels(hmod->name()));
spdlog::get("visor")->debug("{} window_prometheus elapsed time: {}", hmod->name(), sw);
}
}
Expand Down Expand Up @@ -575,7 +597,7 @@ void Policy::_merged_opentelemetry(metrics::v1::ScopeMetrics &scope)
auto metrics_before = scope.metrics_size();
try {
for (auto &[bucket, hmod] : bucket_map) {
hmod->window_opentelemetry(scope, bucket.get(), {{"policy", name()}, {"handler", hmod->schema_key() + "_merged"}});
hmod->window_opentelemetry(scope, bucket.get(), series_labels(hmod->schema_key() + "_merged"));
}
} catch (...) {
scope.mutable_metrics()->DeleteSubrange(metrics_before, scope.metrics_size() - metrics_before);
Expand Down Expand Up @@ -627,7 +649,7 @@ void Policy::opentelemetry_metrics(metrics::v1::ScopeMetrics &scope)
// it fails what it wrote is dropped and sent whole next time.
auto metrics_before = scope.metrics_size();
try {
hmod->window_opentelemetry(scope, {{"policy", name()}, {"handler", hmod->name()}});
hmod->window_opentelemetry(scope, series_labels(hmod->name()));
} catch (...) {
scope.mutable_metrics()->DeleteSubrange(metrics_before, scope.metrics_size() - metrics_before);
throw;
Expand Down
24 changes: 24 additions & 0 deletions src/Policies.h
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#include "HandlerModulePlugin.h"
#include "InputModulePlugin.h"
#include "OpenTelemetry.h"
#include "PolicyAttributes.h"
#include "Taps.h"
#include <chrono>
#include <ctime>
Expand Down Expand Up @@ -65,6 +66,9 @@ class Policy : public AbstractRunnableModule
BucketMap _get_merged_buckets(const std::function<std::unique_ptr<AbstractMetricsBucket>(StreamHandler *, AbstractMetricsBucket *)> &merge);
void _merged_opentelemetry(metrics::v1::ScopeMetrics &scope);

// set on every series the policy's handlers produce, under the labels pktvisor sets itself
LabelMap _attributes;

public:
Policy(const std::string &name)
: AbstractRunnableModule(name)
Expand All @@ -76,6 +80,26 @@ class Policy : public AbstractRunnableModule
return "policy";
}

void set_attributes(LabelMap attrs)
{
_attributes = std::move(attrs);
}

const LabelMap &attributes() const
{
return _attributes;
}

// The labels of every series of this policy's handler: the policy's attributes, then
// `policy` and `handler` on top.
LabelMap series_labels(const std::string &handler_label) const
{
auto labels = _attributes;
labels["policy"] = name();
labels["handler"] = handler_label;
return labels;
}

void set_modules_sequence(bool sequence)
{
_modules_sequence = sequence;
Expand Down
Loading
Loading