Polling pfSense Over SNMP: How We Wired It Into Elasticsearch

1. Confirm SNMP is actually reachable before writing a single config line.

Don’t skip this — it isolates “is the network/daemon the problem” from “is my pipeline config the problem” before you’ve invested time in either. From the collector host:

$ snmpwalk -v2c -c <SNMP_COMMUNITY_STRING> -O n <pfsense-ip> .1.3.6.1.2.1.1.1.0
.1.3.6.1.2.1.1.1.0 = STRING: "pfSense pfSense.home.arpa 2.7.2-RELEASE FreeBSD 14.0-CURRENT amd64"

If that hangs or times out, the fix is on pfSense’s side first: Services → SNMP — confirm the daemon is enabled, bound to the LAN interface (and IPv4 if you’re polling by IPv4), and that a community string is actually set. Nothing downstream matters until this one command returns cleanly.

2. Install the collector.

We’re using Logstash rather than a Fleet/Metricbeat agent (see above for why). On the collector host:

$ sudo apt-get install logstash

Pin the version to match your Elasticsearch cluster’s major version to avoid client/server compatibility surprises.

3. Write one pipeline config per concern.

Each .conf file in /etc/logstash/conf.d/ follows the same three-block shape — input (one or more snmp blocks, using either get for specific OIDs or walk for a subtree), filter (enrichment — tagging, unit conversion, derived fields), output (where it lands in Elasticsearch):

input {
  snmp {
    hosts => [{
      host      => "udp:<pfsense-ip>/161"
      community => "<SNMP_COMMUNITY_STRING>"
      version   => "2c"
    }]
    interval => 60
    get => [ "1.3.6.1.2.1.1.1.0", "1.3.6.1.2.1.1.3.0" ]   # sysDescr, sysUpTime
    use_provided_mibs => true
    ecs_compatibility  => "v8"
  }
}

filter {
  mutate {
    add_field => {
      "event.dataset"    => "pfsense.snmp"
      "observer.type"    => "firewall"
      "observer.vendor"  => "Netgate"
      "observer.product" => "pfSense"
    }
  }
}

output {
  elasticsearch {
    hosts                  => ["https://<es-host>:9200"]
    api_key                => "<ELASTIC_API_KEY>"
    ssl_enabled            => true
    data_stream            => true
    data_stream_type       => "metrics"
    data_stream_dataset    => "pfsense.snmp"
    data_stream_namespace  => "default"
  }
}

Use api_key from the start, not a superuser user/password pair — it’s scoped, it doesn’t silently expire the way a rotated superuser password will, and it’s the same auth pattern the rest of the stack already uses. A plaintext password baked into a config file is a future outage waiting for a routine credential rotation to trigger it.

4. Register each pipeline in pipelines.yml.

Logstash doesn’t auto-discover conf.d/*.conf files as independent pipelines by default — each one needs an explicit entry:

- pipeline.id: pfsense-snmp-main
  path.config: "/etc/logstash/conf.d/pfsense-snmp.conf"

One entry per .conf file, one line each. This is also what makes systemctl status logstash and journalctl -u logstash report per-pipeline, rather than one undifferentiated blob.

5. Attach retention up front.

Don’t let a new metrics data stream default to unmanaged. Either point it at an existing ILM policy that matches your retention needs, or write a dedicated one, before the first document lands — it’s a one-line addition to the output block’s index template at creation time, and a much bigger cleanup exercise if you add it after the stream already exists unmanaged.

6. Restart and verify from the outside in.

$ sudo systemctl restart logstash
$ sudo journalctl -u logstash --since "1 minute ago" -f

Look for Connected to ES instance and Starting pipeline with no ConfigurationError or repeated Scheduled restart job lines. Then confirm end-to-end in Kibana Discover against the new data stream — a clean systemd status is necessary but not sufficient; the only real proof is a document with a timestamp from the last minute.