Thursday, September 2, 2021

Cisco telemetry pipeline recommendations? problems with big muddy pipeline :(

Problem: my telemetry stack I am using will not update the influxDB measurements to add new sensor paths no mater what I try..

My system: I was able to set up our Cisco IOS XR to send model driven telemetry data via gRPC to a telemetry stack I stood up. The stack is based off the git repo for big muddy telemetry, found here. Note: the the repo is archived, you need to change the branch to "final" to see code / issues / etc.

This stack includes some example subscriptions which I employed and do work. They uses these sensor paths:

 sensor-path Cisco-IOS-XR-wdsysmon-fd-oper:system-monitoring/cpu-utilization sensor-path Cisco-IOS-XR-nto-misc-oper:memory-summary/nodes/node/summary sensor-path Cisco-IOS-XR-fia-internal-tcam-oper:controller/dpa/nodes/node sensor-path Cisco-IOS-XR-procmem-oper:processes-memory/nodes/node/process-ids/process-id sensor-path Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters 

But I CANNOT for the life of me figure out how to get the pipeline not to drop any new entires I add to its metrics.json file. I tried adding the tcam yang file found here.

  • Checked to make sure Cisco registered the sensor path
  • Checked cisco to make sure its sending data for that path to my pipeline
  • Checked the pipeline dump file (no new entries for new sensor-paths I add)

The main issue is the big muddy pipeline is dropping any data I am sending to it based off the new JSON structure I created. It DOES preserve the original JSON structured data (CPU, Memory, interface info) but drops any new json structs I add for new yang modules I want to add. This is based on how the big muddy pipeline module uses the JSON format to parse through yang structured data. I have yet to find a automatic way of converting YANG to JSON, so I am forced to trial and error structure (mostly error).

Question 1: Has anyone else used the big muddy pipeline with success? How do I add new sensor paths / convert JSON for this system not to drop gRPC data being sent?

Question 2: Does anyone recommend some other type of gRPC pipeline gatherer besides the one I am using? I really dont want to use this anymore as its archived and not in development anymore, but its the only thing I found that works made by the Cisco team. I like how the system uses influxDB with Grafana, any recommendations for a pipeline that links to these services would be great.



No comments:

Post a Comment