Chris Fowler
2017-01-25 21:25:20 UTC
I'm tying to do something that works on a smaller scale, but on a
larger scale keeping up with MIB syntax errors and such is a problem.
I have a device that has 25M of MIB files. snmptranslate takes too
long to process through '+ALL'. I've used the SNMP Perl module so I
wrote a translation service using SOAP for my applications.
I like it. It uses SNMP::initMib() to go after the 25 and I use
smidump, etc to get extra data for the API. Things like
'LAST-UPDATED'. It also parses traps out and stores that data for
callers.
As I receive zip files with vendor MIBs I add this to the repository,
but I want to cache 200M worth of data on another system. I'll modify
my service on the device to cache when OIDs are seen from traps, but
it can not translate. I've ported the service over to the server.
Here is where I've ran into problems:
I've downloaded a few repositories of MIBS off github. LibreNMS is
one such repository.
Net-SNMP MIBs and the RFC oness generated in Net-SNMP build are stored
in /usr/share/snmp/mibs.
The ones I've added to the device are in
/usr/share/snmp/private
The large group is in
/usr/share/snmp/extra
MIBDIRS on those and MIBS=+ALL will be successful for the first 2, but
the last one has many errors. My line of thinking is that if
LibreNMS uses those are they running into the same thing? If not,
maybe I'm doing something wrong.
Management is an issue that I also need to tackle. "Techs" that "want
to monitor" will provide zips of files they do not know. "These are
the MIBs". I may have those already. I used smidump to get the last
revision date so I could maintain the newest versions of these files.
I use a script to rename the files to the name of the MIB MODULE
contained within. To copy a MIB to one of those directories it must
be pre-processed.
The service works. I like it and it seems to work well. I know that
creating a translation service is not impossible. I don't understand
MIBs within this system enough to know if using a service for
translation is really possible. Perl's SNMP with +ALL does not
provide a way to sort some of this out so I'm doing it as prep work.
I've tried a few tricks to stop using snmptranslate when parsing
traps. I want the labels in log files for the techs (not OIDs). The
device that has 25M will consume 7-9s of sys time before even
receiving a response from snmptranslate. If I'm getting traps every 5s
I'll never catch up. Using the service it is < 1s to get an answer.
Only one &SNMP::initMib() has to be executed at boot vs one for each
trap!
I've tried a walk through $SNMP::MIB and storing the values in a db
file. The result is 19M, large, and the reality is the the MIBs are
the database. I've tried caching the results of a translate in a DB
file. This works great until you run into .N where N may be an
interface, etc and can be 1-1000. The DB file and the caching of past
translations in a db file did not scale well. I always came back to
$SNMP::MIB. I thought maybe I could Storable::n_freeze(SNMP::MIB) to
disk and use that instead. The only time the MIB files change is when
I regenerate a new image on the device. I'm sure that hash is a tied
hash to the Net-SNMP C API and can't be stored.
I'd love to drop my devices down to just the Net-SNMP MIBs plus RFC,
program their service to send OIDs upstream for translation and cache
those locally till next boot. I can write that. I'm not sure if it
is really possible with all these MIBs I collect from all over.
Ideas and suggestions on this issue would be greatly appreciated,
Chris
larger scale keeping up with MIB syntax errors and such is a problem.
I have a device that has 25M of MIB files. snmptranslate takes too
long to process through '+ALL'. I've used the SNMP Perl module so I
wrote a translation service using SOAP for my applications.
I like it. It uses SNMP::initMib() to go after the 25 and I use
smidump, etc to get extra data for the API. Things like
'LAST-UPDATED'. It also parses traps out and stores that data for
callers.
As I receive zip files with vendor MIBs I add this to the repository,
but I want to cache 200M worth of data on another system. I'll modify
my service on the device to cache when OIDs are seen from traps, but
it can not translate. I've ported the service over to the server.
Here is where I've ran into problems:
I've downloaded a few repositories of MIBS off github. LibreNMS is
one such repository.
Net-SNMP MIBs and the RFC oness generated in Net-SNMP build are stored
in /usr/share/snmp/mibs.
The ones I've added to the device are in
/usr/share/snmp/private
The large group is in
/usr/share/snmp/extra
MIBDIRS on those and MIBS=+ALL will be successful for the first 2, but
the last one has many errors. My line of thinking is that if
LibreNMS uses those are they running into the same thing? If not,
maybe I'm doing something wrong.
Management is an issue that I also need to tackle. "Techs" that "want
to monitor" will provide zips of files they do not know. "These are
the MIBs". I may have those already. I used smidump to get the last
revision date so I could maintain the newest versions of these files.
I use a script to rename the files to the name of the MIB MODULE
contained within. To copy a MIB to one of those directories it must
be pre-processed.
The service works. I like it and it seems to work well. I know that
creating a translation service is not impossible. I don't understand
MIBs within this system enough to know if using a service for
translation is really possible. Perl's SNMP with +ALL does not
provide a way to sort some of this out so I'm doing it as prep work.
I've tried a few tricks to stop using snmptranslate when parsing
traps. I want the labels in log files for the techs (not OIDs). The
device that has 25M will consume 7-9s of sys time before even
receiving a response from snmptranslate. If I'm getting traps every 5s
I'll never catch up. Using the service it is < 1s to get an answer.
Only one &SNMP::initMib() has to be executed at boot vs one for each
trap!
I've tried a walk through $SNMP::MIB and storing the values in a db
file. The result is 19M, large, and the reality is the the MIBs are
the database. I've tried caching the results of a translate in a DB
file. This works great until you run into .N where N may be an
interface, etc and can be 1-1000. The DB file and the caching of past
translations in a db file did not scale well. I always came back to
$SNMP::MIB. I thought maybe I could Storable::n_freeze(SNMP::MIB) to
disk and use that instead. The only time the MIB files change is when
I regenerate a new image on the device. I'm sure that hash is a tied
hash to the Net-SNMP C API and can't be stored.
I'd love to drop my devices down to just the Net-SNMP MIBs plus RFC,
program their service to send OIDs upstream for translation and cache
those locally till next boot. I can write that. I'm not sure if it
is really possible with all these MIBs I collect from all over.
Ideas and suggestions on this issue would be greatly appreciated,
Chris