Configuration & Administration

Expand all | Collapse all

snmp trap email missing event details

  • 1.  snmp trap email missing event details

    Posted 29 days ago

    Running 6.3.2 core.

    This works : 
    Should a remote device trigger an alert  (ex disk latency), Zenoss does get the snmp trap.
    An e-mail is generated, indicating an issue.

    But this happens

    However, a few minutes later, once latency goes away, another snmp message comes in as "Clear", increasing the Count by 1; and I never get to see the original message. Almost like snmp is overwritting its own events.

    What I've tried
    In event classes, Configuration properties, I tried changing the zEventAction to all values, but either no event is logged, or the event is over-written.

    I read another post, where if I can one of the fields unique, it won't overwrite itself, but using the Transforms and Examples Wiki, my alerting is just failing.

    Question # 1

    Is it possible to get information from Event Details, and have this sent out via e-mail ?  Can an example be provided ?

    1.3.6.1.2.1.1.3.0
    3169664790
    all
    \nDeviceClass: /CBS/Appliances\nDeviceGroups: None\nDevicePriority: 3\nLocation: None\nSystems: None\nagent: zentrap\ncomponent: \ncreated: 1600897498\ndedupid: None\ndevice: 111.111.111.111\neventClass: /Status/Snmp\neventClassKey: vmwVrops.1.0.46\neventGroup: trap\neventKey: \neventState: 0\nevid: 08d0435e-fde6-11ea-8f55-0242ac110013\nfacility: None\nflappingInterval: None\nflappingSeverity: None\nflappingThreshold: None\nipAddress: 111.111.111.111\nmessage: snmp trap vmwVrops.1.0.46\nmonitor: localhost\nntevid: None\npriority: None\nprodState: 1000\nseverity: 3\nsummary: snmp trap vmwVrops.1.0.46\ntags:
    community
    zenoss
    eventClassMapping
    manager
    ea44b7dd2c0f
    oid
    1.3.6.1.4.1.6876.4.50.1.0.46
    snmpVersion
    2
    vmwVrops.1.2.1.0
    vRealizeClusterNode
    vmwVrops.1.2.10.0
    Virtualization/Hypervisor
    vmwVrops.1.2.11.0
    Performance
    vmwVrops.1.2.12.0
    critical
    vmwVrops.1.2.13.0
    info
    vmwVrops.1.2.14.0
    info
    vmwVrops.1.2.15.0
    CPU|Usage
    vmwVrops.1.2.16.0
    VirtualMachine
    vmwVrops.1.2.17.0
    Virtual machine CPU usage is at 100% for an extended period of time
    vmwVrops.1.2.18.0
    Virtual machine CPU usage is at 100% for an extended period of time
    vmwVrops.1.2.19.0
    health
    vmwVrops.1.2.2.0
    NAME_OF_PC_WITH_A_PROBLEM

    Question # 2
    Is there an example, where eventClassKey gets converted to the value of eventclasskey_year_month_day_hour_minute_second, so that duplicates for issues regardign CPU usage does not get overwritten ?




    ------------------------------
    richard
    ------------------------------


  • 2.  RE: snmp trap email missing event details

    Posted 28 days ago
    Hi Richard,
    You might go and get my paper on   Zenoss Event Management  that describes the event subsystem architecture and has lots of examples.  It was written for Zenoss 4 but nothing significant has changed in the event architecture with Zenoss 6.

    I can believe that your device sends in a "bad news" trap and that it might send a "good news" trap when the issue goes away.  What intrigues me is that you say the second event comes in as "clear" but gets added to the repeat count?

    Zenoss's built-in de-duplication mechanism considers 5 fields when it is deciding whether an event is a "duplicate" - device, component, eventClass, eventKey (NOT eventClassKey) and severity; if eventKey is null, then the summary field will also be used.  So if your incoming traps have different severities then they should not be duplicates.

    That said,you don't actually include the Zenoss severity in the details of your event above.  Can you check that for both your "bad news" and "good news" events?  I am guessing that the vmwVrops.1.2.12.0 attribute might be a severity as one of the SNMP varbinds?  But Zenoss won't automatically be able to pick that varbind out to populate it's own severity attribute.

    There obviously HAS been some processing of the TRAP as you show an eventClassMapping field of vmwVrops.1.0.46 . The eventClassMapping is the logic that converts an incoming TRAP into a Zenoss event - you may have a ZenPack installed that has provided these mappings for you?  Fundamentally an eventClassMapping is used to convert alarms generated outside Zenoss into Zenoss events, starting with the eventClassKey field to determine the eventClass (and all those similar names are VERY confusing for the uninitiated :(  )  Both an eventClass and an eventClassMapping may have a transform - Python code that looks at the incoming alert and picks out different "bits" to populate either standard event attributes or you are perfectly at liberty to create your own event fields.

    The other built-in trick of Zenoss's event system is the auto-clear fingerprint - a mechanism that automatically closes "bad news" events if a "good news" event arrives.  The Zenoss event fields that have to match here are device, component, eventClass and eventKey; if an incoming event has severity of 0 (clear)  (which may have been set by a transform) then it will clear (strictly the status will be cleared, not closed) all events whose auto-clear fingerprint match.

    Soooo....
    Your trick is probably to change the severity of the incoming TRAPs.  I suspect you will find their Zenoss severity is the same so they become duplicates.

    You should find quite a few examples in the paper I referenced.

    Cheers,
    Jane

    ------------------------------
    Jane Curry
    Skills 1st United Kingdom
    jane.curry@skills-1st.co.uk
    ------------------------------



  • 3.  RE: snmp trap email missing event details

    Posted 28 days ago
    Edited by richard martin 27 days ago

    Thanks Jane

    Knowing about Good & Bad traps, to clarify a bit, nothing is clearing the Zenoss events via Clear method.

    What happens, is vmware will send an alert about a disk full, If I'm fast enough, I can see the trap.
    Later a cpu gets busy, and this same "event" comes into Zenoss, with the same footprint as the disk full.  This new event overwrites the disk full log entry.


    I'm trying to find a way to keep all bad news events, or send the data of bad news events to e-mail.

    I will go through the link you sent me as well. Thanks .



    ------------------------------
    richard
    ------------------------------



  • 4.  RE: snmp trap email missing event details

    Posted 25 days ago
    Richard,

    It sounds like the good and bad traps are running through the same event class mapping.  You could fix this by adding a transform to the mapping to set your good traps to the clear severity.  This will prevent them from deduping and will instead leave the bad trap event in place, setting its status to "cleared."

    Your transform could be something as simple as this:

    if "good trap string" in evt.summary:
        evt.severity = 0

    You would just need to pick out some text from your good trap event that doesn't appear in a bad trap event.

    ------------------------------
    Michael J. Rogers
    Senior Instructor - Zenoss
    Austin TX
    ------------------------------