ZenPacks and JSON API

Expand all | Collapse all

Modeling and Component Creation in SDK

Jump to Best Answer
  • 1.  Modeling and Component Creation in SDK

    Posted 10-04-2018 12:53 PM
    Edited by Austin Culbertson 10-04-2018 12:53 PM

    Using Zenoss 4.2.5, with access to zenpacklib.

    Using the Zenoss-SDK (Device Modeling - ZenPack SDK 2.1.1 documentation), I am trying to figure out how I can write a modeler plugin to be added to an existing device that will collect an IP address from an ipmitool command - This is primarily so I can store this IP on a per-machine basis and monitor the status of the device's IPMI URL using other methods with which I'm more familiar.

    I'm struggling to get the component added, however. I have 'stitched together' the ExampleCMD.py.example and what I've found on the SDK URL to get the following:

    # Module-level documentation will automatically be shown as additional
    # information for the modeler plugin in the web interface.
    """
    ExampleCMD
    An example plugin that illustrates how to model devices using SSH.
    """
    
    # This is an example of an CMD-based modeler plugin. It won't be recognized by
    # Zenoss as an available modeler plugin unless the .example extension is
    # removed.
    
    # When configuring modeler plugins for a device or device class, this plugin's
    # name would be community.snmp.ExampleCMD because its filesystem path within
    # the ZenPack is modeler/plugins/community/snmp/ExampleCMD.py. The name of the
    # class within this file must match the filename.
    
    import re
    
    # CommandPlugin is the base class that provides lots of help in modeling data
    # that's available by connecting to a remote machine, running command line
    # tools, and parsing their results.
    from Products.DataCollector.plugins.CollectorPlugin import CommandPlugin
    
    # Classes we'll need for returning proper results from our modeler plugin's
    # process method.
    from Products.DataCollector.plugins.DataMaps import ObjectMap, RelationshipMap
    
    
    class BMCIP(CommandPlugin):
    
        # The command to run.
        command = "/usr/bin/ipmitool lan print"
    
        relname = "ribbonBMCIPs"
        modname = 'ZenPacks.Bandwidth.SBCBMCMonitor.BMCIP'
    
        # Modeler plugins can optionally implement the "condition" method. This
        # allows your plugin to determine if it should be run by looking at the
        # configuration of the device that's about to be modeled. Return True if
        # you want the modeler plugin to execute and False if you do not.
        #
        # The default is to return True. So ordinarily you wouldn't even implement
        # the method if you were just going to blindly return True like this
        # example.
        def condition(self, device, log):
            return True
    
        def process(self, device, results, log):
            log.info("Modeler %s processing data for device %s",
                self.name(), device.id)
    
            objectmaps = []
    
            # For CommandPlugin, the results parameter to the process method will
            # be a string containing all output from the command defined above.
    
            # results contents..
            # major minor  #blocks  name
            #
            #    8     0   41943040 sda
            #    8     1     104391 sda1
            #    8     2   41833260 sda2
            #  253     0   41091072 dm-0
            #  253     1     720896 dm-1
    
            rm = self.relMap()
    
            log.info("rm is %s", rm)
    
            matcher = re.compile(r'^IP Address\s+:\s(\d+.\d+.\d+.\d+)')
    
            for line in results.split('\n'):
                line = line.strip()
    #            match = matcher.search(line)
                match = re.match('IP Address\s+:\s(\d+.\d+.\d+.\d+)', line)
                if match:
    #                objectmaps.append(ObjectMap({
    #                    'id': self.prepId(match.group(0)),
    #                    'description': match.group(0),
    #                    }))
                   rm.append(self.objectMap({
                       'id': self.prepId(match.group(1)),
                       'description': match.group(1),
                       'bmcip': match.group(1),
                       }))
    
                   log.info("Found IP Address: %s", match.group(1))
            # Return a RelationshipMap that describes the component, relationship
            # on that component, and the module name for the created objects. Pass
            # in the previously built list of ObjectMaps that will be used to
            # populate the relationship.
    #        return RelationshipMap(
    #            compname="bMCIP", relname="bMCIP",
    #            modname='ZenPacks.Bandwidth.SBCBMCMonitor.BMCIP',
    #            objmaps=objectmaps)
    
            log.info("rm is %s", rm)
    
            return rm
    


    At present, my zenpack.yaml looks like this:

    name: ZenPacks.Bandwidth.SBCBMCMonitor
    
    classes:
      RibbonSBCDevice:
        base: [zenpacklib.Device]
        label: Ribbon SBC Device
    
      RibbonBMCIP:
        base: [zenpacklib.Component]
        label: BMCIP_Label
        properties:
          bmcip:
            label: BMC IP
    
    class_relationships:
      - RibbonSBCDevice 1:MC RibbonBMCIP
    


    Debugging shows that we are running the command and finding the information:

    2018-10-04 17:47:33,374 DEBUG zen.ZenModeler: Plugin BMCIP.BMCIP results = Set in 
    [snip]
    IP Address Source       : Static Address
    IP Address              : 192.168.125.6
    [/snip]
    2018-10-04 17:47:33,374 INFO zen.ZenModeler: Modeler BMCIP.BMCIP processing data for device 192.168.125.22
    2018-10-04 17:47:33,374 INFO zen.ZenModeler: rm is <RelationshipMap []>
    2018-10-04 17:47:33,375 INFO zen.ZenModeler: Found IP Address: 192.168.125.6
    2018-10-04 17:47:33,375 INFO zen.ZenModeler: rm is <RelationshipMap [<ObjectMap {'bmcip': '192.168.125.6',
     'classname': '',
     'compname': '',
     'description': '192.168.125.6',
     'id': '192.168.125.6',
     'modname': 'ZenPacks.Bandwidth.SBCBMCMonitor.BMCIP'}>]>


    I think this might have something to do with the fact that there is no compname here. However, I don't see this field ever being referenced in the Zenoss SDK documentation.

    I'm certain I'm missing something, but I just don't know what. Is creating my own component not achieved in this way? Am I using a resource that doesn't apply to my version of Zenoss?



    ------------------------------
    Austin Culbertson
    NOC Monitoring Engineer
    ------------------------------


  • 2.  RE: Modeling and Component Creation in SDK

    Posted 10-04-2018 04:23 PM
    First glance I think your modname needs to be RibbonBMCIP at the end
    e.g. 'ZenPacks.Bandwidth.SBCBMCMonitor.RibbonBMCIP'

    ------------------------------
    Patrick McMahon
    Sr. Client Services Engineer
    Zenoss
    ------------------------------



  • 3.  RE: Modeling and Component Creation in SDK

    Posted 10-05-2018 07:10 AM
    There are good examples and explanations on all the ways to update the model in the PythonCollector ZenPack doc (see Updating the Model)
    Specifically it gives some examples for when and how to set "compname"
    https://www.zenoss.com/product/zenpacks/pythoncollector

    There's a good description on how to determine relname and modname (see "3. relname and modname Properties")
    https://zenpack-sdk.zenoss.com/en/latest/tutorial-http-api/modeler-plugin.html

    One thing that isn't obvious is that self.objectMap and self.relMap are what use relname and modname in your modeler class

    You can see that in the code, they're just convenience methods for creating RelationshipMap and ObjectMap
    Products/DataCollector/plugins/DataMaps.py

    def objectMap(self, data={}):
    """Create an object map from the data
    """
    om = ObjectMap(data)
    om.compname = self.compname
    om.modname = self.modname
    om.classname = self.classname
    return om

    def relMap(self):
    """Create a relationship map.
    """
    relmap = RelationshipMap()
    relmap.relname = self.relname
    relmap.compname = self.compname
    return relmap

    ------------------------------
    Patrick McMahon
    Sr. Client Services Engineer
    Zenoss
    ------------------------------



  • 4.  RE: Modeling and Component Creation in SDK

    Posted 10-05-2018 11:27 AM
    Edited by Austin Culbertson 10-05-2018 11:32 AM
    The PythonCollector provides some additional insight, but it's still not working.

    Is it absolutely imperative that the device to which I'm applying this modeler plugin match the device type contained in the zenpack.yaml? A 'RibbonSBCDevice', specifically? 

    In the PythonCollector documentation it indicates:

    # An ObjectMap with no compname or relname will be # applied to the device. ObjectMap({'rackSlot': 'near-the-top'}),

    # An ObjectMap with a compname, but no relname will be # applied to a static object that's always a child of the # device. For example: hw and os. ObjectMap({ 'compname': 'hw', 'totalMemory': 45097156608}),


    What does 'it will be applied to the device' entail? Does that mean that, if I'm in zendmd, and it's properly 'added', I can do something like:

    dev = find('192.168.125.22')
    dev.bmcip
    
    OR
    
    dev.ribbonBMCip[0]
    


    Or something like that? I'm not quite 100% certain on what the implications of this are, in-so-far as how it translate to what I might see in the GUI (if anything). Likewise, what does it mean to be 'a static object that is always a child of the device'?
    ------------------------------
    Austin Culbertson
    NOC Monitoring Engineer
    ------------------------------



  • 5.  RE: Modeling and Component Creation in SDK
    Best Answer

    Posted 10-05-2018 04:17 PM
    First off, couple more thoughts that come to mind
    1) If you're troubleshooting why are changes not being applied when I model... specifically relation issues check zenhub logs (thats what consumes the result of the "process" method and you don't see those errors in the model logs)
    2) You may have just failed to paste it in but you're missing a device_class that uses your RibbonSBCDevice as the zPythonClass in your yaml
    For example NetBotzDevice in https://zenpack-sdk.zenoss.com/en/latest/tutorial-snmp-device/device-modeling.html
    device_classes:
      /NetBotz:
        zProperties:
          zPythonClass: ZenPacks.training.NetBotz.NetBotzDevice
    When your running your modeler plugin in the context of some Device Class, the zPythonClass zProperty in that Device Class tells the modeler plugin what Device to create or attach to relations to. By default I think it's Products.ZenModel.Device.Device (may be a shorter form) and you did not define a relation to Device for your component RibbonBMCIP so that may be the issue.

    Again, you would resolve this by defining a Device Class where the zPythonClass is ZenPacks.Bandwidth.SBCBMCMonitor.RibbonSBCDevice
    And While defining the Device Class, might as well add your modeler plugin to the zCollectorPlugins zProperty
    You may be thinking to yourself I'll just define the class relationship Device 1:MC RibbonBMCIP
    I advise against this for many reasons 1) not every device is going to have this component 2) your setting your self up for pain later the more Devices you have in your Zenoss system

    Question 1) Is it absolutely imperative that the device to which I'm applying this modeler plugin match the device type contained in the zenpack.yaml? A 'RibbonSBCDevice', specifically?
    Answer 1) Kind of covered this in my thoughts, but it's imperative the RelationshipMap you return has been defined e.g. my suggested fix to modname was to match what you had defined RibbonSBCDevice 1:MC RibbonBMCIP

    Question 2) What does 'it will be applied to the device' entail?
    Answer 2) It general it means it is updating properties on the Device (for example you would see these values in the Device "Overview" section of the UI)

    Question 3) Does that mean that, if I'm in zendmd, and it's properly 'added', I can do something like...
    Answer 3) If you defined a property "bmcip" on the device in the yaml, and the modeler set its value properly, something like dev.bmcip would work. You however have define a component with that property so assuming the modeling worked something like dev.ribbonBMCIPs[0].bmcip would work. (keep in mind I'm not on your system and can't verify exact syntax so answers are not exact)

    Question 4)  What I might see in the GUI (if anything).
    Answer 4) Kind of answered in "Answer 2", but the RibbonBMCIPs would show up in the UI under Components (if you get the modeler to work)

    Question 5) what does it mean to be 'a static object that is always a child of the device'
    Answer 5) Not sure on the official explanation but I would say its talking about special components that are already defined to have a relationship with Device e.g. you dont have to define them in a yaml you just set the value in the modeler plugin

    Thanks
    -Patrick


    ------------------------------
    Patrick McMahon
    Sr. Client Services Engineer
    Zenoss
    ------------------------------



  • 6.  RE: Modeling and Component Creation in SDK

    Posted 10-09-2018 03:36 PM
    Edited by Austin Culbertson 10-09-2018 03:37 PM

    I got this to work by manually defining the zPythonClass of the group that I wanted to be 'RibbonSBCDevices' and moving existing devices out of and back to the groups in Zenoss to get the RibbonSBCDevice class to be applied to them.

    My next question is this: Is it possible to 'Extend' or augment an existing device class? i.e. if I change zenpack.yaml to define the device as 'LinuxDevice', is it going to 'extend' or 'overwrite' the LinuxDevice class?



    ------------------------------
    Austin Culbertson
    NOC Monitoring Engineer
    ------------------------------



  • 7.  RE: Modeling and Component Creation in SDK

    Posted 10-11-2018 05:00 AM
    Have a look at the ZenPack Developers' Guide - https://github.com/ZenossDevGuide/DevGuide   . Chapter 8.11 has a discussion about creating new components directly on an existing (zPythonClass) device class, rather than creating a new device class.

    Cheers,
    Jane

    ------------------------------
    Jane Curry
    Skills 1st United Kingdom
    jane.curry@skills-1st.co.uk
    ------------------------------



  • 8.  RE: Modeling and Component Creation in SDK

    Posted 6 days ago
    Edited by Austin Culbertson 6 days ago

    Thank you all for your help so far - I've got this working now when creating my own new device.

    I've tried tying this onto the ZenPacks.zenoss.LinuxMonitor.LinuxDevice type, but zenpacklib doesn't seem to like this:

    Here is my attempted configuration:

    name: ZenPacks.Bandwidth.RibbonSBCMonitor

    classes:
    RibbonSBCDevice:
    base: [zenpacklib.Device]
    meta_type: RibbonSBCDevice
    label: Ribbon SBC Device

    BMCIP:
    base: [zenpacklib.Component]
    meta_type: BMCIP
    label: BMC IP
    properties:
    ipmiIP:
    type: string
    label: IP Address
    short_label: IP
    label_width: 30


    class_relationships:
    #- RibbonSBCDevice 1:MC BMCIP
    - ZenPacks.zenoss.LinuxMonitor.LinuxDevice 1:MC BMCIP

    Here is the error:

    [zenoss@zenoss-03.lab1 zenpack-testing]$ zenpacklib --lint ZenPacks.Bandwidth.RibbonSBCMonitor/ZenPacks/Bandwidth/RibbonSBCMonitor/zenpack.yaml
    ERROR:ZenPacks.Bandwidth.RibbonSBCMonitor:Failed to import class ZenPacks.zenoss.LinuxMonitor.LinuxDevice from ZenPacks.zenoss.LinuxMonitor (Failed while importing class LinuxMonitor from module ZenPacks.zenoss.LinuxMonitor)
    2018-11-08 19:19:59,061 [ZenPacks.Bandwidth.RibbonSBCMonitor] ERROR Failed to import class ZenPacks.zenoss.LinuxMonitor.LinuxDevice from ZenPacks.zenoss.LinuxMonitor (Failed while importing class LinuxMonitor from module ZenPacks.zenoss.LinuxMonitor)
    ERROR:ZenPacks.Bandwidth.RibbonSBCMonitor:Failed to add relationship (bmcips) to imported class (ZenPacks.zenoss.LinuxMonitor.LinuxDevice).
    2018-11-08 19:19:59,062 [ZenPacks.Bandwidth.RibbonSBCMonitor] ERROR Failed to add relationship (bmcips) to imported class (ZenPacks.zenoss.LinuxMonitor.LinuxDevice).

    Is this a path issue, or some type of issue surrounding 'private' device types?



    ------------------------------
    Austin Culbertson
    NOC Monitoring Engineer
    ------------------------------



  • 9.  RE: Modeling and Component Creation in SDK

    Posted 5 days ago
    In your class_relationships I think you need to add LinuxDevice a second time so it's the class inside the module (both with the same name)
    e.g.
    - ZenPacks.zenoss.LinuxMonitor.LinuxDevice.LinuxDevice 1:MC BMCIP


    ------------------------------
    Patrick McMahon
    Sr. Client Services Engineer
    Zenoss
    ------------------------------



  • 10.  RE: Modeling and Component Creation in SDK

    Posted 5 days ago
    Ahh! Excellent, that did the job - Thank you, Patrick!

    ------------------------------
    Austin Culbertson
    NOC Monitoring Engineer
    ------------------------------