Configuration & Administration

Expand all | Collapse all

SUP 743 update for Core

  • 1.  SUP 743 update for Core

    Posted 07-05-2018 05:29 AM
    I noticed this morning that there is a new SUP update, SUP743, available for Core 4.2.5 users.  It is available from the menus on the right from the forum home page.

    As ever, there is no documentation as to what has changed so I have run some "zenup info " commands on the this new SUP and the previous SUP732,  It would appear that we only have 6 new fixes since April 2017:

    [ZEN-27093] ZEN-27093 Resource Manager will allow duplicate IPs
    for devices


    [ZEN-27216] ZEN-27216: reconnect to rabbit it case it closes the
    channel. Better exception handling in
    processInvalidations


    ZEN-27216: add logic to be able to set the prefetch
    count


    [ZEN-27290] ZEN-27290 ProdStateNotSetError when filtering on
    production state AND sorting other fields, backport
    https://github.com/zenoss/zenoss-prodbin/pull/2177


    [ZEN-27578] ZEN-27578 RelStorage Uses MyISAM Temporary Tables
    for Storing Objects backport:
    https://github.com/zenoss/zenoss-prodbin/pull/2364


    [ZEN-27783] ZEN-27783: zenossdbpack needs further optimization,
    dies with OOM error in large environments

    Sadly, there is no fix for https://jira.zenoss.com/browse/ZEN-30167  regarding changes in productionState not getting propagated without a daemon restart or an explicit "Push Changes"; see   https://community.zenoss.com/forum/community-home/digestviewer/viewthread?GroupId=19&MessageKey=bfaf0778-d532-4be5-b641-d3f88bf8db08&CommunityKey=1518fed6-b03e-410b-81b9-f2240ed9037a&tab=digestviewer&ReturnUrl=%2fforum%2fcommunity-home%2fdigestviewer%3fcommunitykey%3d1518fed6-b03e-410b-81b9-f2240ed9037a%26tab%3ddigestviewer and https://community.zenoss.com/forum/community-home/digestviewer/viewthread?MessageKey=e7f78ee2-115c-4239-85a4-0240d3bd283c&CommunityKey=1518fed6-b03e-410b-81b9-f2240ed9037a&tab=digestviewer#bme7f78ee2-115c-4239-85a4-0240d3bd283c  .

     It would be really useful to many of us if they did fix this one - even though they have now passed their end of maintenance date for 4.2.5 - it is a complete showstopper for many. It was introduced with the previous SUP, SUP732 and has had the ticket open since May 22nd 2018 though it does not appear that anyonbe has looked at it with a view to fixing.

    It would also be interesting to know if anyone who has the chargeable Resource Manager product at 4.2.5, has these issues with productionState as I believe you get different patches from Core users.  The problem was introduced with ZEN-24565.

    Cheers,
    Jane



    ------------------------------
    Jane Curry
    Skills 1st United Kingdom
    jane.curry@skills-1st.co.uk
    ------------------------------


  • 2.  RE: SUP 743 update for Core

    Posted 07-05-2018 08:01 AM
    @John, can ZEN-24565 be made public?

    ------------------------------
    jstanley
    ------------------------------



  • 3.  RE: SUP 743 update for Core

    Posted 07-05-2018 08:05 AM
    Looks like these might be the official patch notes

    ZEN-15509 When deleting an organizational group, the devices that belong to that group are not cleanly removed from the group. Subsequently, if the group is recreated, the devices appear in the list of group members, even though the devices claim to not be a member of the group, and the group still claims to have zero members.

    ZEN-22773 Modifying mappings will produce audit logging; however, only the changed 'to' value is logged.

    ZEN-24034 When a TALES expression is invalid (e.g. referencing an attribute that does not exist), the associated command notification will fail to send.

    ZEN-27093 Resource Manager will allow duplicate IPs for devices.

    ZEN-27216 ZenHub invalidation workers stop processing.

    ZEN-27290 If the infrastructure page is sorted by certain fields (e.g. collector, events) AND a filter to production state is applied, the device list will fail to load and a flare "ProdStateNotSetError" is displayed.

    ZEN-27578 RelStorage has been updated to use InnoDB instead of MyISAM engine to improve performance when updating ZODB.

    ZEN-27783 Zenossdbpack needs further optimization since it dies with OOM error in large environments

    ------------------------------
    jstanley
    ------------------------------



  • 4.  RE: SUP 743 update for Core

    Posted 07-10-2018 12:56 AM
    Be adviced that SP743 most likely breaks all notifications via Pager or email. I tested an upgrade and zenactiond was flooded with errors like:

    ERROR zen.actions: 'skipfails'

    and on my zenoss master, I got these events:

    "/opt/zenoss/Products/ZenModel/actions.py", line 309, in execute self.executeBatch(notification, signal, targets) File "/opt/zenoss/Products/ZenModel/actions.py", line 390, in executeBatch subject = processTalSource(notification.content['clear_subject_format'], notification.content['skipfails'], **data) KeyError: 'skipfails'

    I did a roleback of my upgrade, but I suspect that the patch has some flaws regarding notifications.

    Running Zenoss Core 4.2.5 on Centos 6

    ------------------------------
    Thomas Luther
    Senior Consultant
    Netic A/S
    ------------------------------



  • 5.  RE: SUP 743 update for Core

    Posted 07-10-2018 06:49 AM
    Edited by Jay Stanley 07-10-2018 07:13 AM
    Here is a patch I think will fix it

    I can't test this on my lab (no old notifications) to see if this fixes it completely, but I was able to mimic the issue using some dmd magic. If you want to test and let me know if this fixes the issue for you, that would be awesome.

    https://github.com/jstanley23/zenoss_patches/blob/master/actions_skipfails.patch

    ------------------------------
    jstanley
    ------------------------------



  • 6.  RE: SUP 743 update for Core

    Posted 07-11-2018 11:45 AM
    I have to say that SUP743 has not (so far) broken my notifications - but i can see that Jay's code is safer so will implement.

    Zenoss - @John - since Jay has now provided 2 patches, one for each of SUP 732 and 743, both of which are obviously complete showstoppers to some users, - may we ask that Zenoss provides another "last maintenance SUP" for Core that at least includes these two?

    Thanks,
    Jane​​

    ------------------------------
    Jane Curry
    Skills 1st United Kingdom
    jane.curry@skills-1st.co.uk
    ------------------------------



  • 7.  RE: SUP 743 update for Core

    Posted 07-18-2018 05:53 AM
    Hey Jay

    I have now had a chance to test this on our dev-setup, and I can reproduce the error and can then verify that your patch fixes the issue.

    It appears that the problem is only with Page notifications. My mail-notifications did come through even without the patch. But setting up a Page notification, then I had the skipfails error in zenactiond.log.



    ------------------------------
    Thomas Luther
    Senior Consultant
    Netic A/S
    ------------------------------