ZenPacks, JSON API, and Integrations

Expand all | Collapse all

ContextLoader error during zeneventserver start

  • 1.  ContextLoader error during zeneventserver start

    Posted 15 days ago
    Hello All,

    I have noticed an error being thrown in zeneventserver.log during the latest restart of Zenoss Core services. The following is the stacktrace:

    image.png

    It seems that Zep Context is not loading, which causes zeneventserver not to start properly. As a consequence, zenhub fails the health checks, and Zope cannot publish events (it also fails health checks).

    I am using Zenoss Core monitoring 6.1.2 version (Community Edition), with 1 Master and 2 Delegate hosts.
    The rest of the services (mariadb, Solr, RabbitMQ etc.) are up and running normally.

    I have tried removing Lucene-based event indexes (located under /opt/serviced/var/volumes/$SERVICE_ID/zeneventserver/index) and restarting zeneventserver, but it didn't help. Have checked also zenoss_zep database, and it seems to contain the proper event key records.

    Have checked also the list of RabbitMQ queues, using command:
        rabbitmqctl list_queues -p /zenoss
    but, did not notice anything unusual there.

    In the zeneventserver.log I can notice also that the service connects to Amqp Connection Manager, displaying the message: ..... Connected to message broker at amqp://zenoss@127.0.0.1:5672/zenoss">zenoss@127.0.0.1:5672/zenoss.

    Please, does anyone have any ideas what might be wrong with the zeneventserver service? What else can I check from log files, system parameters or running services?

    Thank you in advance for your time!

    Regards,
    Klodian

    ------------------------------------------------ 
    Application Development Specialist
    ------------------------------------------------


  • 2.  RE: ContextLoader error during zeneventserver start

    Posted 14 days ago

    Klodian,

    You said that you deleted the Lucene indices and then restarted zeneventserver.  Was zeneventserver running at the time you deleted Lucene?  If so, some of the files may not have been deleted.

    The normal order of operations for a Lucene deletion is:

    serviced service stop zeneventserver
    rm -rf /opt/serviced/var/volumes/$SERVICE_ID/zeneventserver/index
    serviced service start zeneventserver

    If you already did that, or you walk through the steps again with no joy, I would recommend attaching the zeneventserver.log here for review.



    ------------------------------
    Michael J. Rogers
    Senior Instructor - Zenoss
    Austin TX
    ------------------------------



  • 3.  RE: ContextLoader error during zeneventserver start

    Posted 14 days ago
    Hello Michael,

    Thank you for the instructions!

    I repeated the re-indexing process, this time removing old Lucene indices from all running Zenoss hosts. Apparently, the first time I had tried the process there were remaining old indices in the Zenoss Delegate servers.
    Afterwards, I restarted zeneventserver, and all services (zeneventserver, zenhub, Zope) started successfully.

    Thank you again for your assistance.

    Regards,
    Klodian

    ------------------------------
    Klodian Vangjeli
    ------------------------------