Reporting, Analysis and Analytics

Expand all | Collapse all

Error message, FileSystem template (SSH Linux)

  • 1.  Error message, FileSystem template (SSH Linux)

    Posted 06-12-2018 04:56 AM
    Edited by Jonas Stoltz 06-12-2018 04:56 AM
    When running or testing the template we get an error message;
    Preparing Command...
    Executing command /bin/sh [args omitted] against 172.26.1.48
    exception while executing command
    type: <class 'twisted.internet.error.ReactorNotRestartable'> value:
    DONE in 0 seconds

    Command run;
    /bin/sh -c 'export PATH=$$PATH:/bin:/sbin:/usr/bin:/usr/sbin; if command -v timeout > /dev/null 2>&1; then timeout 30 /usr/bin/env df -kP; else /usr/bin/env df -kP; fi'

    (Zenoss 6.1.0 r46)
    ------------------------------
    Jonas Stoltz

    ------------------------------


  • 2.  RE: Error message, FileSystem template (SSH Linux)

    Posted 06-13-2018 07:29 AM
    Does it work with just the simple DF command? I've not gotten to the point I can troubleshoot that complex command very well as well, but I've been able to work around it by using a simpler command line.

    ------------------------------
    Luke Lofgren
    Infrastructure Architect
    Acxiom Corporation (home based associate)
    Waterford PA
    ------------------------------



  • 3.  RE: Error message, FileSystem template (SSH Linux)

    Posted 06-14-2018 05:26 AM
    Excellent suggestion to test first with a very simple command like df to check that the authentication is setup correctly.

    Then I would try running zencommand standalone in debug and checking the output.  You don't say whether you are on Zenoss 4.x or earlier, or on 5 or 6 (the dockerized versions).  If on Zenoss 4 or less, from a command line, for  a device called fred.class.example.org, use:

    zencommand run -v 10 -d fred.class.example.org --showfullcommand > /tmp/fred 2>&1

    then check /tmp/fred.  The --showfullcommand should reveal the exact command sent, complete with parameter substitution.  It's almost certainly a glitch with parameter substitution, quoting or special characters in the bash command, once you have eliminated basic authentication issues.

    If you are on a Zenoss 5 or 6, you will need to attach to the zencommand container first with:

    serviced service attach zencommand su - zenoss

    Cheers,
    Jane

    ------------------------------
    Jane Curry
    Skills 1st United Kingdom
    jane.curry@skills-1st.co.uk
    ------------------------------



  • 4.  RE: Error message, FileSystem template (SSH Linux)

    Posted 06-18-2018 05:08 AM
    Edited by Jonas Stoltz 06-20-2018 09:04 AM
    Hi,

    Thanks for the suggestion on running just "df -kP" which I already have (did again just to be sure), but same error.
    I did post version in my post (Zenoss 6.1.0 r46) and running the debug did not get me any closer to the problem.

    I'm not sure what to look for in that file (contains a LOT), but searching for "df" did not work (noting found except some other words containing df).

    ------------------------------
    Jonas Stoltz
    ------------------------------



  • 5.  RE: Error message, FileSystem template (SSH Linux)

    Posted 06-29-2018 08:03 PM
    Try using the full path for df

    Example:
    /bin/df

    Also, using timeout before df is a very good idea. Especially if you have network drives.

    I have seen network drives hang from NAS issues and if you try running df, it just hangs. This is bad when Zenoss is sending this command every 5 minutes.

    ------------------------------
    jstanley
    ------------------------------



  • 6.  RE: Error message, FileSystem template (SSH Linux)

    Posted 07-04-2018 06:11 AM
    Jay,

    Thanks for your suggestion but no luck.

    I did on the other hand restart everything and then it worked with the defaults, but just ONCE.
    Restarting all services does the trick, but cannot work like that.

    Any suggestions?

    ------------------------------
    Jonas Stoltz
    CompanyName
    ------------------------------



  • 7.  RE: Error message, FileSystem template (SSH Linux)

    Posted 07-09-2018 07:58 AM
    type: <class 'twisted.internet.error.ReactorNotRestartable'> is the error? If that is the case, something is stopping the twisted reactor. Working after a restart would make sense, since everything gets restarted and the reactor will be running again, executing commands.

    • You said it works once, how do you know it works once?
    • Can you post the output (scrubbed) of a zencommand run -d<deviceId> --showfullcommand
    • Are you running tests from the datasource? If so, can you restart zencommand, then not run any tests and see if things start graphing





    ------------------------------
    jstanley
    ------------------------------



  • 8.  RE: Error message, FileSystem template (SSH Linux)

    Posted 08-06-2018 07:03 AM
    Hi Jay,

    Thanks for the assistance and suggestions here; much appreciated.

    • You said it works once, how do you know it works once?
    I did "test" and saw it working, did another test which failed.

    • Can you post the output (scrubbed) of a zencommand run -d<deviceId> --showfullcommand
    Attached.
    • Are you running tests from the datasource? If so, can you restart zencommand, then not run any tests and see if things start graphing
    Restarted, but no data is coming in (checked graphs for diks util and inode util).
    As far as I can see from the output, it does not run the "FileSystem" commands?

    ------------------------------
    Jonas Stoltz
    CompanyName
    ------------------------------

    Attachment(s)

    txt
    command_output.txt   2K 1 version


  • 9.  RE: Error message, FileSystem template (SSH Linux)

    Posted 08-09-2018 02:49 AM
    Hi,

    It seems to be working now after just restarting, and not testing the setup.
    The only thing left now is that it comes an error message now and then from *all* servers;
    Unable to process COMMAND datasource(s) for device XYZ -- skipping

    It is connected to this, because if I disable the command for file system (disk/idisk) the message is not coming anymore, but also no data of course.

    ------------------------------
    Jonas Stoltz

    ------------------------------