0

I am new to Sensu remediation. I tried to restart process with custom script.

Senario: if my http link goes down then I want to manually run the script and start it.

I tried remediation on sensu which allows to run script automatically if something ur monitoring goes down with custom script for that monitoring check. Yet I'm facing problem all check and connections are good, but when my link is down sensu remediator is not triggering client. I have posted logs and configuration please tell me where I'm going wrong ..

this is Sensu-server log

{"timestamp":"2016-05-16T09:44:52.768622+0000","level":"info","message":"processing event","event":{"id":"9a9f66c2-e70e-45fb-87fb-c9e9085c8e05","client":{"name":"zubron","address":"10.0.0.110","subscriptions":["zubron"],"version":"0.20.3","timestamp":1463391880},"check":{"command":"/etc/sensu/plugins/check_http -H 10.0.0.110 -p 7077","interval":60,"occurrences":2,"handlers":["remediator"],"subscribers":["zubron"],"standalone":false,"remediation":{"remediate-zubron":{"occurrences":[1,3],"severities":[2]},"trigger_on":["zubron"]},"name":"check-zubron-port","issued":1463391892,"executed":1463391892,"duration":0.002,"output":"connect to address 10.0.0.110 and port 7077: Connection refused\nHTTP CRITICAL - Unable to open TCP socket\n","status":2,"history":["0","0","0","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2","2"],"total_state_change":4},"occurrences":18,"action":"create","timestamp":1463391892}}
{"timestamp":"2016-05-16T09:44:52.864908+0000","level":"info","message":"handler output","handler":{"command":"/etc/sensu/handlers/sensu.rb","type":"pipe","severities":["critical"],"name":"remediator"},"output":["/etc/sensu/handlers/sensu.rb:108:in `[]': can't convert String into Integer (TypeError)\n","\tfrom /etc/sensu/handlers/sensu.rb:108:in `block in parse_remediations'\n","\tfrom /etc/sensu/handlers/sensu.rb:106:in `each'\n","\tfrom /etc/sensu/handlers/sensu.rb:106:in `parse_remediations'\n","\tfrom /etc/sensu/handlers/sensu.rb:90:in `handle'\n","\tfrom /var/lib/gems/1.9.1/gems/sensu-plugin-1.2.0/lib/sensu-handler.rb:55:in `block in <class:Handler>'\n","REMEDIATION: Evaluating remediation: zubron {\"remediate-zubron\"=>{\"occurrences\"=>[1, 3], \"severities\"=>[2]}, \"trigger_on\"=>[\"zubron\"]} #=18 sev=2\n"]}

this is my check file on Sensu-server...

{
    "checks": {
            "check-zubron-port": {
                    "command": "/etc/sensu/plugins/check_http -H 10.0.0.110 -p 7077",
                    "interval": 60,
                    "occurrences": 2,
                    "handlers": [
                            "remediator"
                    ],
                    "subscribers": [
                            "zubron"
                    ],
                    "standalone": false,
                    "remediation": {
                            "remediate-zubron": {
                                    "occurrences": [
                                            1,
                                            3
                                    ],
                                    "severities": [
                                            2
                                    ]
                            },
                            "trigger_on": [
                                    "zubron"
                            ]


                  }
                }
        }
}

and this is my remediation file...

{
        "remediate-zubron": {
            "command": "sudo /bin/bash ~/zubron/home/moofwd-zubron-server/bin/start-moofwd.sh",
            "handlers": [],
            "subscribers": [
                "zubron"
            ],
            "standalone": false,
            "publish": false
    }
}

Rest sensu.rb I used from this link

Is there anything I am missing ?

Is there any other monitoring system from where we can run script or commands if anything goes down?

I have already tried nagios nectar and monit.

1
  • Come on man editing post will not give answer ... if you know my problem then answer it dont edit my files ... May 17, 2016 at 14:18

1 Answer 1

0

I see an error on your Remediation file. You're missing "checks" key.

It should be like this,

{
    "checks": {
        "remediate-zubron": {
            "command": "sudo /bin/bash ~/zubron/home/moofwd-zubron-server/bin/start-moofwd.sh",
            "handlers": [],
            "subscribers": [
                "zubron"
            ],
            "standalone": false,
            "publish": false
        }
    }
}
  1. Few other possible issues are your client config should be subscribed to your name, i.e name university should be subscribed to university in subscribers.

    {
        "client": {
            "name": "university",
            "address": "IP ADDRESS",
            "subscriptions": [
              "linux", "web-server", "system", "university"
            ]
        }
    }
    
  2. Another issue might be with api_request(:POST) in remedediator.rb or sensu.rb, Following method should look like the exact given below. In some old code it has '/checks/request' instead of '/request' and results in breakage.

    def trigger_remediation(check, subscribers)
        api_request(:POST, '/request') do |req|
            req.body = JSON.dump('check' => check, 'subscribers' => subscribers)
        end
    end
    
  3. In some cases you need to fix both Case 2 and Case 3.

Following is the reference link. To give you more clarity on the issue.

https://github.com/sensu/sensu-community-plugins/issues/1162

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .