1

During code deployments there is a risk of service/system failures. The Network Operations Center monitors alerts from servers. If the alert comes as a result of a code deployment then it might not be anything to be concerned about. If however nobody is working on the server, then clearly the alert is a cause for concern.

We currently inform the NOC when we are working on a server. I was wondering if there already exists a script/program that can listen for inputs on a given server and when a certain input is received to broadcast the input that was received along with the server it was received on to a specified server.

Lets say I am working on server 'x' and run puppet agent

Could a process running in the background configured to listen for a user input at the terminal with 'puppet agent' and then broadcast a message to a given server when that input is present?

The idea would be to add a list of common commands used by the deployment team that this process would listen for and send broadcasts for all of them.

It would also be useful if it could search a predefined location such as a log for that command (in this case puppet agent) for an exit message such as 'puppet successfully completed..' and in the event that it found that, send an additional broadcast to the given server.

I'm sure it could be created with enough effort, just curious to know if something like this already exists that can be configured for our needs.

Thanks

  • Welcome to Super User. Unfortunately, we are not a code-writing service. Instead of simply asking for code to perform a particular task, please show us what you've tried so far (including any code you currently have) and where you're stuck so that we can help you with your specific problem. Questions that only ask for code are too broad and are likely to be put on hold or closed. Please read How do I ask a good question?. – DavidPostill Jun 24 '15 at 13:00
  • Hi David. I don't think I asked anyone to write any code for me. Just asked if someone knew of a script or tool that already performs this function. I tried searching google but this is one of the situations where the trick is knowing the right question to ask in order to find what you are looking for. Again, if something like this doesn't exist, I can approach RnD. If it does all I am essentially asking for is to be pointed in the right direction. That being said, thank you for the advice. I'll keep that in mind. – ARL Jun 24 '15 at 13:04
  • Just asked if someone knew of a script or tool that already performs this function. - learning material such as asking for scripts is off topic. Personally, I'd try a search engine, or, read a book and learn the skills needed to complete the task yourself – Dave Jun 24 '15 at 13:21
  • "Just asked if someone knew of a script or tool that already performs this function" - that is off topic. Questions seeking product, service, or learning material recommendations are off-topic. See On Topic. Try softwarerecs.stackexchange.com but please first read What is required for a question to contain “enough information”. – DavidPostill Jun 24 '15 at 13:21
0

There probably is no real tool that exists which does what you are looking for, because the paradigm of having special snowflakes that you ssh/rdp into is fading because it does not scale. You should think of your servers as cattle, not pets.

Instead of ssh-ing into your machine, and running puppet apply; it is better to use a puppet server, and leverage parameterized classes to define which build should be released.

Puppet Master
External Node Classifier

Ideally, you should have two to four nearly identical servers, each with identical configurations. (production + stage + qa + dev). That way you can test that your deployment works correctly on a machine before you push your deploy to production.

If your deploys require multiple steps that can not be encapsulated in your module (like remove from load balancer, modify database, ect...). Then you will want to augment your puppet runs with a tool that can do these steps in order.

CloudSlang Orchestration Management
Capistrano Deployment Automation
RunDeck Gui that define jobs / Tasks

Other tools that can help with the deployment procedure.

Consul Service Discovery + Reporting

Lastly, it sounds like you want a feed back loop that tells you if a deploy was successful. Puppet master, and the foreman can both do that. You could also report the status to a REST base database.

InflxDB
Iron.io

Then use a dashboard style tool to show the last x number of results.

dashing
tipboard

Or, you could push straight to a chat room like slack.

You mentioned watching logs to find the output of commands. You could setup log monitoring, which could show messages in a dashboard, or even send alerts if a particular line is logged.

Here is documentation on setting up an ELK Stack (Elasticsearch, Logstash, Kibana )

Other Resources

https://sysadmincasts.com/episodes/47-zero-downtime-deployments-with-ansible-part-4-4

http://aws.amazon.com/codedeploy/

  • Spuder, thanks for your valuable input. I have just started in a new position and am currently learning the ropes. Im sure the resources you have provided will be very useful. Thanks again! – ARL Jun 28 '15 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.