2

This is my default cloud-init

package_update: true
package_upgrade: true
users:
  - name: sammy
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1...
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    groups: sudo
    shell: /bin/bash
write_files:
  - path: /etc/issue.root
    content: |
      SSH access for root is blocked. Please use xxx instead.
  - path: /etc/issue.everybody
    content: |
      Welcome! This Ubuntu 22.04 is spun up by xxx using DigitalOcean APIv2.
  
runcmd:
  - |
    curl https://events.hookdeck.com/e/abc -X POST -d "{ "command": "configure-sshd-config" }"

I actually have more but i truncated the cloud-init yaml

i like to run curl https://events.hookdeck.com/e/abc -X POST -d "{ "event": "dependent on where i run this" }"

in some areas such as before/after adding the sammy user or before/after running the package upgrade and writing files.

I know how to do this easily within my long runcmd.

But i am unsure how to do this for non-runcmd.

The reason is I have a software that will call DigitalOcean API on behalf of my users who have DigitalOcean accounts.

I like to give them a progress bar and the only way I can do that is by calling curl on webhooks.

4
  • 2
    I don't think there's really a way to do what you want without writing it all as one giant bash script as using that as your userdata. One other possibility that doesn't give you much control is cloud-init's reporting feature. It's currently under documented, but you can see an example here: github.com/canonical/cloud-init/blob/main/doc/examples/… . For any particular event defined in the code, it will POST to a web server. An event gets posted after every module run in cloud-init (along with other things). There are currently 111 total events per boot.
    – falcojr
    Jun 8, 2022 at 18:12
  • @falcojr when you say "not much control" what do you mean exactly? i cannot control which event it logs? or i cannot control how they post the event data to the webhook? or both? and do you have an opinion about Slartibartfast answer? I kinda don't grok the answer provided.
    – Kim Stacks
    Jun 9, 2022 at 3:55
  • Yes to both basically. All events gets logged and you really only have control over the destination.
    – falcojr
    Jun 9, 2022 at 13:13
  • I need to be able to identify the server that's sending the data over webhooks. so i guess i cannot add extra data @falcojr to do so?
    – Kim Stacks
    Jun 11, 2022 at 1:15

2 Answers 2

2

This is not a feature of cloud-init (outside of reporting, as @falcojr noted).

Your question has already been answered by others, so I'll share some ancillary information related to how one can measure (and optimize) the performance of cloud-init. This won't get you your status bar, but maybe it will get you a faster boot time (which is often a motivation behind this kind of question).

Performance Tools:

If you are interested in better understanding or optimizing cloud-init boot time, there are some utilities you might consider.

cloud-init analyze: similar to systemd-analyze, it will show a breakdown of what cloud-init spent time on

systemd-analyze: shows a breakdown of what systemd spent time on

Optimization Strategy:

When trying to optimize a system boot time, I typically start by running systemd-analyze blame, looking at the most time-consuming service, and then asking myself: a) Do I need it? and if I do b) Is there anything I can do to make this faster?

Disable/reconfigure, rinse and repeat. If you don't know, don't guess. Do your research before gutting your OS.

Same with cloud-init analyze blame.

Typically there are only a few services worth looking at before you are considering services/modules that are under a second. I've run this a few times on my own servers and I typically find things like package install and updates as well as other network/disk intensive tasks take up the most time. It's easy to accidentally write a config that "works", but is also slow.

Quick Example:

I've seen some cloud-configs out there that do things like:

runcmd:
  - "apt-get install git"
  - "apt-get install neovim"

which won't perform nearly as well as the following (or the builtin module):

runcmd:
  - "apt-get install git neovim"

On a quick GCP test these two had a 3 second delta for me.

This example is simple, but since I've seen blog posts doing the former, my guess is this section might help somebody at some point :-)

2
  • 1
    thank you for the effort, Brett! really appreciative 🙇🏻‍♂️ However, i would say the status bar is not an intermediate goal. At least not now. It's definitely an end goal I seek. The thing is my app is kinda like manager-ware for developer who want to quickly go from idea to go-live so my script via cloud-init will spin up a proper ubuntu VPS starter and install their Django app from their github repo. Hence having a status bar of the steps taken to prepare their new VPS is my end goal. Of course, perhaps months down the road, i will look into optimizing the speed of init. but not now.
    – Kim Stacks
    Jun 11, 2022 at 2:04
  • 1
    at the very least, i will definitely put all my apt-get install modules all in 1 command for low hanging fruits!
    – Kim Stacks
    Jun 11, 2022 at 2:05
1

My suggestion comes from these pages:

Roughly it is:

  • Create a bootcmd ( https://cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot ) that polls until the network is up, then downloads and executes the next stage.
  • The second stage actively watches and processes log data (e.g. from: /var/log/cloud-init.log ) to evaluate where in the install process you are.
  • At that point, you can start sending updates using 'curl' or whatever tool makes the most sense as you see lines show up in the log files.

Potential challenges:

  • Requires programming.
  • Requires understanding the log files.

I suggest performing a few test installs and sending each line as it shows up (e.g. using 'curl') to get a sense what a 'normal' install looks like, finding some key points, and using those points to trigger your updates.

12
  • i'm confused by your answer. So are you essentially saying the same thing as @falcojr in the comment where i have to convert the cloud-init standard package_update package_upgrade etc into a giant bash script so i can call the curl before and after each step in the bash script?
    – Kim Stacks
    Jun 9, 2022 at 3:53
  • I'm not saying the same thing as @falcojr . I'm saying you can run a monitor script in_parallel_to the cloudinit process that reports on its progress by monitoring the log files and sending updates using curl. I like falcojr's 'reporting' idea, but I haven't seen enough documentation to know how useful it would be for your purpose / how to use it for your purpose. Jun 9, 2022 at 4:14
  • 🤔 on first thought I’m not keen on another parallel but I should not dismiss out of hand too quickly let me stew on it for another 24
    – Kim Stacks
    Jun 9, 2022 at 5:12
  • If you end up going down this route and want to consider prior art, genlop is a simple tool you might be interested in. The concepts (install packages vs a collection of services) don't match exactly, but who knows, it might help in some way. Jun 11, 2022 at 0:27
  • @BrettHolman when i google genlop i get so many hits about gentoo. I'm not too sure about a tool when it's that hard to find documentation for it. Might be because i am bad at googling though.
    – Kim Stacks
    Jun 11, 2022 at 1:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .