9

I want to use .sh script for deployment of my app. That script is on my home server (Ubuntu 15.10 Server), marked as executable. Access to this script is done through ssh, using this tutorial, I have set up ssh login, that runs that script. So basically I just call ssh deployer@XXX.com someArguments and it runs my script with someArguments as parameters. The user deployer has uid=0, so its basically root (this will be changed in the future, I have set it only to eliminate permissions problems until it works fine).

And here is where the things get tricky. The script reports /usr/bin/env: php: No such file or directory at command /bin/composer install (using Composer). Things are more weird the more I look on that script. Before this line, there is also called /bin/composer self-update and /bin/composer -V, which both runs correctly and displays correct output.

I have checked following things:

  • /usr/bin/env php -v displays correct PHP version (same as /usr/bin/php -v)
  • whereis php displays php: /usr/bin/php /usr/local/bin/php /usr/share/man/man1/php.1.gz
  • php5-cli package is installed and newest version
  • $PATH contains /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
  • which env displays /usr/bin/env

I have also tried following things:

  • running the script directly as bash deploy.sh under root (since it is same as that user) - works perfectly without errors
  • running failing commands directly - also perfectly without errors

So this seems to me as very specific case, why this command does not work. I spent 12 hours of debugging it and I am out of ideas here.

P.S.: Similar error (/usr/bin/env: node: No such file or directory) occurs when there is bower install (using Bower), but not when running npm install (using NPM).

  • Run sh deploy instead of bash deploy (maybe some bashism). How did you check the "following things"? I recommend to check them in the script, so that you can discover eventual overrides and sanitations of envs. – Giacomo Catenazzi Mar 26 '16 at 11:20
  • regarding "following things": I added them to start of deploy.sh script and they output these things I put down to question. Same output is when I run them alone, though. – Tomáš Blatný Mar 26 '16 at 11:27
  • sh deploy and bash deploy both gives same results – Tomáš Blatný Mar 26 '16 at 11:28
  • Show that line here, please. I recommend you to replace your php command at this line in your script with output to file to examine environment variables at the moment of calling /usr/bin/env: /usr/bin/env > environment.txt – Oleg Bolden Mar 29 '16 at 4:55
6

Make sure that line endings and/or invisible spaces aren't causing the problem.

Remove the spaces in the first line of the script and insert new ones, making sure not to hold CTRL while pressing space.

Also, make sure you don't have DOS line endings (CR+LF). See https://stackoverflow.com/questions/82726/convert-dos-line-endings-to-linux-line-endings-in-vim for details.

  • I am using IDE, which automatically checks (and converts) CR+LF to LF only, removes BOM and cares about white characters, but I double-checked it, and it appears ok (still not working though). Thanks anyway – Tomáš Blatný Mar 30 '16 at 6:29
4
+100

Easiest way.... change the user's shell as the script.

/etc/passwd

Before:
deploy:x:0:0:,,,:/root:/bin/bash

After:
deploy:x:0:0:,,,:/root:/scripts/deploy.sh

Example script (make sure execute bit is set chmod +x)

/scripts/deploy.sh

#!/bin/bash 
PATH=$PATH:/moo etc...
moo.sh

Sample Works everytime! You should even be able to use the script to troubleshoot/debug any env variables which you may feel are unset etc ... also the handling arguments being passed into ssh will work as well..

Note: Its Best Practice to always FULLY QUALIFY the paths for any scripts, executables, etc.. Above is just an example which allows the default path to be set to call moo.sh within the moo folder ;)

That was easy.. Thanks for posting..

Reference: /etc/passwd format

  • Nice answer, but I actually never use the user on server directly, I always ssh to server, and by lane in authorized_keys, the script gets called and then connection ends. How shall I modify authorized_keys to keep this working? – Tomáš Blatný Mar 30 '16 at 6:28
  • It should work the same. You would be authenticating via key and as long as the shell is set to the script for the remote account in question within the remote server's /etc/passwd file the script will execute. I'll add a screenshot to my post shortly. – NotAdmin Dave Mar 30 '16 at 14:02
  • 1
    @Dave can't wait for your book – Burgi Mar 30 '16 at 14:22
  • Thanks for your answer, it did not solve my problem, but was for me the most interesting and actually solved other problems I had. Gave you the bounty, thanks again – Tomáš Blatný Apr 3 '16 at 14:47
4

The env command will look through a user's $PATH to find the first executable of the given name. So, /usr/bin/env php will look for an executable file called php in any of the directories in the $PATH of the user running it.

In your case, that's almost certainly because when running a command over ssh, you don't start a full shell and don't actually read your shell's initialization files. You can check this by running this command (note the single quotes):

ssh deployer@XXX.com 'echo $PATH'

And comparing the output to what you get if you ssh deployer@XXX.com and then run echo $PATH. On my system. for example:

$ echo $PATH
/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:

$ ssh localhost 'echo $PATH'
/usr/bin:/bin:/usr/sbin:/sbin

Therefore, the $PATH that your script has access to when run with ssh deployer@XXX.com is not the same as when you log in to test it.

In any case, the simple solution is to use the full path to the interpreter instead of env. Both env and full paths have their benefits and drawbacks but, in this case, the path is safer:

#!/usr/bin/php
  • ITYM "note the single quotes" not "not". – dave_thompson_085 Mar 30 '16 at 18:25
  • @dave_thompson_085 indeed I did, thanks. – terdon Mar 30 '16 at 18:42
  • I am actually using full paths everywhere, as I mentioned in my question, the error is reported inside composer install command, which i obviously cannot modify. Also $PATH is ok, as I mentioned in my question too, I checked all the things by adding them into the script and running remotely via ssh login. Also I can not check the ssh deployer@XXX.com 'echo $PATH' as you stated, since my ssh login is limited to one script only, but I hovever did check it when I add $PATH to that script (stated above) Anyway, thank you for your answer, and accept +1 for explaining me the env thing – Tomáš Blatný Mar 30 '16 at 19:10
  • @TomášBlatný well, you're using env somewhere, or you wouldn't be seeing that error. I don't know composer but you will probably have to copy or link the php executable to a directory in its path. This sort of thing is hard to debug since there are so many things each depending on the other. – terdon Mar 30 '16 at 19:21
  • Thats true, env is actually called inside composer. But this does not solve the problem, why some composer calls pass, and some not. However I will futher investigate this by looking into its code. Thanks for your time – Tomáš Blatný Mar 30 '16 at 19:57
2

Is it possible that bash needs a reset to its hash table?

If so, you might try adding hash -r somewhere in your script, which forces the shell to look through $PATH again rather than relying on (possibly outdated) info from the hash table.

When needed, hash can also enable the shell to remember paths to executables installed in non-standard locations using the -p option, or forget paths with the -d option.

Sources:

https://unix.stackexchange.com/a/86017/121251
https://stackoverflow.com/a/22543353/2146843

  • This did not solved the problem for me, however its a good knowledge, so I added you a +1 – Tomáš Blatný Apr 1 '16 at 1:59
1

Looks like maybe you need to add php to your path. Try:

vim ~/.bashrc
PATH=$PATH:/usr/local/bin/php
export PATH

You might also want to check where your php lives to be sure that the path there is correct. Try:

which php
  • Well this is not the problem, since php -v outputs correct PHP version and composer --version outputs composer version. As I said, problem is in one command only. – Tomáš Blatný Mar 30 '16 at 6:39
1

It is clear that you have a path problem since your deploy script can't find things that are definitely on the path when you do a normal ssh login.

First thing to do to confirm you have a PATH issue is to update your deploy script to log the output of env or at least echo $PATH. I am guessing that the way your deploy script is called, $PATH is not set as you would expect. This debug output will confirm/deny my theory.

I looked at the tutorial you followed. You should probably make sure at update the command= to be command="/bin/sh /path/to/your/script..." if you haven't already to make sure your script is run by the right shell.

If you do have a PATH problem, a quick/dirty fix is just to explicitly set PATH at the beginning of your deploy script.

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"

Detailed explanation and further options...

On linux when commands are run they inherit the environment of their parent process.

When you login as a normal user via SSH there are things that happen (like running /etc/bashrc /etc/profile ~/.bash_profile ~/.bashrc etc). At that point you have may have updated your process's environment by doing things like export PATH="$PATH:~/mybin" in those scripts. Now any future processes that you run will inherit your current environment.

Running a command instead of getting a login shell means that command is run by the ssh daemon and will inherit the environment of the ssh daemon process...which is likely different than your environment as a logged in user.

The man page for authorized keys covers what happens after authentication. Regarding environment:

  1. Reads the file ~/.ssh/environment, if it exists, and users are allowed to change their environment. See the PermitUserEnvironment option in sshd_config(5).

So the appropriate place to configure environment for the process is in ~/.ssh/environment where ~ is the home directory for the user that is authenticated to run the command. You also need to check your sshd_config to make sure PermitUserEnvironment is allowed.

~/.ssh/environment format is also specified in the man page of course.

         This file is read into the environment at login (if it exists).
         It can only contain empty lines, comment lines (that start with
         '#'), and assignment lines of the form name=value.  The file
         should be writable only by the user; it need not be readable by
         directory becomes accessible.  This file should be writable only
         by the user, and need not be readable by anyone else.

An alternative way to specify the environment without using the above mentioned method is to use the environment="NAME=value" option in the authorized_keys file. See the man page I linked above for details.

  • Regarding Without knowing exactly how you have setup your deploy script to run: in my question at the begining, there is a link, how I set it up (using ~/.ssh/authorized_keys. Thanks for the tip with updating command, I tried it, but unfortunatelly with no difference. I will however futher investigate this and try different shells. Please accept a +1 for that idea. – Tomáš Blatný Mar 30 '16 at 19:15
  • Did you try updating your deploy script to print out the current $PATH? Could you post the result? If it doesn't match what you expect, then you can just try setting the PATH explicitly as I suggested at the beginning of your deploy script. – mattpr Mar 31 '16 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.