I'm trying to get AWS SAM local to work with Python and I'm running into what I suspect are Docker-related issues.

The commands I'm running in my Fish shell (full trascript here) are:

$ mkdir sam-helloworld
$ cd sam-helloworld/
$ virtualenv venv
$ source venv/bin/activate.fish
$ pip install aws-sam-cli
$ sam init --runtime python3.7
$ cd sam-app
$ sam build
$ sam local invoke HelloWorldFunction --event events/event.json

The last command fails with the following message:

Invoking app.lambda_handler (python3.7)
2019-09-15 12:11:16 Found credentials in AWS_CREDENTIAL_FILE.

Fetching lambci/lambda:python3.7 Docker container image......
Mounting /home/evan/Downloads/sam-helloworld/sam-app/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container
Traceback (most recent call last):
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/docker/api/client.py", line 261, in _raise_for_status
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http+docker://localhost/v1.35/containers/51af2a1c32718a46e283c28e18445706d22be43989a7da476a2798e2b168f191/start

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/evan/Downloads/sam-helloworld/venv/bin/sam", line 10, in <module>
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 94, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/lib/telemetry/metrics.py", line 65, in wrapped
    return_value = func(*args, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/commands/local/invoke/cli.py", line 58, in cli
    parameter_overrides)  # pragma: no cover
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/commands/local/invoke/cli.py", line 102, in do_cli
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/commands/local/lib/local_lambda.py", line 93, in invoke
    self.local_runtime.invoke(config, event, debug_context=self.debug_context, stdout=stdout, stderr=stderr)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/local/lambdafn/runtime.py", line 86, in invoke
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/local/docker/manager.py", line 98, in run
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/samcli/local/docker/container.py", line 189, in start
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/docker/models/containers.py", line 400, in start
    return self.client.api.start(self.id, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/docker/api/container.py", line 1093, in start
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/home/evan/Downloads/sam-helloworld/venv/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 400 Client Error: Bad Request ("OCI runtime create failed: container_linux.go:345: starting container process caused "chdir to cwd (\"/var/task\") set in config.json failed: permission denied": unknown")

I am running Arch Linux. I've confirmed through the groups command that I am in the docker group (and have rebooted already). In particular, the commands docker run hello-world and docker run -it ubuntu /bin/bash work as expected, so I think the issue is related to SAM somehow.

Running docker info gives the following data:

 Debug Mode: false

 Containers: 15
  Running: 1
  Paused: 0
  Stopped: 14
 Images: 26
 Server Version: 19.03.2-ce
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d50db0a42053864a270f648048f9a8b4f24eced3.m
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
   Profile: default
 Kernel Version: 5.2.14-arch2-1-ARCH
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.651GiB
 Name: ArchScythe
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
 Live Restore Enabled: false

Things I've tried include changing the permissions on /var/run/docker.sock, running everything as root (sorry), adding the -v flag to SAM, installing SAM through LinuxBrew instead of pip, among other things. Still no luck. Any idea what I'm doing wrong?

(I wasn't sure whether to post this here or on Stack Overflow; moderators should feel free to migrate if SO is more appropriate.)


Many hours later --- it turns out to be a permissions issue. AWS SAM is trying to mount the .aws-sam/build directory into the /var/task directory inside the container, but the docker user doesn't have the permissions to read the mounted directory since in my home directory files have no permissions for other by default.

To hack my way around this I executed

find .aws-sam/build/ -type d -print0 | xargs -0 chmod o+rx
find .aws-sam/build/ -type f -print0 | xargs -0 chmod o+r

after running sam build. This exposes the build artifacts and after that the command sam local invoke HelloWorldFunction --event events/event.json works as expected.

Although at least I can get back to work now, this is somewhat of a hack so I'm not entirely satisfied. If someone else can figure out a better way to deal with the issue, I'll accept that answer instead.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.