s3_object fails with boto3.exceptions.S3UploadFailedError because of permission issues

[alinabuzachis]

Summary

    - name: add object into bucket
      s3_object:
        bucket: "{{ s3_bucket_name }}"
        mode: put
        object: "{{ s3_bucket_object }}"
        # permission: public-read
        src: "{{ zip_file_path }}"

fails with

TASK [lambda_layer : add object into bucket] ***********************************
task path: /home/zuul/.ansible/collections/ansible_collections/amazon/aws/tests/integration/targets/lambda_layer/tasks/main.yml:43
<testhost> ESTABLISH LOCAL CONNECTION FOR USER: zuul
<testhost> EXEC /bin/sh -c 'echo ~zuul && sleep 0'
<testhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/zuul/.ansible/tmp `"&& mkdir "` echo /home/zuul/.ansible/tmp/ansible-tmp-1682701115.9609966-4404-98160544167301 `" && echo ansible-tmp-1682701115.9609966-4404-98160544167301="` echo /home/zuul/.ansible/tmp/ansible-tmp-1682701115.9609966-4404-98160544167301 `" ) && sleep 0'
<testhost> EXEC /bin/sh -c 'test -e /tmp/ansible.91p8zbor.lambda_handler/lambda_handler.zip && sleep 0'
Using module file /home/zuul/.ansible/collections/ansible_collections/amazon/aws/plugins/modules/s3_object.py
Pipelining is enabled.
<testhost> EXEC /bin/sh -c 'ANSIBLE_DEBUG_BOTOCORE_LOGS=True /home/zuul/venv/bin/python && sleep 0'
<testhost> EXEC /bin/sh -c 'rm -f -r /home/zuul/.ansible/tmp/ansible-tmp-1682701115.9609966-4404-98160544167301/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/home/zuul/venv/lib/python3.9/site-packages/boto3/s3/transfer.py", line 288, in upload_file
    future.result()
  File "/home/zuul/venv/lib/python3.9/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
  File "/home/zuul/venv/lib/python3.9/site-packages/s3transfer/futures.py", line 266, in result
    raise self._exception
  File "/home/zuul/venv/lib/python3.9/site-packages/s3transfer/tasks.py", line 139, in __call__
    return self._execute_main(kwargs)
  File "/home/zuul/venv/lib/python3.9/site-packages/s3transfer/tasks.py", line 162, in _execute_main
    return_value = self._main(**kwargs)
  File "/home/zuul/venv/lib/python3.9/site-packages/s3transfer/upload.py", line 758, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/home/zuul/venv/lib/python3.9/site-packages/botocore/client.py", line 415, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/zuul/venv/lib/python3.9/site-packages/botocore/client.py", line 745, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 121, in <module>
  File "<stdin>", line 113, in _ansiballz_main
  File "<stdin>", line 61, in invoke_module
  File "/usr/lib64/python3.9/runpy.py", line 225, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1507, in <module>
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1494, in main
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 1175, in s3_object_do_put
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py", line 740, in upload_s3file
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py", line 105, in deciding_wrapper
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
  File "/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
  File "/home/zuul/venv/lib/python3.9/site-packages/boto3/s3/inject.py", line 143, in upload_file
    return transfer.upload_file(
  File "/home/zuul/venv/lib/python3.9/site-packages/boto3/s3/transfer.py", line 294, in upload_file
    raise S3UploadFailedError(
boto3.exceptions.S3UploadFailedError: Failed to upload /tmp/ansible.91p8zbor.lambda_handler/lambda_handler.zip to ansible-test-76044453-np0003784303-bucket/ansible-test-76044453-np0003784303-obj-1: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs
fatal: [testhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/zuul/venv/lib/python3.9/site-packages/boto3/s3/transfer.py\", line 288, in upload_file\n    future.result()\n  File \"/home/zuul/venv/lib/python3.9/site-packages/s3transfer/futures.py\", line 103, in result\n    return self._coordinator.result()\n  File \"/home/zuul/venv/lib/python3.9/site-packages/s3transfer/futures.py\", line 266, in result\n    raise self._exception\n  File \"/home/zuul/venv/lib/python3.9/site-packages/s3transfer/tasks.py\", line 139, in __call__\n    return self._execute_main(kwargs)\n  File \"/home/zuul/venv/lib/python3.9/site-packages/s3transfer/tasks.py\", line 162, in _execute_main\n    return_value = self._main(**kwargs)\n  File \"/home/zuul/venv/lib/python3.9/site-packages/s3transfer/upload.py\", line 758, in _main\n    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)\n  File \"/home/zuul/venv/lib/python3.9/site-packages/botocore/client.py\", line 415, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n  File \"/home/zuul/venv/lib/python3.9/site-packages/botocore/client.py\", line 745, in _make_api_call\n    raise error_class(parsed_response, operation_name)\nbotocore.exceptions.ClientError: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"<stdin>\", line 121, in <module>\n  File \"<stdin>\", line 113, in _ansiballz_main\n  File \"<stdin>\", line 61, in invoke_module\n  File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py\", line 1507, in <module>\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py\", line 1494, in main\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py\", line 1175, in s3_object_do_put\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_object.py\", line 740, in upload_s3file\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py\", line 105, in deciding_wrapper\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py\", line 119, in _retry_wrapper\n  File \"/tmp/ansible_s3_object_payload_5hulw7o5/ansible_s3_object_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py\", line 68, in _retry_func\n  File \"/home/zuul/venv/lib/python3.9/site-packages/boto3/s3/inject.py\", line 143, in upload_file\n    return transfer.upload_file(\n  File \"/home/zuul/venv/lib/python3.9/site-packages/boto3/s3/transfer.py\", line 294, in upload_file\n    raise S3UploadFailedError(\nboto3.exceptions.S3UploadFailedError: Failed to upload /tmp/ansible.91p8zbor.lambda_handler/lambda_handler.zip to ansible-test-76044453-np0003784303-bucket/ansible-test-76044453-np0003784303-obj-1: An error occurred (AccessControlListNotSupported) when calling the PutObject operation: The bucket does not allow ACLs\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Issue Type

CI Bug Report

CI Jobs

No response

Pull Request

No response

Additional Information

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[Tyler-2]

Wasn't this fixed by 86b20e9
?

I can't figure out what happened to this...

[tremble]

@Tyler-2

"It's complicated". There are actually multiple issues at play here.

• In this specific case the test was explicitly trying to upload using "public-read", when Amazon no longer permit this by default. It was completely appropriate for the error to be thrown here, and this was actually the upload failing in its entirety. This wasn't "ACLs disabled", this was "requested ACL conflicts with current policy". (There's a little more info about this changed behaviour in s3_bucket - Ensure public_access settings are configured before policies #1511 which was exposed by Amazon's new behaviour.
• S3_bucket : Handle setting of permissions while acl is disabled #1168 fixed the issue when you're trying to change permissions (or more importantly when you're not trying to change them but they get read anyway).
• In the case of PutObject boto3 actually catches the exception itself and re-raises it as a different exception (there's a handful of special cases where boto3 re-raises exceptions). This meant the exception wasn't caught at all, and is why it's not quite so obvious what's going on. This was fixed as part of s3_object - fix regression related to leading / in object key names #1549.

IMO This specific issue should be closed "WONTFIX" or "NOTABUG":

• The file doesn't need to be uploaded as publicly readable
• Failing to explictly set "public-read" when uploading to a bucket that doesn't allow public access is, IMHO valid behaviour.

@jillr @alinabuzachis - thoughts?

[ansibullbot]

@alinabuzachis: Greetings! Thanks for taking the time to open this issue. In order for the community to handle your issue effectively, we need a bit more information.

Here are the items we could not find in your description:

• ansible version
• component name

Please set the description of this issue with this template:
https://raw.githubusercontent.com/ansible/ansible/devel/.github/ISSUE_TEMPLATE/bug_report.md

click here for bot help