Having trouble downloading multiple S3 call recording files of your Amazon Connect contact center at once using AWS CLI on Windows machine?

The Problem:

Well, we all know as DevOps Engineers/Architects that we can use AWS CLI, copy or sync commands to download multiple S3 files at once. However, I faced a weird challenge when doing so from my windows machine using AWS CLI.

Let's see an example:

I wanted to download a file from my S3 bucket as below:

27XXXXXX-bcYYYYYY_20230112T20:19_UTC.wav

However, whenever I used sync or cp commands on my AWS CLI, the file name used to be re written with an appended string in hexadecimal format at the end of the file like below:

27XXXXXX-bcYYYYYY_20230112T20:19_UTC.wav.A7Fe4F15

And hence the error that came up was as below, on debugging:

OSError: [WinError 87] The parameter is incorrect.

Now, this is an error thrown out of OS, and not AWS CLI, to try it out I used EC2 on Linux OS, and no problem using cp or sync commands, file gets downloaded fine.

So, it was clear, its an OS issue not a AWS CLI or commands.

Where to next?

Well in my search, I came across AWS Storage gateway, a hybrid cloud storage service that gives you on-premises access to virtually unlimited cloud storage. Storage Gateway provides a standard set of storage protocols such as iSCSI, SMB, and NFS, which allow you to use AWS storage without rewriting your existing applications.

It struck me what if I can rely on SMB protocol for getting my S3 entirely as a Windows Local drive that would be amazing and can also use that folder as on prem syncing option. The cost is not super expensive, and is worth for the use case.

The Solution:

Steps to setup storage gateway with S3 on EC2

Pre-Requisites:

Created a IAM user, with programmatic access and save the access key and secret access key locally for that account, provide AWS admin access to the user.

Running a VPC with a public subnet, where you will launch your EC2 and security group allowing SMB traffic inbound/outbound rule restricted to your ip or your organization's ip.

• Create storage gateway config, head to aws storage gateway services.

• Create a new Key Pair, leave rest default and save it locally

• Once created launch the instance, with the created Keypair:

• Go Next
• Click Next and Activate:
• Ignore next step and leave default, click configure.
• Scroll to the top of the page again and wait for configuration to be completed, it can take few minutes(15-30mins)max.

• Once completed it will show the running state.
• Go to file share
• Configure the basics as below:

• Enter a new password, save this or remember this, as you will need this later.
• Create file share.
• Once completed you will see as below:

• If you get errors please wait for sometime and again refresh:
• Open AWS CLI and write the below commands to access the customer’s aws account using access key id and secret access key, of IAM user you have in your pre requisite.

• To create a new file share type the command in below format:

net use Z: \\Enter your Public IPV4 address of your EC2 \your bucket name will be shown here /user:sgw-XXXX\smbguest.

• Use Z as Windows file share directory, if already exists then change it to any new one. I am using Q as Z was in use.
• Once successful you will be prompted for password, enter your guest password you saved earlier when creating file shares. Once password is entered and if no error:

• Check your file shares and you must see the entire AWS S3 bucket on your local file share:

• Once you can see all files, move them to a directory where you want it to.
• Unmount the file share by deleting it, once your use case is completed.
• Don’t forget to delete the storage gateway, Ec2 instance, VPC, Subnets, SG, Keypairs to avoid any charges.

Summary:

In the above article, we saw how we can leverage AWS Storage gateway to access our entire S3 bucket storage, which stores call recordings, chat transcripts or any files of a contact center in bulk on local windows file share and can leverage on that data for further integrations or use case. I hope this article helps and provides useful insights for your use case.

Thanks :-)