In this short tutorial, you'll see how to install s3cmd on Linux Mint or Ubuntu. You'll also see how to configure and use it.
s3cmd is a tool used in terminal or python code for creating s3 buckets, uploading, retrieving and managing data to Amazon s3 storage. It's a simple, free, fast and reliable way to work with Amazon S3 storage.
Step 1: Install s3cmd on Linux Mint from terminal
s3cmd
is available in default rpm repositories for Debian/Linux Mint/Ubuntu systems and also as a Python package. You can install s3cmd
using simply executing the following command on your system.
On Debian / Ubuntu / Linux mint::
sudo apt-get update
sudo apt-get install s3cmd
Later if you need to remove s3cmd
from your system you can use:
sudo apt-get remove s3cmd
Note: This installation method is the best option for Ubuntu Server. If you have a graphical interface then you can use the Software Center in Linux Mint.
Step 2: Install s3cmd on Linux Mint as Python package (optional)
An alternative s3cmd
can be installed as a Python package in the Python virtual environment. Once your environment is activated you can install s3cmd
package by:
pip install s3cmd
If you need to remove the python package s3cmd
then you can use:
pip uninstall s3cmd
Step 3: s3cmd basic environment configuration
Once the installation is completed, it's time to configure s3cmd
. This can be done by:
s3cmd --configure
This command configures next s3cmd values:
- Access Key
- Secret Key
- Default Region [US]
The result of this command is:
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key:
Secret Key:
Default Region [US]:
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]:
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]:
Step 4: How to use s3cmd in Linux Mint
List files
The first command will list all s3 buckets in current aws account.
s3cmd ls
If you need to list objects in specific bucket using different - secret_key
and access_key
then you can use:
s3cmd --secret_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --access_key=xxxxxxxxxxxxxxxxxxxxxx -s ls -l s3://mybucket/test-files/
parameters like -l
can be added to the command: ls
.
Download file
Downloading files from your s3 bucket by using s3cmd
is available by:
s3cmd --secret_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --access_key=xxxxxxxxxxxxxxxxxxxxxx get s3://my-bucket/my-images/test.csv
Recursive search
This is example of recursive search of s3cmd
s3cmd --secret_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --access_key=xxxxxxxxxxxxxxxxxxxxxx -s ls -l s3://my-bucket/my-images/ --recursive | grep my_photo | cut -c 32-
One more example of recursive search:
aws --secret_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx --access_key=xxxxxxxxxxxxxxxxxxxxxx s3 ls s3://my-bucket/my-images/ --recursive | grep my_photo | cut -c 32-
Searching for all text files with s3cmd
:
s3cmd --recursive ls s3://mybucket |
awk '{ print $4 }' | grep "*.txt"
Delete Data from Bucket
To delete a file test.csv
from s3 bucket - my-bucket - use following command.
s3cmd del s3://my-bucket/test.csv
Uploade file in S3 Bucket
Let's upload file test.csv
to my-bucket using the tool:
s3cmd put test.csv s3://my-bucket/