How To List, Count and Search Files in S3 Bucket with AWS Cli   

In this quick article, we are going to count number of files in S3 Bucket with AWS Cli.

Step 1: List all files from S3 Bucket with AWS Cli

To start let's see how to list all files in S3 bucket with AWS cli. This can be done by using ls method as:

aws s3 ls 's3://my_bucket/input/data'

results in:

file1.txt
file2.txt

Step 2: Count number of files in S3 Bucket

Next step is to count the number of the files in that bucket. This can be done by piping command - | wc -l:

aws s3 ls 's3://my_bucket/input/data' | wc -l

output:

657895

Step 3: Search files in S3 bucket based on name or pattern

Finally we are going to use a pattern or the whole name in order to perform a search in a bucket. For this purpose we are going to use command grep as follows:

aws s3 ls 's3://my_bucket/input/data' | grep 'keyword*'

output:

keyword.txt
keyword1.txt
keyword.csv

The search above will find and list any file from the bucket which starts with keyword. So if you like to find all CSV files from S3 bucket use the next:

aws s3 ls 's3://my_bucket/input/data' | grep '*.csv'

output:

keyword.csv