In this quick article, we are going to count number of files in S3 Bucket with AWS Cli.
Step 1: List all files from S3 Bucket with AWS Cli
To start let's see how to list all files in S3 bucket with AWS cli. This can be done by using
ls method as:
aws s3 ls 's3://my_bucket/input/data'
Step 2: Count number of files in S3 Bucket
Next step is to count the number of the files in that bucket. This can be done by piping command -
| wc -l:
aws s3 ls 's3://my_bucket/input/data' | wc -l
Step 3: Search files in S3 bucket based on name or pattern
Finally we are going to use a pattern or the whole name in order to perform a search in a bucket. For this purpose we are going to use command
grep as follows:
aws s3 ls 's3://my_bucket/input/data' | grep 'keyword*'
The search above will find and list any file from the bucket which starts with
keyword. So if you like to find all CSV files from S3 bucket use the next:
aws s3 ls 's3://my_bucket/input/data' | grep '*.csv'