How to Connect to an S3 Bucket Using Java with Spring Boot

Amazon S3 (Simple Storage Service) is a highly scalable and secure object storage service that enables you to store and retrieve any amount of data, anytime, from anywhere on the web. When developing a Spring Boot application, you might often need to interact with Amazon S3 to manage files such as images, documents, or backups. In this post, we’ll walk you through how to connect to an S3 bucket using Java with Spring Boot without creating a Maven project from scratch. We'll assume you already have Spring Boot set up in your project.

Prerequisites

Before you begin, make sure you have:

1. AWS Account: Ensure you have an AWS account with access to Amazon S3.
2. AWS Access Keys: You’ll need your AWS Access Key ID and AWS Secret Access Key to authenticate requests.
3. Spring Boot Project: Ensure you already have a Spring Boot application set up. If you don't have one, you can quickly generate a Spring Boot app using the Spring Initializr.
4. AWS SDK for Java: This is required to interact with S3.

Step 1: Add AWS SDK Dependency to pom.xml

First, you need to include the AWS SDK for Java in your Spring Boot application to interact with Amazon S3.

Add the following dependency to your pom.xml file:

<dependencies>
    <!-- Spring Boot Starter Web for building web applications -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    
    <!-- AWS SDK for S3 -->
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>s3</artifactId>
        <version>2.17.186</version> <!-- Check for the latest version -->
    </dependency>

    <!-- Spring Boot DevTools for easier development -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-devtools</artifactId>
        <scope>runtime</scope>
    </dependency>

    <!-- Optional: Lombok for reducing boilerplate code -->
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.22</version>
        <scope>provided</scope>
    </dependency>
</dependencies>        

These dependencies include the necessary libraries to interact with Amazon S3 and enable basic Spring Boot functionalities like web support.

Step 2: Configure AWS Credentials

In order to authenticate requests to AWS, your Spring Boot application needs to have the correct credentials. AWS offers several methods to provide credentials:

1. Environment Variables: Set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as environment variables.
2. AWS Credentials File: If you’ve configured the AWS CLI, the credentials are typically stored at ~/.aws/credentials. Ensure that you have these credentials properly set up.
3. IAM Role: If you're running your Spring Boot application on an EC2 instance, you can assign an IAM Role with S3 access. The AWS SDK will automatically use the IAM role's credentials.

For simplicity, we'll use the DefaultCredentialsProvider in the SDK, which checks the environment variables, system properties, and the credentials file.

Step 3: Configure Your AWS S3 Client in Spring Boot

Now, let's configure the S3Client in your Spring Boot application.

3.1 Create a Configuration Class for S3 Client

It's best practice to define the configuration for your S3 client in a separate configuration class to maintain modularity and ease of use.

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;

@Configuration
public class S3Config {

    @Bean
    public S3Client s3Client() {
        // Set the region of your S3 bucket (e.g., us-west-2, eu-central-1)
        Region region = Region.US_EAST_1; // Modify this to match your bucket's region
        
        // Create an S3 client with the default credentials provider
        return S3Client.builder()
                .region(region)
                .credentialsProvider(DefaultCredentialsProvider.create())  // Automatically uses credentials from environment
                .build();
    }
}        

This S3Config class defines a Spring bean for the S3Client which can be injected wherever you need it in your Spring Boot application. The DefaultCredentialsProvider.create() ensures that the AWS SDK picks up your credentials from the environment or the credentials file automatically.

3.2 Inject the S3Client Bean into Your Service

Now that the configuration is set, we can inject the S3Client bean into any Spring service to interact with S3.

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;

import java.util.List;

@Service
public class S3Service {

    private final S3Client s3Client;

    @Autowired
    public S3Service(S3Client s3Client) {
        this.s3Client = s3Client;
    }

    // Method to list all S3 buckets in your AWS account
    public void listBuckets() {
        try {
            ListBucketsRequest listBucketsRequest = ListBucketsRequest.builder().build();
            ListBucketsResponse listBucketsResponse = s3Client.listBuckets(listBucketsRequest);
            List<Bucket> buckets = listBucketsResponse.buckets();
            
            if (buckets.isEmpty()) {
                System.out.println("No buckets found.");
            } else {
                System.out.println("Your S3 Buckets:");
                buckets.forEach(bucket -> System.out.println(bucket.name()));
            }
        } catch (S3Exception e) {
            System.err.println("Error listing buckets: " + e.awsErrorDetails().errorMessage());
        }
    }
}        

In this service class, we have a method to list all the buckets in your AWS account. The S3Service class is marked with @Service, making it a Spring bean that can be injected into other parts of the application.

Step 4: Calling the Service to List S3 Buckets

Now that the service is ready, you can inject it into a REST controller (or run it in the main method for testing). Below is an example of how to call the service from a REST endpoint.

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class S3Controller {

    private final S3Service s3Service;

    @Autowired
    public S3Controller(S3Service s3Service) {
        this.s3Service = s3Service;
    }

    @GetMapping("/list-buckets")
    public void listBuckets() {
        s3Service.listBuckets();
    }
}        

Step 5: Running Your Application

• Ensure AWS Credentials are Set: Make sure your AWS access keys are set in the environment variables or in the ~/.aws/credentials file.
• Run the Application: You can now run your Spring Boot application. If you’re using Spring Boot DevTools, it will automatically reload any changes. You can run the application using:

./mvnw spring-boot:run        

• Test the Endpoint: Once the application is running, navigate to https://localhost:8080/list-buckets to list all your S3 buckets. You should see the list of buckets in your AWS account printed in the console or returned as a response depending on your implementation.

Conclusion

In this blog post, we demonstrated how to connect to an Amazon S3 bucket using Java and Spring Boot. We walked through:

• Adding the AWS SDK for Java dependencies to your project.
• Configuring an S3Client bean in Spring Boot.
• Writing a service to interact with S3.
• Using the service in a REST controller to list S3 buckets.

With this foundation, you can extend the functionality to upload, download, and manage files in your S3 buckets. Integrating Amazon S3 with Spring Boot provides a seamless way to manage cloud storage in your applications.