Hot questions for Using Amazon S3 in java 8

Question:

I want to expose an API to download a S3 bucket file content as stream to its consumers. The API URL is like /downloadfile/** which is GET request.

  1. What should be my return type right now I tried with accept header= application/octet-stream which didn't work.
  2. I don't want to write the content of file to any file and send it. It should be returned as a stream that's it.

Here is the controller pseudo code I wrote till now which is giving me 406 error all the time.

 @GetMapping(value = "/downloadfile/**", produces = { MediaType.APPLICATION_OCTET_STREAM_VALUE })
    public ResponseEntity<Object> downloadFile(HttpServletRequest request) {
       //reads the content from S3 bucket and returns a S3ObjectInputStream
       S3ObjectInputStream object = null;
       object = publishAmazonS3.getObject("12345bucket", "/logs/file1.log").getObjectContent();
       return object
    }

Any suggestions here on the way of doing this and what I am doing wrong?


Answer:

You can solve with the below example

import java.io.ByteArrayOutputStream;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import com.grokonez.s3.services.S3Services;

@RestController
public class DownloadFileController {

    @Autowired
    S3Services s3Services;

    /*
     * Download Files
     */
    @GetMapping("/api/file/{keyname}")
    public ResponseEntity<byte[]> downloadFile(@PathVariable String keyname) {
        ByteArrayOutputStream downloadInputStream = s3Services.downloadFile(keyname);

        return ResponseEntity.ok().contentType(contentType(keyname))
                .header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + keyname + "\"")
                .body(downloadInputStream.toByteArray());
    }

    private MediaType contentType(String keyname) {
        String[] arr = keyname.split("\\.");
        String type = arr[arr.length - 1];
        switch (type) {
        case "txt":
            return MediaType.TEXT_PLAIN;
        case "png":
            return MediaType.IMAGE_PNG;
        case "jpg":
            return MediaType.IMAGE_JPEG;
        default:
            return MediaType.APPLICATION_OCTET_STREAM;
        }
    }
}

Question:

JetS3t fails to get LastModifiedDate for existing S3 object in Java 8 but succeeds in Java 7.

package JsTest

import org.jets3t.service.S3Service;
import org.jets3t.service.ServiceException;
import org.jets3t.service.impl.rest.httpclient.RestS3Service;
import org.jets3t.service.model.S3Object;
import org.jets3t.service.security.AWSCredentials;

public class Jetsettester {

public static void main(String[] args) {
    String awsAccessKey = "<access_key>";
    String awsSecretKey = "<secret>";
    AWSCredentials awsCredentials = new AWSCredentials(awsAccessKey, awsSecretKey);
    S3Service s3Service = new RestS3Service(awsCredentials);
    try {
        S3Object objectDetailsOnly = (S3Object) s3Service.getObjectDetails("ENTER_BUCKET_NAME", "EXISTING_SAMPLE_KEY.TXT");
        Object md = objectDetailsOnly.getMetadata("Last-Modified"); // Object type is 'String' in Java 8 - and 'Date' in Java 7
        Date dt = objectDetailsOnly.getLastModifiedDate(); // Hence, raises ClassCastException only in Java 8 . In Java 7 - it is correct Date class
    } catch (ServiceException e) {
        e.printStackTrace();
    }
}
}
  1. Ensure a bucket and key exists in Amazon S3. Add it in the code below along with AWS secret and key
  2. Run the above code using Java 7 it runs successfully. More specifically, the return type of Object md is type String and return type Date dt is Date.
  3. Repeat with Java 8 it fails with a ClassCastException. More specifically, JetS3t is is returning a String date - instead of Date object.

I think there are two header values - one for 'Last-Modified' and other for lowercase 'last-modified' but I dont know more than that. Can someone help?


Answer:

This was a bug in JetS3t 0.9.3. I raised the bug and James Murty graciously fixed it last year in release 0.9.4 Posting it here in case anyone else experiences it. details bitbucket bug link for jets3t

Question:

I found this code to describe ec2 instances in AWS SDK for java. I need to convert this code snippet to Java 8 by using forEach() method. I know how to use forEach() method for iterate a single list. But I'm struggling with this code since it is nested.

 DescribeInstancesRequest request = new DescribeInstancesRequest();
    while(!done) {
        DescribeInstancesResult response = ec2.describeInstances(request);

        for(Reservation reservation : response.getReservations()) {
            for(Instance instance : reservation.getInstances()) {
                    String instanceID = instance.getInstanceId(),
                    return instanceID;                    
            }
        }

What I want is to iterate through all the instances and get the instanceID for further use by java 8. Can somebody give me a tip to fulfill my requirement? It will be highly appreciated.


Answer:

Here is the code using java 8 stream

DescribeInstanceResult response = ec2.describeInstances(request);

String instanceId = response.getReservations()
                            .stream()
                            .flatMap(reservation -> reservation.getInstances()
                                                               .stream())
                            .findFirst()
                            .orElse(null);