Hot questions for Using Amazon S3 in jclouds

Question:

I need to update the cache control header in all AmazonS3's Cloud Files. However, I can't figure out how I do that using the jclouds API. I'm using apache jclouds plugin. And I got two related answers:

  • jclouds : how do I update metadata for an existing blob?
  • Set Expires header for an existing S3 object using AWS Java SDK

The first answer is suggesting to use SwiftKey Api class which is not available in grails's jcloud plugin. The second answer is using AWS java sdk for which there is already a grails wrapping plugin https://grails.org/plugin/aws-sdk but it doesn't support metadata update.


Answer:

It is possible to change the metadata by performing an object copy (see How to update metadata using Amazon S3 SDK):

ObjectMetadata metadataCopy = new ObjectMetadata();
// copy previous metadata
metadataCopy.addUserMetadata("newmetadata", "newmetadatavalue");

CopyObjectRequest request = new CopyObjectRequest(bucketName, existingKey, bucketName, existingKey)
      .withNewObjectMetadata(metadataCopy);

amazonS3Client.copyObject(request);

Whether this is philosophically an "update" is up to you to decide.

Question:

Normally, when I'm using the AWS S3 Java SDK, I can enable path style access in the following way:

S3ClientOptions clientOptions = new S3ClientOptions()
clientOptions.setPathStyleAccess(true)
awsS3Client.setS3ClientOptions(clientOptions)

We are transitioning our code to use jclouds instead, but I can't find any documentation that indicates either how to specify client options, or setting path style access for the Blob or BlobStore.

Any help would be much appreciated.


Answer:

PROPERTY_S3_VIRTUAL_HOST_BUCKETS controls path-style access, configured as part of ContextBuilder.overrides. The generic S3 API defaults to false, or path-style access, while the specific AWS-S3 provider defaults to true, or host-style access.

Question:

Hi I try to implement a method removeValuesUnusedLongerThan(long minimumAge, TimeUnit unit) that mean I want to delete every unused blob older than minimumAge inside my S3 container. But I don't find which method to use in my BlobStore object to achieve this.

Does JClouds provide a such feature ?


Answer:

jclouds does not provide a utility method for this but you can roll your own:

String marker = null;
while (true) {
    PageSet<StorageMetadata> set = blobStore.list(containerName,
            new ListContainerOptions().afterMarker(marker);
    for (StorageMetadata sm : set) {
        if (sm.getCreateDate() < expiryDate) {
            blobStore.removeBlob(containerName, sm.getName());
        }
    }
    marker = set.getNextMarker();
    if (marker == null) {
        break;
    }
}