Hot questions for Using Amazon S3 in android

Question:

Currently In my android project I am using GeneratePresignedUrl to get the link of a private file from Amazon s3. it worked fine few times on main thread, after it started giving NetworkOnMainThreadException. My question is does GeneratePresignedUrl needs Asynctask? Or is it a bug ?

Android Aws sdk version 2.2.20 (new version).

Code: from Util.java

public static URL getSignedUrl(Context context,String imageString){
    URL url=null;

           try {
               System.out.println("Generating pre-signed URL.");
               java.util.Date expiration = new java.util.Date();
               long milliSeconds = expiration.getTime();
               milliSeconds += 1000 * 60 * 60; // Add 1 hour.
               expiration.setTime(milliSeconds);
                sGenerateSignedUrl = new GeneratePresignedUrlRequest(Constants.BUCKET, imageString);
                sGenerateSignedUrl.setMethod(HttpMethod.GET);
                sGenerateSignedUrl.setExpiration(expiration);
                url = getS3Client(context.getApplicationContext()).generatePresignedUrl(sGenerateSignedUrl);


                System.out.println("Pre-Signed URL = " + url.toString());
               //if(url!=null)
               return url;
           }catch (AmazonServiceException exception) {
                System.out.println("Caught an AmazonServiceException, " +
                        "which means your request made it " +
                        "to Amazon S3, but was rejected with an error response " +
                        "for some reason.");
                System.out.println("Error Message: " + exception.getMessage());
                System.out.println("HTTP  Code: " + exception.getStatusCode());
                System.out.println("AWS Error Code:" + exception.getErrorCode());
                System.out.println("Error Type:    " + exception.getErrorType());
                System.out.println("Request ID:    " + exception.getRequestId());
            } catch (AmazonClientException ace) {
                System.out.println("Caught an AmazonClientException, " +
                        "which means the client encountered " +
                        "an internal error while trying to communicate" +
                        " with S3, " +
                        "such as not being able to access the network.");
                System.out.println("Error Message: " + ace.getMessage());
            }


    return url;
}

Calling Above method like this.

URL url = Util.getSignedUrl(getContext().getApplicationContext(),"image path string");

It worked really fine 20+ times. And later started reporting NetworkOnMainThreadException.


Answer:

AmazonS3.generatePresignedUrl itself doesn't make network request. However to create a presigned Url, it needs credentials from AmazonS3 client, and credentials are provided from CognitoCacheingCredentialsProvider which makes a network calls to STS and Cognito Identity Service. That's the cause of NetworkOnMainThreadException. If credentials are cached from previous calls, then such exception won't be thrown. Hope this explains.

Question:

I am working on Chat Application, we have two Client one is Android and another is web,I am uploading Media files to S3-Amazon,when I am sending media file from Web App to android client Media file are not downloaded showing error as bellow. Media Download interrupted : com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: XXXXXXXXX), S3 Extended Request ID:XXXXXXXXXXX

private void beginDownload(String key, String bucket, String 
 mediaType,final 
DownloadFileFromAwsCompletionListener listener) {
    // Location to download files from S3 to. You can choose any 
accessible
    // file.

    String localFilePath = Strings.EMPTY;
    try {
        //if (!isThumb) {
        localFilePath = MediaHelper.createMediaFile(mediaType, false, false, key);
       /* } else {
            localFilePath = MediaHelper.createMediaFile(mediaType, false, true);
        }*/
    } catch (Exception e) {
        e.printStackTrace();
    }
    if (!StringHelper.isNullOrEmpty(localFilePath)) {
        File file = new File(localFilePath);

        // Initiate the download
        TransferObserver observer = mTransferUtility.download(bucket, key, file);
        final String finalLocalFilePath = localFilePath;
        observer.setTransferListener(new TransferListener() {
            @Override
            public void onStateChanged(int id, TransferState state) {

                //String bucketPath = UrlStrings.XmppStrings.
                if (state.equals(TransferState.COMPLETED)) {
                    listener.onDownloadSuccess(finalLocalFilePath);
                }

            }

            @Override
            public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {

            }

            @Override
            public void onError(int id, Exception ex) {
                listener.onDatabaseError(new AwsFailure(ex));
            }
        });
    } else {
        getLogger().log(Strings.TAG, "xmpp beginDownload(): file could not be created.");
    }
}

Answer:

You should cross check the uploaded path with which path you are using for downloading media files from S3. I think You are using the different path for downloading media files That's why you getting an error.

Question:

If I have a S3Object called s3Object and I want to get its metadata like so:

ObjectMetadata objectMetadata = s3Object.getObjectMetadata();

does the call to this function trigger a download of the object from S3? Or is only the metadata downloaded? I'm trying to estimate how much data consumption over WiFi or 3G will result from this function call (I'm guessing its on the order of a few bytes if it's only getting the metadata, but obviously much more if it has to download the object).


Answer:

Calling .getObjectMetadata makes a HEAD call. It isn't a full GET- in other words, it doesn't get the object itself.

Question:

I need to make my upload as public. I was able to upload a image into s3 bucket. But i was unable to make that public by programmatically I am using AWS SDK 2.1.10 Below is my code for uploading image into s3 bucket

mUpload = getTransferManager().upload( AmazonConstants.BUCKET_NAME.toLowerCase(Locale.US), /* getPrefix(getContext()) // "android_uploads/" */ locationPath + super.getFileName() /** + "." + mExtension */ , mFile); 

mUpload.waitForCompletion();

Please help me. Thank you.


Answer:

You need to use PutObjectRequest in upload().

getTransferManager().upload(
    new PutObjectRequest(String bucketName, String key, File file)
   .withCannedAcl(CannedAccessControlList.PublicRead)
); 

Question:

I am using the Amazon S3 service Android API to upload an image to the server. When I use their code to download the image again, the method parameters requires me to pass a File to save this image to, but I would like to upload this image to an image view. How am I supposed convert this to a drawable?

this is the method:

TransferObserver observer = transferUtility.download(
MY_BUCKET,     /* The bucket to download from */
OBJECT_KEY,    /* The key for the object to download */
MY_FILE        /* The file to download the object to */
);

Answer:

So you need to create a java.io.File on the system. You can do something like this:

File outputDir = context.getCacheDir();
File imageFile = File.createTempFile("prefix", "extension", outputDir);

Then you can call the amazon s3 code:

TransferObserver observer = transferUtility.download(
MY_BUCKET,     /* The bucket to download from */
OBJECT_KEY,    /* The key for the object to download */
MY_FILE        /* The file to download the object to */
);

Make sure to create your TransferListener as well so you know when the file has been downloaded.

Then you can convert the file to a bitmap like this:

Bitmap bitmap = BitmapFactory.decodeFile(imageFile.getAbsolutePath());
mImageView.setImageBitmap(bitmap);

You can also use something like Picasso or other image processing libraries to help as well. They just require the ability to work with a file since that is what S3 will save too.

Question:

How would I transform this async call to use RX Android and have the ability to chain it to other calls? Currently I use EventBus to inform that sending succeeded or failed, but I would like to use RX, mostly for chaining purposes.

public void SendToAmazon(Intent i) {
    try {
        // take params from intent
        // .......................

        // Create an S3 client
        s3Client = new AmazonS3Client(new BasicAWSCredentials(MY_ACCESS_KEY_ID, MY_SECRET_KEY));
        // Set the region of your S3 bucket
        s3Client.setRegion(com.amazonaws.regions.Region.getRegion(Regions.EU_CENTRAL_1));

        String newFilename = fileName+"/img.jpg";
        PutObjectRequest putObjectRequest = new PutObjectRequest(
                AmazonS3FetchParams.MY_BUCKET_NAME, newFilename, new java.io.File( fileURI) )
                .withCannedAcl(CannedAccessControlList.PublicRead)
                .withStorageClass(StorageClass.Standard);
        try {
            putObjectRequest.setGeneralProgressListener(new S3ProgressListener() {
                @Override
                public void onPersistableTransfer(PersistableTransfer persistableTransfer) {

                }

                @Override
                public void progressChanged(com.amazonaws.event.ProgressEvent progressEvent) {
                    if (progressEvent.getEventCode() == ProgressEvent.COMPLETED_EVENT_CODE) {
                        Log.d("Info", "TRANSFER finished: " + progressEvent.getBytesTransferred() + " bytes");
                        SentToS3Event sentEvent = new SentToS3Event(fileName, fileURI, position,
                                0, from, nrRetry, messageDbId, getAmazonImageUrl(fileName));
                        EventBus.getDefault().post(sentEvent);
                    } else if (progressEvent.getEventCode() == ProgressEvent.FAILED_EVENT_CODE) {
                        SentToS3Event sentEvent = new SentToS3Event(fileName, fileURI, position,
                                -1, from, nrRetry, messageDbId, getAmazonImageUrl(fileName));
                        EventBus.getDefault().post(sentEvent);
                    }
                }
            });

            PutObjectResult objectResult = s3Client.putObject(putObjectRequest);
            error = 0;
        } catch (AmazonClientException aex) {
            Log.e("Info", "AMAZON EXCEPTION ! " + aex.getMessage());
        }
    } catch (Exception e) {
        error = -1;
    }
}

`


Answer:

Take a look at this piece of code. I managed to make it work in another situation.

    final PutObjectRequest putObjectRequest = new PutObjectRequest("a", "b", "c");
    Observable.create(new Observable.OnSubscribe<SomeAmazonModel>() {
        @Override
        public void call(final Subscriber<? super SomeAmazonModel> subscriber) {
            putObjectRequest.setGeneralProgressListener(new S3ProgressListener() {
                @Override
                public void onPersistableTransfer(PersistableTransfer persistableTransfer) {

                }

                @Override
                public void progressChanged(com.amazonaws.event.ProgressEvent progressEvent) {
                    if (progressEvent.getEventCode() == ProgressEvent.COMPLETED_EVENT_CODE) {
                        subscriber.onNext(new SomeAmazonModel(ProgressEvent.COMPLETED_EVENT_CODE));
                        subscriber.onCompleted();
                    } else if (progressEvent.getEventCode() == ProgressEvent.FAILED_EVENT_CODE) {
                        subscriber.onNext(new SomeAmazonModel(ProgressEvent.COMPLETED_EVENT_CODE));
                    }
                }
            });
        }
    }).subscribe(new Action1<SomeAmazonModel>() {
        @Override
        public void call(SomeAmazonModel someAmazonModel) {
            // Subscribe to above Observable and handle result
            someAmazonModel.getEventCode();
        }
    });

Question:

I am trying to upload Image via Retrofit to S3 via Spring. So, it's basically 2 tasks.

  1. Pass Image to Spring API
  2. From Spring API upload it to S3 Server.

I know how to upload the Image to S3 and it works fine.

My Problem is the first part. I have tried many solutions on net but didn't work.Here's my code ->

On Android side, first I am Getting file and Uploading it to Spring API ->

private void UploadFiles() {
        File uploadFile = fileArrayList.get(0);
        if (uploadFile != null) {
            Log.d(TAG, "UploadFiles: File Name is -> " + uploadFile.getName());
            RequestBody requestFile = RequestBody.create(MediaType.parse("image/*"), uploadFile);
            MultipartBody.Part multipartBody = MultipartBody.Part.createFormData("image", uploadFile.getName(), requestFile);
            cropImageRequest.setCropId(uploadFile.getParent());
            cropImageRequest.setCropImage(multipartBody);
            FarmerResponse farmerResponse = UserAuth.getInstance(this).getCurrentUser();
            Api.uploadCropImage(farmerResponse.getAuthToken(), farmerResponse.getFarmer().getId(), cropImageRequest, new Callback<BasicResponse>() {
                @Override
                public void onResponse(Call<BasicResponse> call, Response<BasicResponse> response) {
                    Log.d(TAG, "onResponse: Success : " + response.toString());
                }

                @Override
                public void onFailure(Call<BasicResponse> call, Throwable t) {
                    Log.d(TAG, "onResponse: Failure : " + t.getMessage());
                }
            });
        }
    }

I have created a cropImageRequest.java which is basically a POJO class for Request Body. Here's my POJO class, it contains Image and ImageId. ->

import okhttp3.MultipartBody;

public class CropImageRequest {

    private String CropId;
    private MultipartBody.Part CropImage;

    public CropImageRequest() {

    }

    public void setCropId(String cropId) {
        CropId = cropId;
    }

    public void setCropImage(MultipartBody.Part cropImage) {
        CropImage = cropImage;
    }

    public CropImageRequest(String cropId, MultipartBody.Part cropImage) {
        CropId = cropId;
        CropImage = cropImage;
    }

    public String getCropId() {
        return CropId;
    }

    public MultipartBody.Part getCropImage() {
        return CropImage;
    }
}

This is my API declaration, inside API Class. ->

public interface UploadCropImageApi {
        @POST(UPLOAD_FILE_TO_AWS_URL)
        Call<BasicResponse> uploadCropImage(
                @Header(MOBILE_AUTH) String authToken,
                @Header(FARMER_ID) String farmerId,
                @Body CropImageRequest cropImageRequest);
    }
        public static void uploadCropImage(String authToken, String farmerId, CropImageRequest cropImageRequest, Callback<BasicResponse> callback) {
                UploadCropImageApi uploadCropImageApi = retrofit.create(UploadCropImageApi.class);
                Call<BasicResponse> call = uploadCropImageApi.uploadCropImage(authToken, farmerId, cropImageRequest);
                call.enqueue(callback);
    }

On Spring side I have developed a controller, which handles API Request and Uploads Passed Image to S3 ->

@RequestMapping(value = "/UploadCropImage",method = RequestMethod.POST,produces = "application/json")
    @ResponseBody
    public String UploadImage(@RequestBody CropImageRequest cropImageRequest,HttpServletRequest request) {
        uuid = new UUIDUtils();
        fileUtils = new FileUtils();
        try {
            String KeyName = uuid.GenerateUUID(fileUtils.convert(cropImageRequest.getCropImage()));
            System.out.println(KeyName);
            return mAmazonClient.uploadImage(KeyName,cropImageRequest.getCropImage());
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            return null;
        }
    }

This is my Convert function which Converts MultiPart File into File.

public String convert(MultipartFile file) throws IOException {
        if(file!=null){
            File convFile = new File(file.getOriginalFilename());
            convFile.createNewFile();
            FileOutputStream fos = new FileOutputStream(convFile);
            fos.write(file.getBytes());
            fos.close();
            return convFile.getName();
        }
        else{
            System.out.println("Supplied File is null");
            return null;
        }       
    }

The Problem is that I am getting Null Pointer Exception, at Spring side, conditional check. Which means The File is not getting uploaded. On Android Side I have verified by printing out the file name and it's printing correct file name. BTW I am uploading .WebP format file, I dont think that's creating any issue here. Thanks for help in advance.


Answer:

you are sending your information using multipart/form-data so try to use

@ModelAttribute

instead of

@RequestBody

the method signature will be

public String UploadImage(@ModelAttribute CropImageRequest cropImageRequest,HttpServletRequest request)

Question:

I am uploading image file to S3 via the AWS java SDK, but i am getting the below error while uploading , i have checked my credentials and my bucket they are fine

Unable to calculate MD5 hash: /data/user/0/competent.groove.feetport/files/data/user/0/competent.groove.feetport/app_imageDir/IMG_20161122_073058.jpg: open failed: ENOENT (No such file or directory) 

I am capturing the image from camera and saving the image in phone , here is my code

 @Override
        public void onPictureTaken(CameraView cameraView, final byte[] data) {
            Log.d(TAG, "onPictureTaken " + data.length);
            //Toast.makeText(cameraView.getContext(), R.string.picture_taken, Toast.LENGTH_SHORT) .show();
            getBackgroundHandler().post(new Runnable() {
                @Override
                public void run() {
                    // This demo app saves the taken picture to a constant file.
                    // $ adb pull /sdcard/Android/data/com.google.android.cameraview.demo/files/Pictures/picture.jpg
                    ///storage/7A52-13E8/Android/data/competent.groove.feetport/files/Pictures/picture.jpg
                    //make a new picture file
                    File pictureFile = getOutputMediaFile();

                    if (pictureFile == null) {
                        Toast toast = Toast.makeText(getActivity(), "Picture Not saved", Toast.LENGTH_LONG);
                        toast.show();
                        return;
                    }
                    try {
                        //write the file
                        FileOutputStream fos = new FileOutputStream(pictureFile);
                        fos.write(data);
                        fos.close();
                        Toast toast = Toast.makeText(getActivity(), "Picture saved: " + pictureFile.getName(), Toast.LENGTH_LONG);
                        toast.show();

                        Log.e("-pictureFile--length-----",""+pictureFile.length());
                        Log.e("---pictureFile-----"+pictureFile.getName(),""+pictureFile.getAbsolutePath());
                        Log.e("-fileName--",""+fileName);

                        editor = sharedPref.edit();
                        editor.putString(Constants.PIC_PATH,""+fileName);
                        editor.commit();


                    } catch (FileNotFoundException e) {
                    } catch (IOException e) {
                    }



                }
            });
        }

    };




String fileName="";
    //make picture and save to a folder
    private  File getOutputMediaFile() {
        //make a new file directory inside the "sdcard" folder
       /* File mediaStorageDir = new File("/sdcard/", "JCG Camera");

        //if this "JCGCamera folder does not exist
        if (!mediaStorageDir.exists()) {
            //if you cannot make this folder return
            if (!mediaStorageDir.mkdirs()) {
                return null;
            }
        }*/
        String extStorageDirectory = Environment.getExternalStorageDirectory().toString();

        ContextWrapper cw = new ContextWrapper(getActivity());
        File directory = cw.getDir("imageDir", Context.MODE_PRIVATE);
        Log.e("---directory-----", "" + directory);

        //take the current timeStamp
        String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
        File mediaFile;
        ///data/user/0/competent.groove.biometric/files/IMG_20160803_181740.jpg
        //and make a media file:

        fileName = directory + File.separator + "IMG_" + timeStamp + ".jpg";
        mediaFile = new File(directory + File.separator + "IMG_" + timeStamp + ".jpg");

        Log.e("---extStorageDirectory-----", "" + extStorageDirectory);
        //Log.e("---mediaFile-----",""+mediaFile.length());
        //Log.e("---mediaFile-----"+mediaFile.getName(),""+mediaFile.getAbsolutePath());

        return mediaFile;

    }

and my aws code to upload file is public class UploadAws extends AsyncTask{

private  int total, percentage = 0;
Context context;
String s3_server_path = "";
ObscuredSharedPreferences shrdpref;
ObscuredSharedPreferences.Editor editor;

String res = ""; 


public UploadAws(Context context){
    Log.e(""+getClass(), "UploadAws constructor called");
    this.context = context;
    shrdpref = new ObscuredSharedPreferences(context, context.getSharedPreferences(Constants.PREFERENCES, Context.MODE_PRIVATE));

}

@Override
protected Object doInBackground(Object... params) {
    try {
        Log.d("" + getClass(), "UploadAws doInBackground called");
        File pictures = null;
        AmazonS3Client s3Client = null;
        TransferUtility tx = null;
        percentage = 0;

    try {                           
        s3_server_path = shrdpref.getString(Constants.PIC_PATH,"");

        File dir = context.getFilesDir();
        pictures = new File(dir, s3_server_path);
        s3Client = new AmazonS3Client(new BasicAWSCredentials(shrdpref.getString(Constants.AWS_Access_Key,""),shrdpref.getString(Constants.AWS_Secret_Key,"")));
        tx = new TransferUtility(s3Client,context);

    } catch (NullPointerException e) {
        e.printStackTrace();                
    }                 

        try {

            Log.e("------File length--------=", "" + pictures.length());
            Log.e("------File getName--------=", "" + pictures.getName());
            Log.e("------File getAbsolutePath--------=", "" + pictures.getAbsolutePath());
            s3Client.setEndpoint("s3.amazonaws.com");

            String Feetport_bucket = shrdpref.getString(Constants.fs_bucket, "");

            if (s3Client.doesBucketExist(Feetport_bucket)) {
                Log.e("Warn", "service called bucket exist");
            } else {
                s3Client.createBucket(Feetport_bucket);
                Log.e("FOS signature", "Bucket created");
            }

            int imageIndex = pictures.toString().lastIndexOf("/");
            String pictureString = pictures.toString().substring(imageIndex + 1, pictures.toString().length());

            String selfie_url = "FeetPort/" + shrdpref.getString(Constants.company_name, "") + "/attend/" + Utils.getCurrentDate() + "/" + pictureString;
            Log.d("UploadAws selfie_url called: ", "" + selfie_url);

            //https://feetport.s3.amazonaws.com/8632/17/20161122/mayank1_I_1624_065610_COL22.jpeg

            final TransferObserver observer = tx.upload(Feetport_bucket, pictures.getName(), pictures);

            observer.setTransferListener(new TransferListener() {
                @Override
                public void onStateChanged(int arg0, TransferState state) {
                    Log.e("onStateChanged", "on state changed " + state);
                }

                @Override
                public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
                    Log.e("onProgressChanged", "total bytes " + observer.getBytesTotal());
                    Log.e("onProgressChanged", "total bytes transfered " + observer.getBytesTransferred());
                    percentage = (int) (bytesCurrent / bytesTotal * 100);
                    Log.e("onProgressChanged", "total percentage " + percentage);
                }

                @Override
                public void onError(int arg0, Exception arg1) {
                    Log.e("onError=" + arg0, "on Error  " + arg1.toString());
                    percentage = 101;
                }
            });


            do {


            } while (percentage < 100);

            Log.e("percentage", "percentage  " + percentage);
            if (percentage == 100) {
                /**
                 * CGPL-17 9-5-2016 dynamic bucket and url.
                 */
                String S3_SERVER = "https://".concat(Feetport_bucket).concat(".s3.amazonaws.com/");
                String imageUrl = S3_SERVER + selfie_url;

                //Session.setselfie_url(shrdpref, imageUrl);

                editor = shrdpref.edit();
                editor.putString(Constants.PIC_PATH, "" + imageUrl);
                editor.commit();

                Log.e("Selfie url ", "imageUrl --- " + imageUrl);
        /*JsonParameters object = new JsonParameters(context);
        object.markedAttendance(callback);*/
            } else {
        /*JsonParameters object = new JsonParameters(context);
        object.markedAttendance(callback);*/
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

    } catch (Exception exception) {
        exception.printStackTrace();

    }
    return null;
}

}


Answer:

This isn't anything related to S3.

ENOENT (No such file or directory) is a local error: file not found.

Take a very close look at the path in the error message. I've included the path here, and added some line breaks for readability. I did not otherwise edit this, and, importantly, I did not paste any of it here more than once:

/data/user/0
/competent.groove.feetport
/files/data/user/0
/competent.groove.feetport
/app_imageDir/IMG_20161122_073058.jpg

I'm not saying that this is wrong, but it certainly looks wrong. It certainly looks like you're trying to upload the file from a path that you have incorrectly constructed, with some duplication in the directory structure... and there's no such directory, no such file.

Question:

I understand the concept of folders and objects in Aws S3 bucket, that there is no folder and all are objects. But I need to create an empty folder where user can enter and upload their files. I have tried several things like adding a test .txt file with a path which extends the empty folder's name after "/" and it do create the folder but it keeps the test.txt file which if I delete also deletes the folder created.

So any idea about how to create an zero size empty folder object as is created in the web console of S3 storage? Any help would be great


Answer:

Use your SDK of choice to create an S3 object named, for example, folder1/folder2/cats/ with a size of 0 bytes. This assumes that your preferred folder separator is /, which is obviously the most commonly used separator, but you could use anything you like.

Here is a generic Java upload an object example. Modify this so that the object key is folder1/folder2/cats/ and no content is uploaded.

Question:

From the documentation i can see that you can download a file, but i was wondering is there a way to just get the content of a text file from an Amazon S3 bucket, without having to store the file locally ?


Answer:

Yes, you can use getObject() on an instance of AmazonS3 to download a file:

String bucketName = "BUCKET_NAME";
String key = "KEY_OF_OBJECT";

AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                    .withRegion(Regions.DEFAULT_REGION)
                    .withCredentials(new ProfileCredentialsProvider())
                    .build();

System.out.println("Downloading an object");
S3Object fullObject = s3Client.getObject(new GetObjectRequest(bucketName, key));
System.out.println("Content-Type: " + fullObject.getObjectMetadata().getContentType());
System.out.println("Content: ");

// fullObject.getObjectContent() is an InputStream 

BufferedReader reader = new BufferedReader(new InputStreamReader(fullObject.getObjectContent()));
String line = null;
while ((line = reader.readLine()) != null) {
    System.out.println(line);
}

If you only need part of the data, you can use the withRange() modifier on GetObjectRequest() to create a range request.

Question:

I'm developing a file manager app for Amazon S3 in Android. I'm using a stack to hold folder objects to help navigation. Each object in the stack is a folder which holds an arraylist mObjects of the files and folders in that folder.

An onClick event listener pushes the previously clicked object onto the stack and calls a setter for the arraylist for the clicked object.

The problem is that subsequent objects pushed onto the stack change the mObjects field of the very first object in the stack. I don't understand why because the mObjects field has modifier private for each object.

Many hours of code staring and Google searches have turned up a blank. I've done a stack trace in Android Studio and what I find weird is that the mObjects field for the first object on the stack changes at the line adapter.setData(o.getmObjects()) in my BucketObjectListFragment class.

public class BucketObjectListFragment extends ListFragment {
    public static List<S3ObjectSummary> s3ObjList;
    private final int position = 2;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        generateS3Objects();
        for(S3ObjectSummary s : s3ObjList) {
            Log.i("s3ObjList", s.getKey());
        }

        //Create the stack
        ObjectStack myStack = new ObjectStack();

        //Create a new object to hold the home list view
        BucketObject o = new BucketObject();
        o.setPrefixKey("/");

        //Create the home object's list of folders and files
        o.setHomeObjects();

        //Create an adapter to hold the list of files and folders
        ObjectAdapter adapter = new ObjectAdapter(o.getmObjects());
        setListAdapter(adapter);

        //Replicate an Android onClick event. First push the home view onto the stack.
        myStack.push(o);
        Log.i("Stack dump", "...");
        myStack.printStack();

        //Create a new BucketObject for the clicked folder.
        o = ((ObjectAdapter) getListAdapter()).getItem(position);
        o.setmObjects(o.getPrefixKey());
        adapter.setData(o.getmObjects());

        myStack.push(o);
        Log.i("Stack dump", "...");
        myStack.printStack();
    }

    public static void generateS3Objects(){
        //Generate a random list of files and directories that look like S3 objects.
        S3ObjList = new ArrayList<S3ObjectSummary>();
        for(int i=0; i<5; i++){
            S3ObjectSummary obj = new S3ObjectSummary();
            s3ObjList.add(obj);
        }
    }

    private class ObjectAdapter extends ArrayAdapter<BucketObject> {
        private ObjectAdapter(ArrayList<BucketObject> objects) {
            super(getActivity(), android.R.layout.simple_list_item_1, objects);
        }

        public void setData(ArrayList<BucketObject> items) {
            clear();
            setNotifyOnChange(false);
            if (items != null) {
                for (BucketObject item : items)
                    add(item);
            }
            notifyDataSetChanged();
        }
    }
}

My BucketObject class.

public class BucketObject {
    private UUID id;
    private String prefixKey;
    private String key;
    private ArrayList<BucketObject> mObjects;

    public BucketObject() {
        id = UUID.randomUUID();
    }

    public BucketObject(String mPrefixKey, String mKey) {
        id = UUID.randomUUID();
        prefixKey = mPrefixKey;
        key = mKey;
    }

    @Override
    public String toString() {
        return prefixKey;
    }

    public String getPrefixKey() {
        return prefixKey;
    }

    public void setPrefixKey(String mPrefixKey) {
        prefixKey = mPrefixKey;
    }

    public ArrayList<BucketObject> getmObjects(){
        return mObjects;
    }

    public ArrayList<BucketObject> setHomeObjects(){
        ArrayList<BucketObject> mFiles = new ArrayList<>();
        mObjects = new ArrayList<>();

        //Use a HashSet to hold unique folders and directories
        Set<String> mySet = new HashSet<String>();

        for (S3ObjectSummary summary : BucketObjectListFragment.s3ObjList) {
            String obj = summary.getKey();
            int firstSlash = obj.indexOf("/");

            if (firstSlash != -1){ //if object has multiple forward slashes
                String myKey = obj.substring(0, firstSlash);
                if(mySet.add(myKey)){ //add to set if not a duplicate directory
                    BucketObject b = new BucketObject((myKey + "/"), myKey);
                    mObjects.add(b);
                }
            }
            else {
                //Get files
                BucketObject b = new BucketObject(obj, obj);
                mFiles.add(b);
            }
        }
        mObjects.addAll(mFiles);
        return mObjects;
    }

    public ArrayList<BucketObject> setmObjects(String prefixKey){
        ArrayList<BucketObject> mFiles = new ArrayList<>();
        mObjects = new ArrayList<>();

        //Use a HashSet to hold unique folders and directories
        Set<String> mySet = new HashSet<String>();

        for (S3ObjectSummary summary : BucketObjectListFragment.s3ObjList) {
            String obj = summary.getKey();
            if (obj.startsWith(prefixKey)) {
                String suffix = obj.replaceFirst(prefixKey, "");
                int firstSlash = suffix.indexOf("/");

                if (firstSlash != -1) {
                    String key = suffix.substring(0, firstSlash);
                    if (mySet.add(key)) { //add to set if not a duplicate directory
                        BucketObject b = new BucketObject(obj, key);
                        mObjects.add(b);
                    }
                }
                else{
                    if(!suffix.equals("")) {
                        BucketObject b = new BucketObject(obj, suffix);
                        mFiles.add(b);
                    }
                }
            }
        }
        mObjects.addAll(mFiles);
        return mObjects;
    }
}

My ObjectStack class.

public class ObjectStack {
    private Stack<BucketObject> s3ObjectStack;

    public ObjectStack(){
        s3ObjectStack = new Stack<>();
    }

    public void push(BucketObject o) {
        s3ObjectStack.push(o);
    }

    public void printStack(){
        //Prints the contents of mObjects for each BucketObject in the stack.

        for(BucketObject o : s3ObjectStack){
            int i = 0;
            Log.i("BucketObject ", o.toString());
            for(BucketObject b : o.getmObjects()){
                Log.i("mObjects " + Integer.toString(i), b.toString());
                i++;
            }
        }
    }
}

Class to generate S3 objects.

public class S3ObjectSummary {
    private final String key;
    private static final String abc = "abcdefghijklmnopqrstuvwxyz";
    private static SecureRandom rnd = new SecureRandom();

    public S3ObjectSummary(){
        key = randomString(7);
    }

    public String getKey(){
        return key;
    }

    private String randomString(int len){
        StringBuilder sb = new StringBuilder(len);
        for(int i = 0; i < len; i++){
            sb.append(abc.charAt(rnd.nextInt(abc.length())));
        }
        sb.insert(3, '/');
        return sb.toString();
    }
}

Main Activity class.

public class StartActivity extends AppCompatActivity {
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_start);
    }
}

The xml for the activity with fragment.

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context="ie.codemaker.test_panda.StartActivity">

    <fragment
        android:name="ie.codemaker.test_panda.BucketObjectListFragment"
        android:id="@+id/fooFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</android.support.constraint.ConstraintLayout>

EDIT: I've edited the above to provide a complete, minimal and verifiable program. This has helped me narrow the problem down to the setData() method of the adapter. Even if I rename the clicked object to something other than o, which is the first object on the stack, the first object's private field still changes at setData(). If anyone also needs the Manifest.xml to run the code I can provide it.


Answer:

The problem here was, as someone else has pointed out in an answer to another question, "a typical Java error with pointers" using Android's ArrayAdapter: https://stackoverflow.com/a/14408492/3350944

In the following line I'm passing the array mObjects to the adapter:

ObjectAdapter adapter = new ObjectAdapter(o.getmObjects());

Then, in this line I'm passing a new array mObjects to the adapter but the adapter is still pointing to the first array:

adapter.setData(p.getmObjects());

I solved the problem with a new arrayList<> in class BucketObjectListFragment which I passed to the adapter:

private ArrayList<BucketObject> newArray;
ObjectAdapter adapter = new ObjectAdapter(newArray);

Then when i want to update the adapter, I clear the adapter's list and add the items to newArray from mObjects:

viewList.clear();
for(BucketObject obj : p.getmObjects()){
    newArray.add(obj);
}
adapter.notifyDataSetChanged();

Question:

I'm working on an Android App with AWS Android SDK, especifically using S3, and I started receiving the following error:

Source not found - The JAR of this class file belongs to container 
'Android private libraries' which does not allow modifications to source attachments
on its entries.

that happens on the line

s3Client = new AmazonS3Client(credentials);

when I try to create a new client to connect to S3.

I had to replace an old version of the SDK (1.7.1.1) with a newer version (2.1.0) in order to use PersistableDownload class to pause/resume downloads. I don't understand why is this happening since I'm not trying to change the library. What I remember doing:

- Removed the old library
- Added the new one
- Removed and added again several imports on the start of .java files
- Using an older device for the weekend (android API level 8)

Here is a extend part of the code. That might help.

AWSCredentials credentials = new BasicAWSCredentials(ACCESS_KEY_ID, SECRET_KEY);

Region region = Region.getRegion(regions);

AmazonS3Client s3Client = null;
try {
    s3Client = new AmazonS3Client(credentials);
    s3Client.setRegion(region);
} catch (Exception e) {
    e.printStackTrace();
}
return s3Client;

Any help is aprpreciated to solve this. Thanks!

Edit: I forgot to say that I tried rolling back the old library and that gives me the same error. Most annoying part is that it was working before.


Answer:

I believe this is related to the Android version. During the weekend I was using a cell with version 2.2.3, and now I'm using another device with version 4.4.4.

Version 1.7.1 of the SDK had no S3 issues with Android 2.2.3. Already posted a issue on the gitHub project and if/when the developers reply I'll update this.

Question:

Problem: I've a website I'm trying to display in an Android WebView. The text appears fine, but the pictures seem to be blank. After further investigation, the site uses image storing on the Amazon S3 storage space. The images are then retrieved every time from servers the page is loaded. Any help will be appreciated, code and pictures are below.

Pictures:

In Browser

In App

Code from trying my best to debug the WebView activity. Please note, this was my first time using the chrome debugging for Android Webview. I narrowed it down to the profile picture code for now to see what was happening. Profile picture URL in the code shows up fine, but the picture itself does not show up. Code is below:

<div class="profile-photo pull-right"><img src="http://realzim.s3.amazonaws.com/file/user/realzim/India/25/user_image/1440307265.jpg" width="55" height="55" class="img-thumbnail"></div>

Error code(s):

Mixed Content: The page at 'https://www.realzim.com/mobile/' was loaded over HTTPS, but requested an insecure image 'http://realzim.s3.amazonaws.com/file/user/realzim/India/25/user_image/1440307265.jpg'. This request has been blocked; the content must be served over HTTPS.

Uncaught Error: Bootstrap's JavaScript requires jQuery version 1.9.1 or higher

If I did get the wrong bit, please let me know. It's my first time working with this type of issue. Input will be greatly appreciated, thanks.


Answer:

Okay, so I think I've got it. First of all, a big thank you to Mark B for introducing me to the wonderful Chrome debugging tool as well as pointing me in the right direction.

In the original question I've posted up two errors I come across. After knowing what the problem exactly was, and doing a bit more research I managed to implement an effective solution. Below is the code that helped solve this issue:

if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.KITKAT){
        webSettings.setMixedContentMode(WebSettings.MIXED_CONTENT_ALWAYS_ALLOW);
    }

This basically helps set the mixed content on always for any phone running KITKAT SDK or above.

Hope this solution is able to help someone else out in the future.

Resource:

StackOverflow - Android WebView blocks redirect from https to http

Question:

I am beginner to android. I am working on client side code I have to read images store on server and display it in imageview. I have refer some stackoverflow question but not able to succeed. I did simple plain java program with HttpsURLConnection class I got the some binary response from server for image, thats ok with plain java code. But when I tried the same thing in android with little bit change in code, I got the Exception:

01-29 18:39:28.199: WARN/System.err(2045): java.io.IOException: SSL handshake failure: I/O error during system call, Unknown error: 0
01-29 18:39:28.249: WARN/System.err(2045):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.nativeconnect(Native Method)
01-29 18:39:28.249: WARN/System.err(2045):     at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:316)
01-29 18:39:28.249: WARN/System.err(2045):     at org.apache.harmony.luni.internal.net.www.protocol.http.HttpConnection.getSecureSocket(HttpConnection.java:168)
01-29 18:39:28.249: WARN/System.err(2045):     at org.apache.harmony.luni.internal.net.www.protocol.https.HttpsURLConnectionImpl$HttpsEngine.connect(HttpsURLConnectionImpl.java:399)

This is my public URL of sample image:

https://s3-ap-southeast-1.amazonaws.com/edt-demo-app/app-images/Mobile.jpg

The code which works fine for me in plain java is as follows:

package com.psl.dao; 
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;

mport javax.net.ssl.HttpsURLConnection;

   public class HttpsClient{

   public static void main(String[] args)
   {
        String url = "https://s3-ap-southeast-1.amazonaws.com/edt-demo-app/app-images/Mobile.jpg";
        try {
            new HttpsClient().print(url);
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }   
   }


   public void print(String url)throws Exception
   {
     String httpsURL = url;
     URL myurl = new URL(httpsURL);



     System.setProperty("https.proxyHost", "puproxy.company.co.in");
     System.setProperty("https.proxyPort", "8080");

     HttpsURLConnection con = (HttpsURLConnection)myurl.openConnection();
     System.out.println(con.toString());
     InputStream ins = con.getInputStream();
     InputStreamReader isr = new InputStreamReader(ins);
     BufferedReader in = new BufferedReader(isr);

     String inputLine;

     while ((inputLine = in.readLine()) != null)
     {
       System.out.println(inputLine);
     }

     in.close();
   }    
}

Then I have refer some android tutorial and tried following code in my android app:

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
import java.net.URLConnection;
import java.security.cert.X509Certificate;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

public class TestSSL {

    public static void main(String[] args) throws Exception {
        // Create a trust manager that does not validate certificate chains
        TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
            public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                return null;
            }
            public void checkClientTrusted(X509Certificate[] certs, String authType) {
            }
            public void checkServerTrusted(X509Certificate[] certs, String authType) {
            }
        } };
        // Install the all-trusting trust manager
        final SSLContext sc = SSLContext.getInstance("TLS");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
        // Create all-trusting host name verifier
        HostnameVerifier allHostsValid = new HostnameVerifier() {
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        };


        // Install the all-trusting host verifier
        HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

        System.setProperty("https.proxyHost", "puproxy.company.co.in");
       System.setProperty("https.proxyPort", "8080");

        URL url = new URL("https://s3-ap-southeast-1.amazonaws.com/edt-demo-app/app-images/Mobile.jpg");
        URLConnection con = url.openConnection();
        final Reader reader = new InputStreamReader(con.getInputStream());
        final BufferedReader br = new BufferedReader(reader);        
        String line = "";
        while ((line = br.readLine()) != null) {
            Log.d("tag",line+"");
        }        
        br.close();
    } // End of main 
} // End of the class //

But I got exception as above said:

01-29 18:39:28.199: WARN/System.err(2045): java.io.IOException: SSL handshake failure: I/O error during system call, Unknown error: 0

Any other code that works fine with Apache HttpClient or Async HttP client library by using Asynynchronusly loading of image would welcome. Also I dont know how to do proxy setting in android. so please do it for me with respective parameter

I have tried follwing tutorial but my image is not displayed.

http://javatechig.com/android/download-image-using-asynctask-in-android

The above tutorial works fine for Http URL but not for Https

I want code like in above tutorial that works fine for HTTPS URL For time being consider server certificate is not important for me.


Answer:

Please try this code

  • MainActivity
  package com.example.androidhttpsdemo;

  /*
   * 
   * MainActivity.java
   * @author Santosh Shinde
   * Date: 30/01/2015 12:28:16 PM
   * 
   */
  import java.io.IOException;
  import java.io.InputStream;
  import java.net.URL;
  import java.net.URLConnection;

  import javax.net.ssl.HttpsURLConnection;

  import android.app.Activity;
  import android.graphics.Bitmap;
  import android.graphics.BitmapFactory;
  import android.os.AsyncTask;
  import android.os.Bundle;
  import android.widget.ImageView;

  public class MainActivity extends Activity {
      //public static final String URL  ="https://googledrive.com/host/0B_DiX4MiMa3HTHdiYVRmUHBMcW8/image1.jpg";
      public static final String URL="https://s3-ap-southeast-1.amazonaws.com/edt-demo-app/app-images/Mobile.jpg";
      ImageView imageView;

      /** Called when the activity is first created. */
      @Override
      public void onCreate(Bundle savedInstanceState) {
          super.onCreate(savedInstanceState);
          setContentView(R.layout.activity_main);
          imageView = (ImageView) findViewById(R.id.ImageView1);

          // Create an object for subclass of AsyncTask
          GetXMLTask task = new GetXMLTask();
          // Execute the task
          task.execute(new String[] { URL });
      }

      private class GetXMLTask extends AsyncTask {
          @Override
          protected Bitmap doInBackground(String... urls) {
              Bitmap map = null;
              for (String url : urls) {
                  map = downloadImage(url);
              }
              return map;
          }

          // Sets the Bitmap returned by doInBackground
          @Override
          protected void onPostExecute(Bitmap result) {
              imageView.setImageBitmap(result);
              System.out.println("finished");
          }

          // Creates Bitmap from InputStream and returns it
          private Bitmap downloadImage(String url) {
              Bitmap bitmap = null;
              InputStream stream = null;
              BitmapFactory.Options bmOptions = new BitmapFactory.Options();
              bmOptions.inSampleSize = 1;

              try {
                   stream = getHttpConnection(url);
                   bitmap = BitmapFactory.
                            decodeStream(stream, null, bmOptions);
                   stream.close();
              } catch (IOException e1) {
                   e1.printStackTrace();
              }
              return bitmap;
          }

          // Makes HttpURLConnection and returns InputStream
          private InputStream getHttpConnection(String urlString)
                                              throws IOException {
              InputStream stream = null;
              URL url = new URL(urlString);
              URLConnection connection = url.openConnection();

              try {
                   HttpsURLConnection httpConnection = (HttpsURLConnection) connection;
                   httpConnection.setRequestMethod("GET");
                   httpConnection.connect();

                   if (httpConnection.getResponseCode() == HttpsURLConnection.HTTP_OK ||
                           httpConnection.getResponseCode() == HttpsURLConnection.HTTP_NOT_MODIFIED ) {

                           stream = httpConnection.getInputStream(); 
                   } else { // just in case.. 

                            //log.d("Surprize HTTP status was: " ,httpConnection.getResponseCode()); 
                           } 
              } catch (Exception ex) {
                  ex.printStackTrace();
              }
              return stream;
          }
      }
  }