Hot questions for Using Amazon S3 in excel

Question:

I am uploading an excel(xls) file to s3 and then another application should download that file from s3 and parse using Apache POI reader. The reader accepts inputstream type as the input but for proper parsing of the excel it expects PushbackInputStream. The inputstream i get from the file downloaded from s3 is of type S3ObjectInputStream. How do i convert S3ObjectInputStream to PushbackInputStream?

I tried directly passing the S3ObjectInputStream (since this is an inputStream) to PushbackInputStream, but it resulted in the following exception :

org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:147)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96)
.....
.....
Caused by: java.lang.IllegalStateException: InputStream MUST either support mark/reset, or be wrapped as a PushbackInputStream
at org.springframework.batch.item.excel.poi.PoiItemReader.openExcelFile(PoiItemReader.java:82)
.....

I tried casting S3ObjectInputStream to PushbackInputStream, but it resulted in classcastexception.

java.lang.ClassCastException: com.amazonaws.services.s3.model.S3ObjectInputStream cannot be cast to java.io.PushbackInputStream

Anyone knows the solution for this


Answer:

This solved my issue.

InputStream inputStream = s3object.getObjectContent(); 
PushbackInputStream pushbackInputStream = new PushbackInputStream(inputStream);

Question:

Hi I am using amazon s3 sdk for java and I am trying to upload excel which was created dynamically.What happens that I can able to save file on local computer using:

//creating local file
    try {
        FileOutputStream fileOut = new FileOutputStream(new File("C:\\test.xls"));
        excelWorkBook.write(fileOut);
        fileOut.flush();
        fileOut.close();

    } catch (Exception e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

but When I try to upload on amazon s3 by means of stream nothing happen and I get the error string like

com.amazonaws.services.s3.model.AmazonS3Exception: Status Code: 400, AWS Service: Amazon S3, AWS Request ID: BC86F8AED980C7A3, AWS Error Code: RequestTimeout, AWS Error Message: Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed., S3 Extended Request ID: P1yiux9HeVH92ifelvhMfTyQ27Hgu3sC4xx8DXstO+HLizMera2OkYv3BSVCEJtxtFz10/SRbMk=
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:767)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:414)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:228)
    at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3316)
    at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1336)
    at com.sterling.smartdata.service.upload.impl.UploadReportImpl.uploadToCloud(UploadReportImpl.java:96)
    at com.sterling.smartdata.service.upload.impl.UploadCloudLocalImplTest.testUpload(UploadCloudLocalImplTest.java:33)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
    at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:85)
    at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:86)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:241)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:87)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:180)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

I am stuck from last 2 days and I have tried searching the solutions but I haven't found one.

code:

public void uploadToCloud() {
        AWSCredentials credentials = new BasicAWSCredentials(
                accessKey, 
                secretKey);

        AmazonS3 s3client = new AmazonS3Client(credentials);

        HSSFWorkbook excelWorkBook=new HSSFWorkbook();
        missingAbbreviationReport.createMissingChargeAbbreviationSheet(excelWorkBook);
        missingAbbreviationReport.createMissingDispositionAbbreviationSheet(excelWorkBook);

        //creating local file
        try {
            FileOutputStream fileOut = new FileOutputStream(new File("C:\\test.xls"));
            excelWorkBook.write(fileOut);
            fileOut.flush();
            fileOut.close();

        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }


        //upload to amazon s3
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        try {
            excelWorkBook.write(byteArrayOutputStream);
            byteArrayOutputStream.close();
        } catch (IOException e) {

            LOGGER.error(e.getMessage());
        }

        byte[] bf=new byte[byteArrayOutputStream.toByteArray().length];
        ByteArrayInputStream bi=new ByteArrayInputStream(bf);
        try {
            bi.read(byteArrayOutputStream.toByteArray());
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }



        ObjectMetadata objectMetaData =new ObjectMetadata();
        objectMetaData.setContentType("application/vnd.ms-excel");
        objectMetaData.setContentLength(byteArrayOutputStream.toByteArray().length);

        s3client.putObject(new PutObjectRequest("smartdata-missingslg-reports", "test.xls",bi,objectMetaData) );

    }

Answer:

Change the following 2 lines:-

byte[] bf=new byte[byteArrayOutputStream.toByteArray().length];
ByteArrayInputStream bi=new ByteArrayInputStream(bf);

to

ByteArrayInputStream bi= ByteArrayInputStream(byteArrayOutputStream.toByteArray());

Question:

I've been trying to extract an .xlsx file from a AWS bucket I created and store it as a multipartfile variable. I've tried many different approaches, but at best I get weird characters. I'm not finding much documentation on how to do this. Thanks!


Answer:

// you may need to initialize this differently to get the correct authorization
final AmazonS3Client s3Client = AmazonS3ClientBuilder.defaultClient(); 
final S3Object object = s3Client.getObject("myBucket", "fileToDownload.xlsx");

// with Java 7 NIO
final Path filePath = Paths.get("localFile.xlsx");
Files.copy(object.getObjectContent(), filePath);
final File localFile = filePath.toFile();

// or Apache Commons IO
final File localFile = new File("localFile.xlsx");
FileUtils.copyToFile(object.getObjectContent(), localFile);

I'm not 100% sure what you mean by "MultipartFile" - that's usually in the context of a file that's been sent to your HTTP web service via a multipart POST or PUT. The file you're getting from S3 is technically part of the response to an HTTP GET request, but the Amazon Java Library abstracts this away for you, and just gives you the results as an InputStream.