Hot questions for Using Amazon S3 in amazon elastic beanstalk

Top Java Programmings / Amazon S3 / amazon elastic beanstalk

Question:

What I have done ?

Developed a Web app using JSP which allows user to register , login and upload file to AWS S3. I am deploying this app to aws using Elastic Beanstalk by uploading war file of app. For login and register modules I have used RDS and it is working fine.

Problem

When I want to upload files to S3 , I need to use AWS SDK jar and its supporting JAR files in WEB app. When I finished with development part and exported war file , it was around 75 MB. So problem is that if I change anything to app , I need to upload this large size war to aws. Size of the war file is effected due to JAR files of AWS SDK.

What are the options to deal with such situation ?


Answer:

Use build tools like Maven. This will ensure the dependency files downloaded in the server's local maven repository. So it makes the upload size of your project, reduced. Here is a official quick guide

Question:

I just migrated my Spring applications from a VPS provider to Amazon Web Service. So what I did were:

  1. Redeploy my app inside EC2,
  2. Change the DB connection from localhost:3306 to RDS MariaDB,
  3. Implementing S3 for all of the file upload operations
  4. And etc.

And I managed to make my app steadily running under AWS now.

Now I need to make some enhancement on my app, then I realized that I couldn't simply replicate those RDS and S3 features in my laptop. So I turned to Eclipse's AWS Toolkit because it provides a nice development and testing experience. But I found out that most of AWS Toolkit tutorial in making a java web app is by using AWS Elastic Beanstalk resource. Which my current app is not using.

My question now: Do I have to refactor my app to use Elastic Beanstalk? (this approach surely not in my favor)... or is there anyway (must be) I can develop java web app in AWS without using Elastic Beanstalk and still using the privileges of AWS Toolkit?

Please anybody who has done this before care to show me a link or tutorial?

Regards, Bromo


Answer:

You do not have to, or even need to use Elastic Beanstalk. What you have done so far is perfectly reasonable. Also, Elastic Beanstalk really won't help you with the issue of running things locally. It would help you to spin up a test environment on AWS, but you would still have the same issues if you wanted to run it entirely locally.

Regarding running things from your local laptop, you should be able to run a database server locally. Is there a specific reason you can't seem to get your application to connect to a local database when running on your local computer, instead of trying to connect to an RDS database?

For S3 you could maintain a separate set of code that uses the local file system when running locally, although that would prevent you from testing your S3 code. Alternatively you could setup a "test" S3 bucket and have your code connect to that S3 bucket when running locally. Another alternative would be to run some sort of S3 emulator service locally, like this, but I've never used one of those so I can't say how well that would work.

Question:

I am using the following code to write to S3 bucket which has lots of xml files now. I am getting a No space left on device error. The subsequent uploads are failing. I see lots of files getting created in var/cache/tomcat8/temp folder. I am deleting the files through my code. Can someone please help me out in understanding what is wrong here? Actually we want to delete the temp file as soon as we upload it to S3.

Path path = Files.createTempFile(filename, suffix);
file = path.toFile();

Files.write(path, incomingXml.getBytes(StandardCharsets.UTF_8));

if(file!=null){
    file.deleteOnExit();
}
s3client.putObject(new PutObjectRequest(fullBucketPathByDate, filename, file));
LOGGER.info("Upload completed");

Answer:

From the File.deleteOnExit docs (emphasis mine):

Requests that the file or directory denoted by this abstract pathname be deleted when the virtual machine terminates.

That means that the temp file would only be deleted once you shutdown your Tomcat process. What you need to do is delete the file as soon as you are finished uploading it to S3. Something like this:

Path path = Files.createTempFile(filename, suffix);
file = path.toFile();

Files.write(path, incomingXml.getBytes(StandardCharsets.UTF_8));

if(file!=null){
    s3client.putObject(new PutObjectRequest(fullBucketPathByDate, filename, file));
    file.delete();
}
LOGGER.info("Upload completed");

Alternatively, look into streaming the file contents directly to S3 without using temp files at all.