Hot questions for Using Amazon S3 in amazon dynamodb

Top Java Programmings / Amazon S3 / amazon dynamodb

Question:

I know this sounds like a simple question, but for some reason I can't find a clear answer online or through StackOverflow.

I have a DynamoDB with a Table named "ABC". The primary key is "ID" as a String and one of the other attributes is "Name" as a String. How can I delete an item from this table using Java?

    AmazonDynamoDBClient dynamoDB;
    .
    .
    .
    DeleteItemRequest dir = new DeleteItemRequest();
    dir.withConditionExpression("ID = 214141").withTableName("ABC");
    DeleteItemResult deleteResult = dynamoDB.deleteItem(dir);

I have a validation exception:

Exception in thread "main" com.amazonaws.AmazonServiceException: 1 validation error detected: Value null at 'key' failed to satisfy constraint: Member must not be null (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: RQ70OIGOQAJ9MRGSUA0UIJLRUNVV4KQNSO5AEMVJF66Q9ASUAAJG)
    at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1160)
    at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:748)
    at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:302)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:3240)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.deleteItem(AmazonDynamoDBClient.java:972)
    at DynamoDBUploader.deleteItems(DynamoDBUploader.java:168)
    at Main.main(Main.java:56)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

If I need to know the Hash Key in order to delete an item in a DynamoDB Table, I think I may need to redesign my database in order to delete items efficiently.

My table looks like this: If that is the case, ahh... I think I need to re-design my database table.

ID | Name | Date | Value
-----------------------------------
1  | TransactionA | 2015-06-21 | 30
2  | TransactionB | 2015-06-21 | 40
3  | TransactionC | 2015-06-21 | 50

Basically, I would like to easily delete all transactions with Date "2015-06-21". How can I do this simply and quickly without having to deal with the Hash Key ID?


Answer:

AWS DynamoDB knows the column that is hash key of your table. You just need to specify the value to be deleted. DeleteItemRequest has a fluent API for that :

Key keyToDelete = new Key().withHashKeyElement(new AttributeValue("214141"));
DeleteItemRequest dir = new DeleteItemRequest()
    .withTableName("ABC")
    .withKey(keyToDelete);

Question:

A Kinesis Firehose stream receives messages. There is an option to persist into S3, but my use case is to insert into dynamodb table. Firehose has an option to enable Lambda function. Shall i write insert logic into dynamodb table using Lambda? Is this the right approach? If so, then how to insert records into DynamoDB using Lambda written in Java.


Answer:

There isn't a standard way of inserting Firehose stream data into DynamoDB (such as S3 or Redshift). The recommended way is to do a Lambda and insert the records into DynamoDB with that.

Use dynamoDB.batchWriteItem or dynamoDB.putItem, more info in this article or this one.

public String handleRequest(KinesisFirehoseEvent event, Context context)
    List<KinesisFirehoseEvent.Record> records = event.getRecords();
    for(KinesisFirehoseEvent.Record rec : records)
    {
        String recordId = rec.getRecordId();
        String data = StandardCharsets.UTF_8.decode(rec.getData()).toString();

        Item item = transformStringToItem(data);
        // Write the item to the table 
        table.putItem(item);
    }
    return "success";
}

The firehoseStream structure is documented here, and a Java example example is here, and more info on firehose to lambda here.

Question:

I've been attempting to delete an item from a table in DynamoDB through java code, but every attempt I've made results in the same error:

com.amazonaws.AmazonServiceException: The provided key element does not match the schema (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException;

My current attempt is very simple and looks like this:

final DynamoDB dynamoDB = new DynamoDB(new  AmazonDynamoDBClient(credentials));  

Table table =dynamoDB.getTable(tableName);    

DeleteItemSpec itemSpec = new  DeleteItemSpec().withPrimaryKey("cognitoId", cognitoId);  
table.deleteItem(itemSpec);

tablename is simply the table name, the credentials have been verified to be correct, and cognitoId is the actual ID of the item I'm trying to delete. The table in question has cognitoId as the primary key and I don't understand why the deletion isn't matching the schema. The table also has a sort key, or range key (I'm not sure what it is because the documentation is quite vague). I've been referring to the documentation here: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithTables.html#WorkingWithTables.primary.key


Answer:

Did you have a sort key while creating a table? If so, then you have to specify the sort key too as you have a composite key on the table. Having a sort key means that you could have multiple records with the same primary key, however the sort key must be unique

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithTables.html#WorkingWithTables.primary.key

The sort key may also be referred to as range or range key in the AWS Dynamo DB documentation and the console.

So your delete item would be like

DeleteItemSpec itemSpec = new DeleteItemSpec().withPrimaryKey("cognitoId", "my_id", "sortKeyField", "sort_key_id");
DeleteItemOutcome outcome = table.deleteItem(itemSpec);

Question:

I am using aws serivces. I have a Dynamo DB table which is populated using Lambda function. I created lambda function for delete rows from Dynamo DB table as well. I have used full table scan to delete a particular row.my lambda got throttled so i requested for increasing the limit and now its working fine. But now Dynamo DB getting throttled showing throttled read requests and I find some of the rows not deleted in my Dynamo DB table.I increased the read request capacity but not sure how much to increase it.I have checked the amazon docs as well they are showing you can increase it as per your requirements. Any help will be helpful.


Answer:

From official DynamoDB documentation:

One read capacity unit represents one strongly consistent read per second, or two eventually consistent reads per second, for items up to 4 KB in size.

The capacity you want to provision depends on the number of requests per second and the size of your items.