I have a DynamoDB table. When I check the Overview tab for the table on AWS Web Console, I see two options to enable streams.
Manage Streams to Kinesis
Manage DynamoDB Stream.
How these two streams are different from one another?
DynamoDB offers two streaming models for change data capture: Kinesis Data Streams for DynamoDB and DynamoDB Streams.
For the differences between the two and to help you choose the right solution for your application, please refer to Streaming Options for Change Data Capture here
Related
I'm just getting started on iot and note that iot-core uses DynamoDB. That's not going to work for me (no table joins, no stored procs etc.). Is it possible to use a different DB? I usually use Postgres but appreciate it may not be the quickest for a lot of inserts. What do others use?
For IoT use case and telemetry data storage, I'd suggest to use AWS Timestream. When data is sent to AWS IoT Core, you can use an IoT Rule to forward data to Kinesis Data Stream (useful here for decoupling as well as doing batch processing from downstream pipeline). AWS Lambda can be used to pull data from Kinesis Data Stream, enrich the data if needed (like adding building, users, company information) before storing the data into Timestream. You can then use Grafana to visualize the data from Timestream.
I need to get data from DynamoDB into EventBrdige. Can this happen without the need of a Lambda Function? Can EventBridge listen to DynamoDB changes as a rule?
No. There is currently no direct integration between DynamoDB Stream and EventBridge.
You need to consider the different use cases for these two services. Event Bridge is for highly important low-volume events for simpler systems integration, while DynamoDB is for high-volume events, which are already recorded in the DB. DynamoDB Stream is designed chiefly for analytical use cases with the ability to aggregate or filter the events to analyze.
There are a few other event processing services in AWS, such as Kinesis and Managed Kafka, which are designed differently for scale than Event-Bridge.
I'm working on the serverless backend development and I have used aws dynamodb as database, Now I want to generate audit log for all the actions(get, update, delete) of record and store in seperate table. Is there any build in feature by aws? or how to implement it?
Thanks
You can enable DynamoDB streams on your DynamoDB table for exactly this purpose.
From the amazon documentation:
A DynamoDB stream is an ordered flow of information about changes to
items in a DynamoDB table. When you enable a stream on a table,
DynamoDB captures information about every modification to data items
in the table.
Whenever an application creates, updates, or deletes items in the
table, DynamoDB Streams writes a stream record with the primary key
attributes of the items that were modified. A stream record contains
information about a data modification to a single item in a DynamoDB
table. You can configure the stream so that the stream records capture
additional information, such as the "before" and "after" images of
modified items.
DynamoDB Streams helps ensure the following:
Each stream record appears exactly once in the stream.
For each item that is modified in a DynamoDB table, the stream records
appear in the same sequence as the actual modifications to the item.
DynamoDB Streams writes stream records in near-real time so that you
can build applications that consume these streams and take action
based on the contents.
For more information and examples, please check out the following resources
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html
https://aws.amazon.com/blogs/database/dynamodb-streams-use-cases-and-design-patterns/
It is possible to setup CloudTrail for DynamoDB operations incl actual data change.
Please take a look at this link.
I found this article regarding data transfer prices for certain services, but not including DynamoDB.
What is the rate for DynamoDB data transfer out?
It is the same price as EC2, but this is only for when you are accessing DynamoDB from outside of that AWS region. If you want to verify, use the AWS calculator and do 1 GB/day of EC2 out data transfer and so the same for DynamoDB. It should be $1.40 for each.
Three examples where you might run into this charge are:
Apps in us-west-2 accessing a DynamoDB table in us-east-1.
Apps accessing a DynamoDB table over a VPN or dedicated link.
Accessing a DynamoDB table over the Internet. For obvious security purposes, this is likely not a good idea. If you are going to do it, really lock it down with IAM permissions!!!
I have a lot of documents in offline books. It's in table format and I don't want to manually input these tables into a dynamoDB table. Can I use AWS rekognition to help me here OR I should look at some other service ?
Rekognition won't generally do a good job at this. AWS has a new OCR service called Textract. It's able to figure out tables and forms (content and structure).
You can join the preview at https://aws.amazon.com/textract/