|Storage of Messages||As much as you want. On the cloud, you pay for storage.||24 hours by default. Up to 7 days with a config change|
|Ordering of Messages||Partition level||Shard level|
|Message Delivery Semantics||Kafka guarantees at-least-once delivery by default. Kafka supports exactly-once delivery in Kafka Streams||Kinesis Data Streams has at least once semantics|
|Replication||Use Confluence’s MirrorMaker to replicate a topic||All message automatically replicates to all 3 availability zones|
|Scaling||Add more partitions to a topic||API call to increase the number of shards|
|Partition/Shard Modification||Increase only and does not repartition existing data||Re-shard by merging or splitting shards|
|Partition/Shard Limitation||No limit. Optimal partitions depend on the use case||500 shards in US East (N. Virginia), US West (Oregon), and EU (Ireland) regions. 200 shards in all other regions.|
|Security||Either SSL or SASL and authentication of connections to Kafka Brokers from clients; authentication of connections from brokers to ZooKeeper; data encryption with SSL/TLS||Data can be secured at-rest by using server-side encryption and AWS KMS master keys on sensitive data within KDS. Access data privately via your Amazon Virtual Private Cloud (VPC)|
|Tools||Kafka Connect – gets data in and out of Kafka.|
Kafka Streams – stream processing of the data that flows through Kafka.
|Monitoring||Yammer Metrics for metrics reporting in the server||AWS CloudWatch and CloudTrail|
|Limitations||Default is 1MB per message, but can be changed||Message size is limited to 1MB.|
You can only get records 5 times per second and up to 2MB per shard.
Need to add more shards (at an additional cost per shard) to scale.
No concept of elasticity
|Per shard hour|
Per storage after 24 hours
Per PUT request
Shards are elastic, so you can decrease shards during slow periods