Comparison of Kafka vs Kinesis

Feature	Kafka	Kinesis
Storage of Messages	As much as you want. On the cloud, you pay for storage.	24 hours by default. Up to 7 days with a config change
Ordering of Messages	Partition level	Shard level
Message Delivery Semantics	Kafka guarantees at-least-once delivery by default. Kafka supports exactly-once delivery in Kafka Streams	Kinesis Data Streams has at least once semantics
Replication	Use Confluence’s MirrorMaker to replicate a topic	All message automatically replicates to all 3 availability zones
Scaling	Add more partitions to a topic	API call to increase the number of shards
Partition/Shard Modification	Increase only and does not repartition existing data	Re-shard by merging or splitting shards
Partition/Shard Limitation	No limit. Optimal partitions depend on the use case	500 shards in US East (N. Virginia), US West (Oregon), and EU (Ireland) regions. 200 shards in all other regions.
Security	Either SSL or SASL and authentication of connections to Kafka Brokers from clients; authentication of connections from brokers to ZooKeeper; data encryption with SSL/TLS	Data can be secured at-rest by using server-side encryption and AWS KMS master keys on sensitive data within KDS. Access data privately via your Amazon Virtual Private Cloud (VPC)
Tools	Kafka Connect – gets data in and out of Kafka. Kafka Streams – stream processing of the data that flows through Kafka.	Video Streams Data Streams Data Firehose Data Analytics
Monitoring	Yammer Metrics for metrics reporting in the server	AWS CloudWatch and CloudTrail
Limitations	Default is 1MB per message, but can be changed	Message size is limited to 1MB. You can only get records 5 times per second and up to 2MB per shard. Need to add more shards (at an additional cost per shard) to scale.
Dependency	ZooKeeper	DynamoDB
Pricing	Per node No concept of elasticity	Per shard hour Per storage after 24 hours Per PUT request Shards are elastic, so you can decrease shards during slow periods