Current state: Accepted
...
Apache Kafka is a distributed event store and stream-processing platform, and it is a popular solution for data streaming needs. Many community users expect Milvus to support Kafka because they have already used it in the production environment.
Summary
Milvus supports Kafka as a message stream, we can use the configuration option to decide to use Pulsar or Kafka on cluster mode.
Code Block | ||||
---|---|---|---|---|
| ||||
pulsar: address: localhost # Address of pulsar port: 6650 # Port of pulsar maxMessageSize: 5242880 # 5 * 1024 * 1024 Bytes, Maximum size of each message in pulsar. kafka: address: localhost port: 9092 |
Design Details
- add kafka and zookepper dev docker
- optimization mq_factory configuration initialization
- remove reader
- implement msg_stream with kafka
...