Current state: "Under DiscussionRejected"
ISSUE: #7210
PRs:
Keywords: arrow/column-based/row-based
...
After the Proxy receives the inserted data, create multiple Arrow Array by field, instead of RecordBatch.
PROBLEM: The primitive unit of serialized data in Arrow is RecordBatch. Arrow does not provide interface to serialize Arrow Array.
...
- Arrow data can only be serialized and deserialized by unit of RecordBatch;
- Cannot copy out row data from RecordBatch;
- RecordBatch must be regenerated after sending via pulsar.
Arrow is suitable for data analysis scenario (data is sealed and read only).
...