Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Current state: "Under DiscussionRejected"

ISSUE: #7210

PRs: 

Keywords: arrow/column-based/row-based

...

After the Proxy receives the inserted data, create multiple Arrow Array by field, instead of RecordBatch.

PROBLEM: The primitive unit of serialized data in Arrow is RecordBatch. Arrow does not provide interface to serialize Arrow Array.

...

  1. Arrow data can only be serialized and deserialized by unit of RecordBatch;
  2. Cannot copy out row data from RecordBatch;
  3. RecordBatch must be regenerated after sending via pulsar.


Arrow is suitable for data analysis scenario (data is sealed and read only).

...