Skip to main content

Module fragmentation

Module fragmentation 

Source
Expand description

Splits payloads larger than one crypto provider message into fragments and reassembles them on the receiving side.

Fragmentation runs above the CryptoProvider, so each fragment is an ordinary OutgoingMessage that the provider encrypts on its own. A fragment’s topic is the original topic behind the reserved FRAGMENT_TOPIC prefix, and its payload is a FragmentFrame: a CBOR FragmentHeader followed by the raw fragment data, so the data is copied once on each side. Peers without fragmentation support deliver such messages to subscribers as an unknown topic.

The sender awaits each fragment before sending the next and the communication backends deliver in order, so fragments are appended as they arrive and any gap discards the message.

Every peer is untrusted, so the Reassembler bounds the reassembled size, and the number of incomplete messages and the bytes reserved for them, both per source and per pool. Web pages are numerous and untrusted, so all web sources share a smaller pool that cannot consume the pool of the other sources. An incomplete message is discarded when no fragment arrives for REASSEMBLY_IDLE_TIMEOUT, or REASSEMBLY_TIMEOUT after its first fragment.

ModulesΒ§

reassembler πŸ”’
split πŸ”’

StructsΒ§

FragmentFrame πŸ”’
Payload of a fragment message: the FragmentHeader in CBOR followed directly by the raw data, so the data is not re-encoded.
FragmentHeader πŸ”’
Metadata that identifies a fragment within its message.

ConstantsΒ§

FRAGMENT_TOPIC πŸ”’
Reserved topic that marks a message as a fragment. A fragment of a message with a topic carries IpcFragment/<topic>.
MAX_HEADER_LEN πŸ”’
Upper bound on the encoded size of a FragmentHeader.
MAX_PAYLOAD_SIZE πŸ”’
Largest payload that can be sent, and the largest reassembled payload a receiver accepts. IPC is meant for messages, so larger data has to be streamed some other way.

Type AliasesΒ§

OriginalTopic πŸ”’
Topic of the message a fragment belongs to, which may itself have no topic.