Skip to content

Make it easy to seek through large objects #264

@roman-khimov

Description

@roman-khimov

There are several ways to do this:

  1. Store the size of each chunk in the link object along with the hash. It has the most flexibility, but it also means additional per-part overhead to store this.
  2. Keep link object as is, but store the offset from the beginning in each regular object and at least allow to bisect into some particular offset instead of looping through the whole list (10 HEADs is much better than 1000 HEADs).
  3. Demand individual parts to be of the same size, specify this size in the split data header. It's pretty easy to do, we probably HEAD header.split.previous anyway, so checking that the size of the object is the same as the previous one costs nothing.

Metadata

Metadata

Assignees

Labels

I2Regular impactS2Regular significanceU4Nothing urgentenhancementImproving existing functionality

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions