Handle expiration of storage groups #1225
Description
- based on and blocked by Use well-known expiration attribute for SGs neofs-api#205
Overview
Storage groups have limited lifetime in NeoFS. All system parts should handle SG expiration: storage nodes shouldn't store expired STORAGE_GROUP objects, inner ring nodes should not perform data audit of them.
Inner ring
Data audit uses SGSource primitive to get list of storage groups to audit data for. The implementation selects objects from the container. We need to complicate some of the components with the logic which excludes expired storage groups.
I propose to have a component that allows you to select storage groups in a container:
type StorageGroup struct {
Descriptor storagegroup.StorageGroup
ExpiresAfter uint64 // epoch
}
type StorageGroupSource interface {
Iterate(container cid.ID, handle func(StorageGroup)) error
}
// implements StorageGroupSource via NeoFS API protocol.
type neofsContainerClient struct {
// wraps NeoFS SDK client
}
// StorageGroupSource mediator which excludes expired storage groups.
type aliveStorageGroups struct {
// wraps base StorageGroupSource with network state
}The data auditing component will use this interface to sequentially process storage groups.
Storage node
For these network members, storage group lifetimes are only relevant for garbage collection. Based on the fact that the SG descriptor is embedded in the binary structure of the object, and the expiration mechanism is the same as other types of objects, STORAGE_GROUP objects do not require additional logic to be processed by GC implementation. Within the framework of this task, it is only possible to double-check this fact.
NeoFS CLI
User interface should print expiration date in epochs. Seems like it is already implemented
I'd pay attention on the approach of storage group descriptor decoding
Since expiration is written in object attribute it's not enough to decode the object payload. I propose to create a function which links
Object and StorageGroup types:
// StorageGroupFromObject reads StorageGroup from the Object instance.
func ReadStorageGroupFromObject(sg *storagegroup.StorageGroup, obj object.Object)May be we also need a way to select alive storage groups only in storagegroup command.