"Initializing cache..." does not save progress

[owenh000]

Thanks for attic!

I use a remote attic repository, across a somewhat unreliable connection, with many accumulated archives. It worked well until recently when the local cache needed to be re-initialized. I am not sure why (but I may have done something to require this).

Now any attic operation requires that the cache be re-initialized. There are so many archives that I do not have a reliable connection for long enough to complete the operation. When I retry, attic appears to restart from the beginning. Also, if the connection is interrupted for a while, attic appears to hang but never quits with an error.

If I get this resolved, I will prune the repository more thoroughly. Until then, I need a cache! :)

Thanks.

[tgharold]

I'm having a similar issue. When Attic needs to initialize the cache on the local system (source of the backup) to match what is on the remote system (where the repository lives), it is spending a very long time transmitting data and then processing/analyzing.

In the case of a 1.7 million file backup (74GB), with dozens of archive points, over a 1.5Mbps line, that if the local cache directory ever gets corrupted, it will take days to rebuild the local cache. Even with just 11 days worth of archives (roughly 1 backup per day), it is taking a very long time to transmit/process the cache information. So there's definite inefficiencies here that are concerning.

The "index.####" file on the remote server is only 161MB. This process started at 11:44 and it is now 12:29. The ~/.cache/UUID/chunks file in the local cache is now 177MB. Over a T1 line (1.5Mbps) that should have taken about 24 minutes to transmit -- and looking at the current bandwidth utilization, that seems to be true.

Suggested improvements:

Note: It seems like (based on the bandwidth usage graph) that Attic cache initialization does a bulk download of something from the repository, then processes it locally?

• When doing the bulk transfer over the wire of the raw information from the remote repository, put up some indicator showing the total estimated size of the transfer along with the current performance.
• Each "Analyzing archive" output line should have a percentage counter at the end that updates as it works. If you don't know the percentage or can't calculate it, then printing the number of entries processed so far and updating at most once every 2 seconds would be enough that the user does not have to resort to looking at "top" to figure out if Attic is hung or still working.
• After each "Analyzing archive" step, save state so that Attic can restart the process without losing too much progress.
• If the cache initialization is interrupted, figure out a way to restart it without downloading everything from the remote server again.