Add caching to more functions

[p12tic]

On cppreference-doc we are processing around 4000 comparatively large pages using premailer. This takes a while with the current implementation. After looking into profiler results and doing some experimental changes it turns out that simply caching results of more functions and increasing the size of the caches speed up premailer by around 2.5 times.

This PR implements the above-mentioned changes in a way that the new behavior can be enabled on runtime explicitly, old-behavior is default and existing users are almost not impacted.

The function_cache decorator is modified to accept the size of the cache as a argument to the wrapper, so that it can be changed on runtime. The size of the caches may now be controlled via additional parameter to the Premailer.__init__ method.

Also, function_cache is modified not to drop the contents of the cache when it overflows: this is the canonical behavior of caches. Small cache may still be very useful for handling the majority of requests and since it already reached the maximum size without inducing out of memory condition, being at the maximum size instead of zero will likely not induce out of memory condition later. If users are concerned about an out of memory condition arising later, when doing other tasks, we should provide a method to clear caches explicitly.

Furthermore, function_cache is modified to convert list arguments to tuples so that they can be hashed. _HashedSeq can be retired this way.

Caching behavior has been added to the top functions that I saw in the profiling results. The cache itself has very low overhead, so we know that at worst case we will only increase memory usage without reducing performance too much.

Finally, python 2.6 and 3.3 have been removed from the travis.yml as the upstream testing tools no longer support these python versions.

Please let me know what do you think about this. Thanks for your time!

[coveralls]

Coverage remained the same at 100.0% when pulling 21436d4 on p12tic:more-caching into ebfd310 on peterbe:master.

[peterbe]

There are so many good things with this PR and I don't want to entirely block your progress but there is one great option we should consider and that's Python 3's collections.lru_cache. Yes, it means we need a backport for Python 2.

I'm not sure it would work for us right away, unless we do some tricks, like you did, with converting lists to tuples. But functools.lru_cache is wicked smart and highly optimized.

Another very attractive option is to depend on CacheTools. It supports Python 2, has a LFU implementation too if that's applicable to us.

[peterbe]

Now is not the time but the correct name ought to be "function_memoize" or something like that.

[peterbe]

It's a bit strange that the caching just decides to stubbornly refuse to add more entries. I've already forgotten what you said in the justification in the PR description. Can you please add a code comment here to explain this reasoning.

[peterbe]

I don't think these need to be here at all. They're operational, not functional. They clutter up the code and obscures the options.

The likelihood of someone preferring to only set it to 500 or 10_000 is extremely rare and I think there are much better ways to do that. Like this:

import os

def function_cache():
    ...snip...
    # indicator of cache missed
    sentinel = object()

    max_cache_entries = int(os.environ.get('PREMAILER_MAX_CACHE_ENTRIES', 1000))

    @functools.wraps(func)
    def inner(*args, **kwargs):
        ...
    return inner

That will make the code a lot simpler. It'll give all the power to your use case too

[peterbe]

AWESOME!!

[peterbe]

Unlike the cache_*_size options, this one is actually not operational. Let's keep this. Setting this will potentially have an effect on the generated HTML output.

[peterbe]

Actually, the more I think about it, we should consider CacheTools. It's just so convenient and we don't need to do any heavy lifting mess with the difference between Python 2 and 3.

I still think the max cache size should be controlled from a documented environment variable though.

[peterbe]

By the way, see #203
Watch out for merge conflicts if you rebase this.

[peterbe]

cachetools is awesome. I think we should continue this effort over in #206

[peterbe]

#206 landed.