Performance of splatting a number

[KristofferC]

In a few places, the performance of splatting a number has shown up: #29060, #28764 (comment) and working around this give signficiant speed-ups.
The code is well inferred but there is a call to _apply instead of the actual method. This dissapears if the number is wrapped in a tuple.

It would be nice to have this fixed so these type of workarounds are no longer needed.

Ref #27434 (comment)

cc @mbauman, @martinholters

[Keno]

We could special case numbers as we do tuples in the inliner. The more generic solution would have to emulate the iteration protocol.

[mbauman]

Forgive me if this is a naive understanding of inference, but it seems like there's a middle-ground here — isn't it fairly easy to see that iterate(::X,::Any)::Const(nothing) for Xs that only iterate once.

[Keno]

Yes, but that's still emulating the iteration protocol in the optimizer (inference already does some of this) but the iterate isn't part of the code (since _apply does it internally). Basically you need to implement this TODO:

julia/base/compiler/ssair/inlining.jl

Line 831 in 3c8119f

# TODO: We could basically run the iteration protocol here

or special case it there.

[c42f]

I just noticed we share basically the same problem over at JuliaArrays/StaticArrays.jl#361.

What if we change lowering of splatting so that f(x...) turns into something like Core._apply(f, Base.splatarg(x))? We can have Base.splatarg(x) = x by default (or possibly collect into a tuple via iteration?) For some more trivial cases like Number and StaticVector a couple of overrides would solve this issue:

Base.splatarg(x::Number) = (x,)
Base.splatarg(v::StaticVector) = Tuple(v)

This does mean that splatting would become configurable by user types but that seems generally consistent with the way we usually lower syntax into function calls.

[yurivish]

Nice idea. Maybe it can be Base.splattable for consistency with Base.broadcastable.

[c42f]

Base.splattable ...

Interesting analogy; we could go with it, though I think the name is less evocative than splatarg.

@Keno's point about making inference aware of iteration in _apply is might just be a better option if we want splatting to be inherently defined by iteration. Right now I can't think of a reason why you'd ever want splatting to be different, but maybe someone with a better vision of the Julian far future can (I'm looking at you @andyferris :-P)

[andyferris]

Haha... no, I think I believe in strong interfaces, and guarantees the compiler doesn’t break for the purpose of performance optimizations.

Using splatarg at least puts control/correctness in the hands of users. On the other hand, it puts control/correctness in the hands of users... ! It feels a bit like literal_pow - a clever and useful lowering trick but relies on users’ good behaviour to maintain the illusion of the fundamental guarantee of referencial transparency (not saying I’ve thought of anything better...).

So, to my personal tastes we would only resort to that if Keno’s approach is too messy. But my opinion isn’t strong.

[c42f]

Yes, something which can be overridden only really makes sense if there's a compelling reason to allow splatting to be different from iteration in some cases. Otherwise it's just a nasty performance hack. My question was really whether such a use case exists. In hindsight that's actually rather orthogonal to solving this issue.

Another thought: I don't know how hard it is to teach inference about iteration in _apply, but an interesting alternative would be to surface the C implementation of argument list construction in _apply to the Julia level. That is, to lower

f(x, y..., z...)

into something like

args = flatten_args((x,), y, z)
_apply(f, args)

where flatten_args applies the iteration protocol to construct the splatted argument list.

[andyferris]

OK - that seems pretty interesting!

[c42f]

It's currently tricky to surface the C implementation of _apply argument flattening in a useful way. Ideally we'd want something roughly like

1. "When possible", arguments are flattened into a single tuple via iteration
2. With dynamically known small total length, fall back to using a stack allocated buffer of Any. I guess that's not possible right now.
3. With unknown total length, flatten args into a Vector{Any}

One approach to implementing "when possible" would be to generalize the IteratorSize trait to express lengths which are known based on the type of the input; a Base version of StaticArrays.Length{N}(). This would require that users know about and implement Base.IteratorSize for statically sized objects. A difficulty is that IteratorSize mixes together the expression of length via HasLength and size via HasShape.

Doing more in inference to constant propagate the iteration state for small finite iterations seems helpful and we might get rid of Base.indexed_iterate as part of that. I found the output of the following confusing in fairly recent julia-1.3:

julia> function iter_unpack(t)
           i1 = iterate(t)
           a = i1[1]
           i2 = iterate(t, i1[2])
           b = i2[1]
           b
       end
iter_unpack (generic function with 1 method)

julia> @code_warntype optimize=true iter_unpack((1,2))
Variables
  #self#::Core.Compiler.Const(iter_unpack, false)
  t::Tuple{Int64,Int64}
  i1::Union{Nothing, Tuple{Int64,Int64}}
  a::Int64
  i2::Union{Nothing, Tuple{Int64,Int64}}
  b::Int64

# big pile of code ...
# i1 and i2 are partially const but that seems not to be inferred