Skip to content

broadcasting unchunked diskarrays with string eltype fails #284

@tiemvanderdeure

Description

@tiemvanderdeure

MWE

using DiskArrays.TestTypes
a = UnchunkedDiskArray(fill("a", (10, 20, 5)))
a .* copy(a)

I'll post the entire stacktrace but what happens is that since Base.BroadcastStyle(typeof(a)) is ChunkStyle, it ends up dispatching on a copyto! that assume at least some of these are chunked - but none of them are. It only errors on types without a definitive size, though. But what I don't really understand is why non-chunked diskarrays need to broadcast as if they are chunked? Isn't the chunking information available at compile time?

ERROR: Type String does not have a definite size.
Stacktrace:
  [1] sizeof(x::Type)
    @ Base .\essentials.jl:783
  [2] call_composed
    @ .\operators.jl:1099 [inlined]
  [3] (::ComposedFunction{typeof(sizeof), typeof(eltype)})(x::Array{String, 3}; kw::@Kwargs{})       
    @ Base .\operators.jl:1096
  [4] MappingRF
    @ .\reduce.jl:92 [inlined]
  [5] afoldl(::Base.MappingRF{…}, ::Base._InitialValue, ::Array{…}, ::UnchunkedDiskArray{…})
    @ Base .\operators.jl:599
  [6] _foldl_impl(op::Base.MappingRF{…}, init::Base._InitialValue, itr::Tuple{…})
    @ Base .\reduce.jl:60
  [7] foldl_impl(op::Base.MappingRF{…}, nt::Base._InitialValue, itr::Tuple{…})
    @ Base .\reduce.jl:40
  [8] mapfoldl_impl(f::ComposedFunction{…}, op::typeof(Base.add_sum), nt::Base._InitialValue, itr::Tuple{…})
    @ Base .\reduce.jl:36
  [9] mapfoldl(f::Function, op::Function, itr::Tuple{Array{…}, UnchunkedDiskArray{…}}; init::Base._InitialValue)
    @ Base .\reduce.jl:167
 [10] mapfoldl
    @ .\reduce.jl:167 [inlined]
 [11] mapreduce
    @ .\reduce.jl:299 [inlined]
 [12] sum(f::Function, a::Tuple{Array{String, 3}, UnchunkedDiskArray{String, 3, Array{String, 3}}})  
    @ Base .\reduce.jl:524
 [13] common_chunks(::Tuple{Int64, Int64, Int64}, ::Array{String, 3}, ::Vararg{Any})
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\broadcast.jl:65
 [14] copyto!(dest::Array{…}, broadcasted::Base.Broadcast.Broadcasted{…})
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\broadcast.jl:34
 [15] materialize!
    @ .\broadcast.jl:905 [inlined]
 [16] materialize!
    @ .\broadcast.jl:902 [inlined]
 [17] _disk_copyto!(dest::Array{String, 3}, source::UnchunkedDiskArray{String, 3, Array{String, 3}}) 
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\array.jl:7
 [18] copyto!(dest::Array{String, 3}, source::UnchunkedDiskArray{String, 3, Array{String, 3}})       
    @ DiskArrays C:\Users\tsh371\.julia\dev\DiskArrays\src\array.jl:70
 [19] copymutable
    @ .\abstractarray.jl:1201 [inlined]
 [20] copy(a::UnchunkedDiskArray{String, 3, Array{String, 3}})
    @ Base .\abstractarray.jl:1144
 [21] top-level scope
    @ c:\Users\tsh371\.julia\dev\DiskArrays\test\runtests.jl:12
Some type information was truncated. Use `show(err)` to see complete types.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions