Update/julia master #2

udesou · 2023-01-25T10:51:35Z

This PR updates the binding to the latest Julia master (up to this commit). Should be merged together with mmtk/mmtk-julia#18.

qinsoon

LGTM

This PR updates the binding to the latest Julia master (up to this commit: 134f3e7).

@CCall

…#51489) This exposes the GC "stop the world" API to the user, for causing a thread to quickly stop executing Julia code. This adds two APIs (that will need to be exported and documented later): ``` julia> @CCall jl_safepoint_suspend_thread(#=tid=mmtk#1::Cint, #=magicnumber=mmtk#2::Cint)::Cint # roughly tkill(1, SIGSTOP) julia> @CCall jl_safepoint_resume_thread(#=tid=mmtk#1::Cint)::Cint # roughly tkill(1, SIGCONT) ``` You can even suspend yourself, if there is another task to resume you 10 seconds later: ``` julia> ccall(:jl_enter_threaded_region, Cvoid, ()) julia> t = @task let; Libc.systemsleep(10); print("\nhello from $(Threads.threadid())\n"); @CCall jl_safepoint_resume_thread(0::Cint)::Cint; end; ccall(:jl_set_task_tid, Cint, (Any, Cint), t, 1); schedule(t); julia> @time @CCall jl_safepoint_suspend_thread(0::Cint, 2::Cint)::Cint hello from 2 10 seconds (6 allocations: 264 bytes) 1 ``` The meaning of the magic number is actually the kind of stop that you want: ``` // n.b. suspended threads may still run in the GC or GC safe regions // but shouldn't be observable, depending on which enum the user picks (only 1 and 2 are typically recommended here) // waitstate = 0 : do not wait for suspend to finish // waitstate = 1 : wait for gc_state != 0 (JL_GC_STATE_WAITING or JL_GC_STATE_SAFE) // waitstate = 2 : wait for gc_state != 0 (JL_GC_STATE_WAITING or JL_GC_STATE_SAFE) and that GC is not running on that thread // waitstate = 3 : wait for full suspend (gc_state == JL_GC_STATE_WAITING) -- this may never happen if thread is sleeping currently // if another thread comes along and calls jl_safepoint_resume, we also return early // return new suspend count on success, 0 on failure ``` Only magic number 2 is currently meaningful to the user though. The difference between waitstate 1 and 2 is only relevant in C code which is calling this from JL_GC_STATE_SAFE, since otherwise it is a priori known that GC isn't running, else we too would be running the GC. But the distinction of those states might be useful if we have a concurrent collector. Very important warning: if the stopped thread is holding any locks (e.g. for codegen or types) that you then attempt to acquire, your thread will deadlock. This is very likely, unless you are very careful. A future update to this API may try to change the waitstate to give the option to wait for the thread to release internal or known locks.

@something

`@something` eagerly unwraps any `Some` given to it, while keeping the variable between its arguments the same. This can be an issue if a previously unpacked value is used as input to `@something`, leading to a type instability on more than two arguments (e.g. because of a fallback to `Some(nothing)`). By using different variables for each argument, type inference has an easier time handling these cases that are isolated to single branches anyway. This also adds some comments to the macro, since it's non-obvious what it does. Benchmarking the specific case I encountered this in led to a ~2x performance improvement on multiple machines. 1.10-beta3/master: ``` [sukera@tower 01]$ jl1100 -q --project=. -L 01.jl -e 'bench()' v"1.10.0-beta3" BenchmarkTools.Trial: 10000 samples with 1 evaluation. Range (min … max): 38.670 μs … 70.350 μs ┊ GC (min … max): 0.00% … 0.00% Time (median): 43.340 μs ┊ GC (median): 0.00% Time (mean ± σ): 43.395 μs ± 1.518 μs ┊ GC (mean ± σ): 0.00% ± 0.00% ▆█▂ ▁▁ ▂▂▂▂▂▂▂▂▂▁▂▂▂▃▃▃▂▂▃▃▃▂▂▂▂▂▄▇███▆██▄▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂ ▃ 38.7 μs Histogram: frequency by time 48 μs < Memory estimate: 0 bytes, allocs estimate: 0. ``` This PR: ``` [sukera@tower 01]$ julia -q --project=. -L 01.jl -e 'bench()' v"1.11.0-DEV.970" BenchmarkTools.Trial: 10000 samples with 1 evaluation. Range (min … max): 22.820 μs … 44.980 μs ┊ GC (min … max): 0.00% … 0.00% Time (median): 24.300 μs ┊ GC (median): 0.00% Time (mean ± σ): 24.370 μs ± 832.239 ns ┊ GC (mean ± σ): 0.00% ± 0.00% ▂▅▇██▇▆▅▁ ▂▂▂▂▂▂▂▂▃▃▄▅▇███████████▅▄▃▃▂▂▂▂▂▂▂▂▂▂▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▂▂ ▃ 22.8 μs Histogram: frequency by time 27.7 μs < Memory estimate: 0 bytes, allocs estimate: 0. ``` <details> <summary>Benchmarking code (spoilers for Advent Of Code 2023 Day 01, Part 01). Running this requires the input of that Advent Of Code day.</summary> ```julia using BenchmarkTools using InteractiveUtils isdigit(d::UInt8) = UInt8('0') <= d <= UInt8('9') someDigit(c::UInt8) = isdigit(c) ? Some(c - UInt8('0')) : nothing function part1(data) total = 0 may_a = nothing may_b = nothing for c in data digitRes = someDigit(c) may_a = @something may_a digitRes Some(nothing) may_b = @something digitRes may_b Some(nothing) if c == UInt8('\n') digit_a = may_a::UInt8 digit_b = may_b::UInt8 total += digit_a*0xa + digit_b may_a = nothing may_b = nothing end end return total end function bench() data = read("input.txt") display(VERSION) println() display(@benchmark part1($data)) nothing end ``` </details> <details> <summary>`@code_warntype` before</summary> ```julia julia> @code_warntype part1(data) MethodInstance for part1(::Vector{UInt8}) from part1(data) @ Main ~/Documents/projects/AOC/2023/01/01.jl:7 Arguments #self#::Core.Const(part1) data::Vector{UInt8} Locals @_3::Union{Nothing, Tuple{UInt8, Int64}} may_b::Union{Nothing, UInt8} may_a::Union{Nothing, UInt8} total::Int64 c::UInt8 digit_b::UInt8 digit_a::UInt8 val@_10::Any val@_11::Any digitRes::Union{Nothing, Some{UInt8}} @_13::Union{Some{Nothing}, Some{UInt8}, UInt8} @_14::Union{Some{Nothing}, Some{UInt8}} @_15::Some{Nothing} @_16::Union{Some{Nothing}, Some{UInt8}, UInt8} @_17::Union{Some{Nothing}, UInt8} @_18::Some{Nothing} Body::Int64 1 ── (total = 0) │ (may_a = Main.nothing) │ (may_b = Main.nothing) │ %4 = data::Vector{UInt8} │ (@_3 = Base.iterate(%4)) │ %6 = (@_3 === nothing)::Bool │ %7 = Base.not_int(%6)::Bool └─── goto mmtk#24 if not %7 2 ┄─ Core.NewvarNode(:(digit_b)) │ Core.NewvarNode(:(digit_a)) │ Core.NewvarNode(:(val@_10)) │ %12 = @_3::Tuple{UInt8, Int64} │ (c = Core.getfield(%12, 1)) │ %14 = Core.getfield(%12, 2)::Int64 │ (digitRes = Main.someDigit(c)) │ (val@_11 = may_a) │ %17 = (val@_11::Union{Nothing, UInt8} !== Base.nothing)::Bool └─── goto mmtk#4 if not %17 3 ── (@_13 = val@_11::UInt8) └─── goto mmtk#11 4 ── (val@_11 = digitRes) │ %22 = (val@_11::Union{Nothing, Some{UInt8}} !== Base.nothing)::Bool └─── goto mmtk#6 if not %22 5 ── (@_14 = val@_11::Some{UInt8}) └─── goto mmtk#10 6 ── (val@_11 = Main.Some(Main.nothing)) │ %27 = (val@_11::Core.Const(Some(nothing)) !== Base.nothing)::Core.Const(true) └─── goto mmtk#8 if not %27 7 ── (@_15 = val@_11::Core.Const(Some(nothing))) └─── goto mmtk#9 8 ── Core.Const(:(@_15 = Base.nothing)) 9 ┄─ (@_14 = @_15) 10 ┄ (@_13 = @_14) 11 ┄ %34 = @_13::Union{Some{Nothing}, Some{UInt8}, UInt8} │ (may_a = Base.something(%34)) │ (val@_10 = digitRes) │ %37 = (val@_10::Union{Nothing, Some{UInt8}} !== Base.nothing)::Bool └─── goto mmtk#13 if not %37 12 ─ (@_16 = val@_10::Some{UInt8}) └─── goto mmtk#20 13 ─ (val@_10 = may_b) │ %42 = (val@_10::Union{Nothing, UInt8} !== Base.nothing)::Bool └─── goto mmtk#15 if not %42 14 ─ (@_17 = val@_10::UInt8) └─── goto mmtk#19 15 ─ (val@_10 = Main.Some(Main.nothing)) │ %47 = (val@_10::Core.Const(Some(nothing)) !== Base.nothing)::Core.Const(true) └─── goto mmtk#17 if not %47 16 ─ (@_18 = val@_10::Core.Const(Some(nothing))) └─── goto mmtk#18 17 ─ Core.Const(:(@_18 = Base.nothing)) 18 ┄ (@_17 = @_18) 19 ┄ (@_16 = @_17) 20 ┄ %54 = @_16::Union{Some{Nothing}, Some{UInt8}, UInt8} │ (may_b = Base.something(%54)) │ %56 = c::UInt8 │ %57 = Main.UInt8('\n')::Core.Const(0x0a) │ %58 = (%56 == %57)::Bool └─── goto mmtk#22 if not %58 21 ─ (digit_a = Core.typeassert(may_a, Main.UInt8)) │ (digit_b = Core.typeassert(may_b, Main.UInt8)) │ %62 = total::Int64 │ %63 = (digit_a * 0x0a)::UInt8 │ %64 = (%63 + digit_b)::UInt8 │ (total = %62 + %64) │ (may_a = Main.nothing) └─── (may_b = Main.nothing) 22 ┄ (@_3 = Base.iterate(%4, %14)) │ %69 = (@_3 === nothing)::Bool │ %70 = Base.not_int(%69)::Bool └─── goto mmtk#24 if not %70 23 ─ goto mmtk#2 24 ┄ return total ``` </details> <details> <summary>`@code_native debuginfo=:none` Before </summary> ```julia julia> @code_native debuginfo=:none part1(data) .text .file "part1" .globl julia_part1_418 # -- Begin function julia_part1_418 .p2align 4, 0x90 .type julia_part1_418,@function julia_part1_418: # @julia_part1_418 # %bb.0: # %top push rbp mov rbp, rsp push r15 push r14 push r13 push r12 push rbx sub rsp, 40 mov rax, qword ptr [rdi + 8] test rax, rax je .LBB0_1 # %bb.2: # %L17 mov rcx, qword ptr [rdi] dec rax mov r10b, 1 xor r14d, r14d # implicit-def: $r12b # implicit-def: $r13b # implicit-def: $r9b # implicit-def: $sil mov qword ptr [rbp - 64], rax # 8-byte Spill mov al, 1 mov dword ptr [rbp - 48], eax # 4-byte Spill # implicit-def: $al # kill: killed $al xor eax, eax mov qword ptr [rbp - 56], rax # 8-byte Spill mov qword ptr [rbp - 72], rcx # 8-byte Spill # implicit-def: $cl jmp .LBB0_3 .p2align 4, 0x90 .LBB0_8: # in Loop: Header=BB0_3 Depth=1 mov dword ptr [rbp - 48], 0 # 4-byte Folded Spill .LBB0_24: # %post_union_move # in Loop: Header=BB0_3 Depth=1 movzx r13d, byte ptr [rbp - 41] # 1-byte Folded Reload mov r12d, r8d cmp qword ptr [rbp - 64], r14 # 8-byte Folded Reload je .LBB0_13 .LBB0_25: # %guard_exit113 # in Loop: Header=BB0_3 Depth=1 inc r14 mov r10d, ebx .LBB0_3: # %L19 # =>This Inner Loop Header: Depth=1 mov rax, qword ptr [rbp - 72] # 8-byte Reload xor ebx, ebx xor edi, edi movzx r15d, r9b movzx ecx, cl movzx esi, sil mov r11b, 1 # implicit-def: $r9b movzx edx, byte ptr [rax + r14] lea eax, [rdx - 58] lea r8d, [rdx - 48] cmp al, -10 setae bl setb dil test r10b, 1 cmovne r15d, edi mov edi, 0 cmovne ecx, ebx mov bl, 1 cmovne esi, edi test r15b, 1 jne .LBB0_7 # %bb.4: # %L76 # in Loop: Header=BB0_3 Depth=1 mov r11b, 2 test cl, 1 jne .LBB0_5 # %bb.6: # %L78 # in Loop: Header=BB0_3 Depth=1 mov ebx, r10d mov r9d, r15d mov byte ptr [rbp - 41], r13b # 1-byte Spill test sil, 1 je .LBB0_26 .LBB0_7: # %L82 # in Loop: Header=BB0_3 Depth=1 cmp al, -11 jbe .LBB0_9 jmp .LBB0_8 .p2align 4, 0x90 .LBB0_5: # in Loop: Header=BB0_3 Depth=1 mov ecx, r8d mov sil, 1 xor ebx, ebx mov byte ptr [rbp - 41], r8b # 1-byte Spill xor r9d, r9d xor ecx, ecx cmp al, -11 ja .LBB0_8 .LBB0_9: # %L90 # in Loop: Header=BB0_3 Depth=1 test byte ptr [rbp - 48], 1 # 1-byte Folded Reload jne .LBB0_23 # %bb.10: # %L115 # in Loop: Header=BB0_3 Depth=1 cmp dl, 10 jne .LBB0_11 # %bb.14: # %L122 # in Loop: Header=BB0_3 Depth=1 test r15b, 1 jne .LBB0_15 # %bb.12: # %L130.thread # in Loop: Header=BB0_3 Depth=1 movzx eax, byte ptr [rbp - 41] # 1-byte Folded Reload mov bl, 1 add eax, eax lea eax, [rax + 4*rax] add al, r12b movzx eax, al add qword ptr [rbp - 56], rax # 8-byte Folded Spill mov al, 1 mov dword ptr [rbp - 48], eax # 4-byte Spill cmp qword ptr [rbp - 64], r14 # 8-byte Folded Reload jne .LBB0_25 jmp .LBB0_13 .p2align 4, 0x90 .LBB0_23: # %L115.thread # in Loop: Header=BB0_3 Depth=1 mov al, 1 # implicit-def: $r8b mov dword ptr [rbp - 48], eax # 4-byte Spill cmp dl, 10 jne .LBB0_24 jmp .LBB0_21 .LBB0_11: # in Loop: Header=BB0_3 Depth=1 mov r8d, r12d jmp .LBB0_24 .LBB0_1: xor eax, eax mov qword ptr [rbp - 56], rax # 8-byte Spill .LBB0_13: # %L159 mov rax, qword ptr [rbp - 56] # 8-byte Reload add rsp, 40 pop rbx pop r12 pop r13 pop r14 pop r15 pop rbp ret .LBB0_21: # %L122.thread test r15b, 1 jne .LBB0_15 # %bb.22: # %post_box_union58 movabs rdi, offset .L_j_str1 movabs rax, offset ijl_type_error movabs rsi, 140008511215408 movabs rdx, 140008667209736 call rax .LBB0_15: # %fail cmp r11b, 1 je .LBB0_19 # %bb.16: # %fail movzx eax, r11b cmp eax, 2 jne .LBB0_17 # %bb.20: # %box_union54 movzx eax, byte ptr [rbp - 41] # 1-byte Folded Reload movabs rcx, offset jl_boxed_uint8_cache mov rdx, qword ptr [rcx + 8*rax] jmp .LBB0_18 .LBB0_26: # %L80 movabs rax, offset ijl_throw movabs rdi, 140008495049392 call rax .LBB0_19: # %box_union movabs rdx, 140008667209736 jmp .LBB0_18 .LBB0_17: xor edx, edx .LBB0_18: # %post_box_union movabs rdi, offset .L_j_str1 movabs rax, offset ijl_type_error movabs rsi, 140008511215408 call rax .Lfunc_end0: .size julia_part1_418, .Lfunc_end0-julia_part1_418 # -- End function .type .L_j_str1,@object # @_j_str1 .section .rodata.str1.1,"aMS",@progbits,1 .L_j_str1: .asciz "typeassert" .size .L_j_str1, 11 .section ".note.GNU-stack","",@progbits ``` </details> <details> <summary>`@code_warntype` After</summary> ```julia [sukera@tower 01]$ julia -q --project=. -L 01.jl julia> data = read("input.txt"); julia> @code_warntype part1(data) MethodInstance for part1(::Vector{UInt8}) from part1(data) @ Main ~/Documents/projects/AOC/2023/01/01.jl:7 Arguments #self#::Core.Const(part1) data::Vector{UInt8} Locals @_3::Union{Nothing, Tuple{UInt8, Int64}} may_b::Union{Nothing, UInt8} may_a::Union{Nothing, UInt8} total::Int64 val@_7::Union{} val@_8::Union{} c::UInt8 digit_b::UInt8 digit_a::UInt8 #JuliaLang#215::Some{Nothing} #JuliaLang#216::Union{Nothing, UInt8} #JuliaLang#217::Union{Nothing, Some{UInt8}} #JuliaLang#212::Some{Nothing} #JuliaLang#213::Union{Nothing, Some{UInt8}} #JuliaLang#214::Union{Nothing, UInt8} digitRes::Union{Nothing, Some{UInt8}} @_19::Union{Nothing, UInt8} @_20::Union{Nothing, UInt8} @_21::Nothing @_22::Union{Nothing, UInt8} @_23::Union{Nothing, UInt8} @_24::Nothing Body::Int64 1 ── (total = 0) │ (may_a = Main.nothing) │ (may_b = Main.nothing) │ %4 = data::Vector{UInt8} │ (@_3 = Base.iterate(%4)) │ %6 = @_3::Union{Nothing, Tuple{UInt8, Int64}} │ %7 = (%6 === nothing)::Bool │ %8 = Base.not_int(%7)::Bool └─── goto mmtk#24 if not %8 2 ┄─ Core.NewvarNode(:(val@_7)) │ Core.NewvarNode(:(val@_8)) │ Core.NewvarNode(:(digit_b)) │ Core.NewvarNode(:(digit_a)) │ Core.NewvarNode(:(#JuliaLang#215)) │ Core.NewvarNode(:(#JuliaLang#216)) │ Core.NewvarNode(:(#JuliaLang#217)) │ Core.NewvarNode(:(#JuliaLang#212)) │ Core.NewvarNode(:(#JuliaLang#213)) │ %19 = @_3::Tuple{UInt8, Int64} │ (c = Core.getfield(%19, 1)) │ %21 = Core.getfield(%19, 2)::Int64 │ %22 = c::UInt8 │ (digitRes = Main.someDigit(%22)) │ %24 = may_a::Union{Nothing, UInt8} │ (#JuliaLang#214 = %24) │ %26 = Base.:!::Core.Const(!) │ %27 = #JuliaLang#214::Union{Nothing, UInt8} │ %28 = Base.isnothing(%27)::Bool │ %29 = (%26)(%28)::Bool └─── goto mmtk#4 if not %29 3 ── %31 = #JuliaLang#214::UInt8 │ (@_19 = Base.something(%31)) └─── goto mmtk#11 4 ── %34 = digitRes::Union{Nothing, Some{UInt8}} │ (#JuliaLang#213 = %34) │ %36 = Base.:!::Core.Const(!) │ %37 = #JuliaLang#213::Union{Nothing, Some{UInt8}} │ %38 = Base.isnothing(%37)::Bool │ %39 = (%36)(%38)::Bool └─── goto mmtk#6 if not %39 5 ── %41 = #JuliaLang#213::Some{UInt8} │ (@_20 = Base.something(%41)) └─── goto mmtk#10 6 ── %44 = Main.Some::Core.Const(Some) │ %45 = Main.nothing::Core.Const(nothing) │ (#JuliaLang#212 = (%44)(%45)) │ %47 = Base.:!::Core.Const(!) │ %48 = #JuliaLang#212::Core.Const(Some(nothing)) │ %49 = Base.isnothing(%48)::Core.Const(false) │ %50 = (%47)(%49)::Core.Const(true) └─── goto mmtk#8 if not %50 7 ── %52 = #JuliaLang#212::Core.Const(Some(nothing)) │ (@_21 = Base.something(%52)) └─── goto mmtk#9 8 ── Core.Const(nothing) │ Core.Const(:(val@_8 = Base.something(Base.nothing))) │ Core.Const(nothing) │ Core.Const(:(val@_8)) └─── Core.Const(:(@_21 = %58)) 9 ┄─ %60 = @_21::Core.Const(nothing) └─── (@_20 = %60) 10 ┄ %62 = @_20::Union{Nothing, UInt8} └─── (@_19 = %62) 11 ┄ %64 = @_19::Union{Nothing, UInt8} │ (may_a = %64) │ %66 = digitRes::Union{Nothing, Some{UInt8}} │ (#JuliaLang#217 = %66) │ %68 = Base.:!::Core.Const(!) │ %69 = #JuliaLang#217::Union{Nothing, Some{UInt8}} │ %70 = Base.isnothing(%69)::Bool │ %71 = (%68)(%70)::Bool └─── goto mmtk#13 if not %71 12 ─ %73 = #JuliaLang#217::Some{UInt8} │ (@_22 = Base.something(%73)) └─── goto mmtk#20 13 ─ %76 = may_b::Union{Nothing, UInt8} │ (#JuliaLang#216 = %76) │ %78 = Base.:!::Core.Const(!) │ %79 = #JuliaLang#216::Union{Nothing, UInt8} │ %80 = Base.isnothing(%79)::Bool │ %81 = (%78)(%80)::Bool └─── goto mmtk#15 if not %81 14 ─ %83 = #JuliaLang#216::UInt8 │ (@_23 = Base.something(%83)) └─── goto mmtk#19 15 ─ %86 = Main.Some::Core.Const(Some) │ %87 = Main.nothing::Core.Const(nothing) │ (#JuliaLang#215 = (%86)(%87)) │ %89 = Base.:!::Core.Const(!) │ %90 = #JuliaLang#215::Core.Const(Some(nothing)) │ %91 = Base.isnothing(%90)::Core.Const(false) │ %92 = (%89)(%91)::Core.Const(true) └─── goto mmtk#17 if not %92 16 ─ %94 = #JuliaLang#215::Core.Const(Some(nothing)) │ (@_24 = Base.something(%94)) └─── goto mmtk#18 17 ─ Core.Const(nothing) │ Core.Const(:(val@_7 = Base.something(Base.nothing))) │ Core.Const(nothing) │ Core.Const(:(val@_7)) └─── Core.Const(:(@_24 = %100)) 18 ┄ %102 = @_24::Core.Const(nothing) └─── (@_23 = %102) 19 ┄ %104 = @_23::Union{Nothing, UInt8} └─── (@_22 = %104) 20 ┄ %106 = @_22::Union{Nothing, UInt8} │ (may_b = %106) │ %108 = Main.:(==)::Core.Const(==) │ %109 = c::UInt8 │ %110 = Main.UInt8('\n')::Core.Const(0x0a) │ %111 = (%108)(%109, %110)::Bool └─── goto mmtk#22 if not %111 21 ─ %113 = may_a::Union{Nothing, UInt8} │ (digit_a = Core.typeassert(%113, Main.UInt8)) │ %115 = may_b::Union{Nothing, UInt8} │ (digit_b = Core.typeassert(%115, Main.UInt8)) │ %117 = Main.:+::Core.Const(+) │ %118 = total::Int64 │ %119 = Main.:+::Core.Const(+) │ %120 = Main.:*::Core.Const(*) │ %121 = digit_a::UInt8 │ %122 = (%120)(%121, 0x0a)::UInt8 │ %123 = digit_b::UInt8 │ %124 = (%119)(%122, %123)::UInt8 │ (total = (%117)(%118, %124)) │ (may_a = Main.nothing) └─── (may_b = Main.nothing) 22 ┄ (@_3 = Base.iterate(%4, %21)) │ %129 = @_3::Union{Nothing, Tuple{UInt8, Int64}} │ %130 = (%129 === nothing)::Bool │ %131 = Base.not_int(%130)::Bool └─── goto mmtk#24 if not %131 23 ─ goto mmtk#2 24 ┄ %134 = total::Int64 └─── return %134 ``` </details> <details> <summary>`@code_native debuginfo=:none` After </summary> ```julia julia> @code_native debuginfo=:none part1(data) .text .file "part1" .globl julia_part1_1203 # -- Begin function julia_part1_1203 .p2align 4, 0x90 .type julia_part1_1203,@function julia_part1_1203: # @julia_part1_1203 ; Function Signature: part1(Array{UInt8, 1}) # %bb.0: # %top #DEBUG_VALUE: part1:data <- [DW_OP_deref] $rdi push rbp mov rbp, rsp push r15 push r14 push r13 push r12 push rbx sub rsp, 40 vxorps xmm0, xmm0, xmm0 #APP mov rax, qword ptr fs:[0] #NO_APP lea rdx, [rbp - 64] vmovaps xmmword ptr [rbp - 64], xmm0 mov qword ptr [rbp - 48], 0 mov rcx, qword ptr [rax - 8] mov qword ptr [rbp - 64], 4 mov rax, qword ptr [rcx] mov qword ptr [rbp - 72], rcx # 8-byte Spill mov qword ptr [rbp - 56], rax mov qword ptr [rcx], rdx #DEBUG_VALUE: part1:data <- [DW_OP_deref] 0 mov r15, qword ptr [rdi + 16] test r15, r15 je .LBB0_1 # %bb.2: # %L34 mov r14, qword ptr [rdi] dec r15 mov r11b, 1 mov r13b, 1 # implicit-def: $r12b # implicit-def: $r10b xor eax, eax jmp .LBB0_3 .p2align 4, 0x90 .LBB0_4: # in Loop: Header=BB0_3 Depth=1 xor r11d, r11d mov ebx, edi mov r10d, r8d .LBB0_9: # %L114 # in Loop: Header=BB0_3 Depth=1 mov r12d, esi test r15, r15 je .LBB0_12 .LBB0_10: # %guard_exit126 # in Loop: Header=BB0_3 Depth=1 inc r14 dec r15 mov r13d, ebx .LBB0_3: # %L36 # =>This Inner Loop Header: Depth=1 movzx edx, byte ptr [r14] test r13b, 1 movzx edi, r13b mov ebx, 1 mov ecx, 0 cmove ebx, edi cmovne edi, ecx movzx ecx, r10b lea esi, [rdx - 48] lea r9d, [rdx - 58] movzx r8d, sil cmove r8d, ecx cmp r9b, -11 ja .LBB0_4 # %bb.5: # %L89 # in Loop: Header=BB0_3 Depth=1 test r11b, 1 jne .LBB0_8 # %bb.6: # %L102 # in Loop: Header=BB0_3 Depth=1 cmp dl, 10 jne .LBB0_7 # %bb.13: # %L106 # in Loop: Header=BB0_3 Depth=1 test r13b, 1 jne .LBB0_14 # %bb.11: # %L114.thread # in Loop: Header=BB0_3 Depth=1 add ecx, ecx mov bl, 1 mov r11b, 1 lea ecx, [rcx + 4*rcx] add cl, r12b movzx ecx, cl add rax, rcx test r15, r15 jne .LBB0_10 jmp .LBB0_12 .p2align 4, 0x90 .LBB0_8: # %L102.thread # in Loop: Header=BB0_3 Depth=1 mov r11b, 1 # implicit-def: $sil cmp dl, 10 jne .LBB0_9 jmp .LBB0_15 .LBB0_7: # in Loop: Header=BB0_3 Depth=1 mov esi, r12d jmp .LBB0_9 .LBB0_1: xor eax, eax .LBB0_12: # %L154 mov rcx, qword ptr [rbp - 56] mov rdx, qword ptr [rbp - 72] # 8-byte Reload mov qword ptr [rdx], rcx add rsp, 40 pop rbx pop r12 pop r13 pop r14 pop r15 pop rbp ret .LBB0_15: # %L106.thread test r13b, 1 jne .LBB0_14 # %bb.16: # %post_box_union47 movabs rax, offset jl_nothing movabs rcx, offset jl_small_typeof movabs rdi, offset ".L_j_str_typeassert#1" mov rdx, qword ptr [rax] mov rsi, qword ptr [rcx + 336] movabs rax, offset ijl_type_error mov qword ptr [rbp - 48], rsi call rax .LBB0_14: # %post_box_union movabs rax, offset jl_nothing movabs rcx, offset jl_small_typeof movabs rdi, offset ".L_j_str_typeassert#1" mov rdx, qword ptr [rax] mov rsi, qword ptr [rcx + 336] movabs rax, offset ijl_type_error mov qword ptr [rbp - 48], rsi call rax .Lfunc_end0: .size julia_part1_1203, .Lfunc_end0-julia_part1_1203 # -- End function .type ".L_j_str_typeassert#1",@object # @"_j_str_typeassert#1" .section .rodata.str1.1,"aMS",@progbits,1 ".L_j_str_typeassert#1": .asciz "typeassert" .size ".L_j_str_typeassert#1", 11 .section ".note.GNU-stack","",@progbits ``` </details> Co-authored-by: Sukera <[email protected]>

@turbo

Adds a convenient way to enable PGO+LTO on Julia and LLVM together: 1. `cd contrib/pgo-lto` 2. `make -j$(nproc) stage1` 3. `make clean-profiles` 4. `./stage1.build/julia -O3 -e 'using Pkg; Pkg.add("LoopVectorization"); Pkg.test("LoopVectorization")'` 5. `make -j$(nproc) stage2` <details> <summary>* Output looks roughly like as follows</summary> ```c++ $ make -C contrib/pgo-lto top make: Entering directory '/dev/shm/julia/contrib/pgo-lto' llvm-profdata show --topn=50 /dev/shm/julia/contrib/pgo-lto/profiles/merged.prof | c++filt Instrumentation level: IR entry_first = 0 Total functions: 85943 Maximum function count: 7867557260 Maximum internal block count: 3468437590 Top 50 functions with the largest internal block counts: llvm::BitVector::operator|=(llvm::BitVector const&), max count = 7867557260 LateLowerGCFrame::ComputeLiveness(State&), max count = 3468437590 llvm::hashing::detail::hash_combine_recursive_helper::hash_combine_recursive_helper(), max count = 1742259834 llvm::SUnit::addPred(llvm::SDep const&, bool), max count = 511396575 llvm::LiveRange::overlaps(llvm::LiveRange const&, llvm::CoalescerPair const&, llvm::SlotIndexes const&) const, max count = 508061762 llvm::StringMapImpl::LookupBucketFor(llvm::StringRef), max count = 505682177 std::map<llvm::BasicBlock*, BBState, std::less<llvm::BasicBlock*>, std::allocator<std::pair<llvm::BasicBlock* const, BBState> > >::operator[](llvm::BasicBlock* const&), max count = 395628888 llvm::LiveRange::advanceTo(llvm::LiveRange::Segment const*, llvm::SlotIndex) const, max count = 384642728 llvm::LiveRange::isLiveAtIndexes(llvm::ArrayRef<llvm::SlotIndex>) const, max count = 380291040 llvm::PassRegistry::enumerateWith(llvm::PassRegistrationListener*), max count = 352313953 ijl_method_instance_add_backedge, max count = 349608221 llvm::SUnit::ComputeHeight(), max count = 336604330 llvm::LiveRange::advanceTo(llvm::LiveRange::Segment*, llvm::SlotIndex), max count = 331030109 llvm::SmallPtrSetImplBase::insert_imp(void const*), max count = 272966545 llvm::LiveIntervals::checkRegMaskInterference(llvm::LiveInterval&, llvm::BitVector&), max count = 257449540 LateLowerGCFrame::ComputeLiveSets(State&), max count = 252096274 /dev/shm/julia/src/jltypes.c:has_free_typevars, max count = 230879464 ijl_get_pgcstack, max count = 216953592 LateLowerGCFrame::RefineLiveSet(llvm::BitVector&, State&, std::vector<int, std::allocator<int> > const&), max count = 188013152 /dev/shm/julia/src/flisp/flisp.c:apply_cl, max count = 174863813 /dev/shm/julia/src/flisp/builtins.c:fl_memq, max count = 168621603 ``` </details> This results quite often in spectacular speedups for time to first X as it reduces the time spent in LLVM optimization passes by 25 or even 30%. Example 1: ```julia using LoopVectorization function f!(a, b) @turbo for i in eachindex(a) a[i] *= b[i] end return a end f!(rand(1), rand(1)) ``` ```console $ time ./julia -O3 lv.jl ``` Without PGO+LTO: 14.801s With PGO+LTO: 11.978s (-19%) Example 2: ```console $ time ./julia -e 'using Pkg; Pkg.test("Unitful");' ``` Without PGO+LTO: 1m47.688s With PGO+LTO: 1m35.704s (-11%) Example 3 (taken from issue JuliaLang#45395, which is almost only LLVM): ```console $ JULIA_LLVM_ARGS=-time-passes ./julia script-45395.jl ``` Without PGO+LTO: ``` ===-------------------------------------------------------------------------=== ... Pass execution timing report ... ===-------------------------------------------------------------------------=== Total Execution Time: 101.0130 seconds (98.6253 wall clock) ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name --- 53.6961 ( 54.7%) 0.1050 ( 3.8%) 53.8012 ( 53.3%) 53.8045 ( 54.6%) Unroll loops 25.5423 ( 26.0%) 0.0072 ( 0.3%) 25.5495 ( 25.3%) 25.5444 ( 25.9%) Global Value Numbering 7.1995 ( 7.3%) 0.0526 ( 1.9%) 7.2521 ( 7.2%) 7.2517 ( 7.4%) Induction Variable Simplification 6.0541 ( 5.1%) 0.0098 ( 0.3%) 5.0639 ( 5.0%) 5.0561 ( 5.1%) Combine redundant instructions mmtk#2 ``` With PGO+LTO: ``` ===-------------------------------------------------------------------------=== ... Pass execution timing report ... ===-------------------------------------------------------------------------=== Total Execution Time: 72.6507 seconds (70.1337 wall clock) ---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name --- 36.0894 ( 51.7%) 0.0825 ( 2.9%) 36.1719 ( 49.8%) 36.1738 ( 51.6%) Unroll loops 16.5713 ( 23.7%) 0.0129 ( 0.5%) 16.5843 ( 22.8%) 16.5794 ( 23.6%) Global Value Numbering 5.9047 ( 8.5%) 0.0395 ( 1.4%) 5.9442 ( 8.2%) 5.9438 ( 8.5%) Induction Variable Simplification 4.7566 ( 6.8%) 0.0078 ( 0.3%) 4.7645 ( 6.6%) 4.7575 ( 6.8%) Combine redundant instructions mmtk#2 ``` Or -28% time spent in LLVM. `perf` reports show this is mostly fewer instructions and reduction in icache misses. --- Finally there's a significant reduction in binary sizes. For libLLVM.so: ``` 79M usr/lib/libLLVM-13jl.so (before) 67M usr/lib/libLLVM-13jl.so (after) ``` And it can be reduced by another 2MB with `--icf=safe` when using LLD as a linker anyways. - [x] Two out-of-source builds would be better than a single in-source build, so that it's easier to find good profile data --------- Co-authored-by: Oscar Smith <[email protected]> Co-authored-by: Lilith Orion Hafner <[email protected]>

@isdefined

…aLang#55600) As an application of JuliaLang#55545, this commit avoids the insertion of `:throw_undef_if_not` nodes when the defined-ness of a slot is guaranteed by abstract interpretation. ```julia julia> function isdefined_nothrow(c, x) local val if c val = x end if @isdefined val return val end return zero(Int) end; julia> @code_typed isdefined_nothrow(true, 42) ``` ```diff diff --git a/old b/new index c4980a5c9c..3d1d6d30f0 100644 --- a/old +++ b/new @@ -4,7 +4,6 @@ CodeInfo( 3 ┄ %3 = φ (mmtk#2 => x, #1 => #undef)::Int64 │ %4 = φ (mmtk#2 => true, #1 => false)::Bool └── goto mmtk#5 if not %4 -4 ─ $(Expr(:throw_undef_if_not, :val, :(%4)))::Any -└── return %3 +4 ─ return %3 5 ─ return 0 ) => Int64 ```

Prior to this, especially on macOS, the gc-safepoint here would cause the process to segfault as we had already freed the current_task state. Rearrange this code so that the GC interactions (except for the atomic store to current_task) are all handled before entering GC safe, and then signaling the thread is deleted (via setting current_task = NULL, published by jl_unlock_profile_wr to other threads) is last. ``` ERROR: Exception handler triggered on unmanaged thread. Process 53827 stopped * thread mmtk#5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008) frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt] 269 assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD); 270 jl_atomic_store_release(&ptls->gc_state, state); 271 if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE) -> 272 jl_gc_safepoint_(ptls); 273 return old_state; 274 } 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, Target 0: (julia) stopped. (lldb) up frame #1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt] 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, 276 int8_t state) 277 { -> 278 return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state)); 279 } 280 #ifdef __clang_gcanalyzer__ 281 // these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically) (lldb) frame mmtk#2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt] 534 ptls->root_task = NULL; 535 jl_free_thread_gc_state(ptls); 536 // then park in safe-region -> 537 (void)jl_gc_safe_enter(ptls); 538 } ``` (test incorporated into JuliaLang#55793)

@alexfanqi

Rebase and extension of @alexfanqi's initial work on porting Julia to RISC-V. Requires LLVM 19. Tested on a VisionFive2, built with: ```make MARCH := rv64gc_zba_zbb MCPU := sifive-u74 USE_BINARYBUILDER:=0 DEPS_GIT = llvm override LLVM_VER=19.1.1 override LLVM_BRANCH=julia-release/19.x override LLVM_SHA1=julia-release/19.x ``` ```julia-repl ❯ ./julia _ _ _ _(_)_ | Documentation: https://docs.julialang.org (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 1.12.0-DEV.1374 (2024-10-14) _/ |\__'_|_|_|\__'_| | riscv/25092a3982* (fork: 1 commits, 0 days) |__/ | julia> versioninfo(; verbose=true) Julia Version 1.12.0-DEV.1374 Commit 25092a3* (2024-10-14 09:57 UTC) Platform Info: OS: Linux (riscv64-unknown-linux-gnu) uname: Linux 6.11.3-1-riscv64 #1 SMP Debian 6.11.3-1 (2024-10-10) riscv64 unknown CPU: unknown: speed user nice sys idle irq #1 1500 MHz 922 s 0 s 265 s 160953 s 0 s mmtk#2 1500 MHz 457 s 0 s 280 s 161521 s 0 s mmtk#3 1500 MHz 452 s 0 s 270 s 160911 s 0 s mmtk#4 1500 MHz 638 s 15 s 301 s 161340 s 0 s Memory: 7.760246276855469 GB (7474.08203125 MB free) Uptime: 16260.13 sec Load Avg: 0.25 0.23 0.1 WORD_SIZE: 64 LLVM: libLLVM-19.1.1 (ORCJIT, sifive-u74) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) Environment: HOME = /home/tim PATH = /home/tim/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games TERM = xterm-256color julia> ccall(:jl_dump_host_cpu, Nothing, ()) CPU: sifive-u74 Features: +zbb,+d,+i,+f,+c,+a,+zba,+m,-zvbc,-zksed,-zvfhmin,-zbkc,-zkne,-zksh,-zfh,-zfhmin,-zknh,-v,-zihintpause,-zicboz,-zbs,-zvknha,-zvksed,-zfa,-ztso,-zbc,-zvknhb,-zihintntl,-zknd,-zvbb,-zbkx,-zkt,-zvkt,-zicond,-zvksh,-zvfh,-zvkg,-zvkb,-zbkb,-zvkned julia> @code_native debuginfo=:none 1+2. .text .attribute 4, 16 .attribute 5, "rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0_zifencei2p0_zmmul1p0_zba1p0_zbb1p0" .file "+" .globl "julia_+_3003" .p2align 1 .type "julia_+_3003",@function "julia_+_3003": addi sp, sp, -16 sd ra, 8(sp) sd s0, 0(sp) addi s0, sp, 16 fcvt.d.l fa5, a0 ld ra, 8(sp) ld s0, 0(sp) fadd.d fa0, fa5, fa0 addi sp, sp, 16 ret .Lfunc_end0: .size "julia_+_3003", .Lfunc_end0-"julia_+_3003" .type ".L+Core.Float64#3005",@object .section .data.rel.ro,"aw",@progbits .p2align 3, 0x0 ".L+Core.Float64#3005": .quad ".L+Core.Float64#3005.jit" .size ".L+Core.Float64#3005", 8 .set ".L+Core.Float64#3005.jit", 272467692544 .size ".L+Core.Float64#3005.jit", 8 .section ".note.GNU-stack","",@progbits ``` Lots of bugs guaranteed, but with this we at least have a functional build and REPL for further development by whoever is interested. Also requires Linux 6.4+, since the fallback processor detection used here relies on LLVM's `sys::getHostCPUFeatures`, which for RISC-V is implemented using hwprobe introduced in 6.4. We could probably add a fallback that parses `/proc/cpuinfo`, either by building a CPU database much like how we've done for AArch64, or by parsing the actual ISA string contained there. That would probably also be a good place to add support for profiles, which are supposedly the way forward to package RISC-V binaries. That can happen in follow-up PRs though. For now, on older kernels, use the `-C` arg to Julia to specify an ISA. Co-authored-by: Alex Fan <[email protected]>

* Add filesystem func to transform a path to a URI (#55454) In a few places across Base and the stdlib, we emit paths that we like people to be able to click on in their terminal and editor. Up to this point, we have relied on auto-filepath detection, but this does not allow for alternative link text, such as contracted paths. Doing so (via OSC 8 terminal links for example) requires filepath URI encoding. This functionality was previously part of a PR modifying stacktrace printing (#51816), but after that became held up for unrelated reasons and another PR appeared that would benefit from this utility (#55335), I've split out this functionality so it can be used before the stacktrace printing PR is resolved. * constrain the path argument of `include` functions to `AbstractString` (#55466) Each `Module` defined with `module` automatically gets an `include` function with two methods. Each of those two methods takes a file path as its last argument. Even though the path argument is unconstrained by dispatch, it's documented as constrained with `::AbstractString`: https://docs.julialang.org/en/v1.11-dev/base/base/#include Furthermore, I think that any invocation of `include` with a non-`AbstractString` path will necessarily throw a `MethodError` eventually. Thus this change should be harmless. Adding the type constraint to the path argument is an improvement because any possible exception would be thrown earlier than before. Apart from modules defined with `module`, the same issue is present with the anonymous modules created by `evalfile`, which is also addressed. Sidenote: `evalfile` seems to be completely untested apart from the test added here. Co-authored-by: Florian <[email protected]> * Mmap: fix grow! for non file IOs (#55849) Fixes https://github.com/JuliaLang/julia/issues/54203 Requires #55641 Based on https://github.com/JuliaLang/julia/pull/55641#issuecomment-2334162489 cc. @JakeZw @ronisbr --------- Co-authored-by: Jameson Nash <[email protected]> * codegen: split gc roots from other bits on stack (#55767) In order to help avoid memory provenance issues, and better utilize stack space (somewhat), and use FCA less, change the preferred representation of an immutable object to be a pair of `<packed-data,roots>` values. This packing requires some care at the boundaries and if the expected field alignment exceeds that of a pointer. The change is expected to eventually make codegen more flexible at representing unions of values with both bits and pointer regions. Eventually we can also have someone improve the late-gc-lowering pass to take advantage of this increased information accuracy, but currently it will not be any better than before at laying out the frame. * Refactoring to be considered before adding MMTk * Removing jl_gc_notify_image_load, since it's a new function and not part of the refactoring * Moving gc_enable code to gc-common.c * Addressing PR comments * Push resolution of merge conflict * Removing jl_gc_mark_queue_obj_explicit extern definition from scheduler.c * Don't need the getter function since it's possible to use jl_small_typeof directly * WIP: Adding support for MMTk/Immix * Refactoring to be considered before adding MMTk * Adding fastpath allocation * Fixing removed newlines * Refactoring to be considered before adding MMTk * Adding a few comments; Moving some functions to be closer together * Fixing merge conflicts * Applying changes from refactoring before adding MMTk * Update TaskLocalRNG docstring according to #49110 (#55863) Since #49110, which is included in 1.10 and 1.11, spawning a task no longer advances the parent task's RNG state, so this statement in the docs was incorrect. * Root globals in toplevel exprs (#54433) This fixes #54422, the code here assumes that top level exprs are always rooted, but I don't see that referenced anywhere else, or guaranteed, so conservatively always root objects that show up in code. * codegen: fix alignment typos (#55880) So easy to type jl_datatype_align to get the natural alignment instead of julia_alignment to get the actual alignment. This should fix the Revise workload. Change is visible with ``` julia> code_llvm(Random.XoshiroSimd.forkRand, (Random.TaskLocalRNG, Base.Val{8})) ``` * Fix some corner cases of `isapprox` with unsigned integers (#55828) * 🤖 [master] Bump the Pkg stdlib from ef9f76c17 to 51d4910c1 (#55896) * Profile: fix order of fields in heapsnapshot & improve formatting (#55890) * Profile: Improve generation of clickable terminal links (#55857) * inference: add missing `TypeVar` handling for `instanceof_tfunc` (#55884) I thought these sort of problems had been addressed by d60f92c, but it seems some were missed. Specifically, `t.a` and `t.b` from `t::Union` could be `TypeVar`, and if they are passed to a subroutine or recursed without being unwrapped or rewrapped, errors like JuliaLang/julia#55882 could occur. This commit resolves the issue by calling `unwraptv` in the `Union` handling within `instanceof_tfunc`. I also found a similar issue inside `nfields_tfunc`, so that has also been fixed, and test cases have been added. While I haven't been able to make up a test case specifically for the fix in `instanceof_tfunc`, I have confirmed that this commit certainly fixes the issue reported in JuliaLang/julia#55882. - fixes JuliaLang/julia#55882 * Install terminfo data under /usr/share/julia (#55881) Just like all other libraries, we don't want internal Julia files to mess with system files. Introduced by https://github.com/JuliaLang/julia/pull/55411. * expose metric to report reasons why full GCs were triggered (#55826) Additional GC observability tool. This will help us to diagnose why some of our servers are triggering so many full GCs in certain circumstances. * Revert "Improve printing of several arguments" (#55894) Reverts JuliaLang/julia#55754 as it overrode some performance heuristics which appeared to be giving a significant gain/loss in performance: Closes https://github.com/JuliaLang/julia/issues/55893 * Do not trigger deprecation warnings in `Test.detect_ambiguities` and `Test.detect_unbound_args` (#55869) #55868 * do not intentionally suppress errors in precompile script from being reported or failing the result (#55909) I was slightly annoying that the build was set up to succeed if this step failed, so I removed the error suppression and fixed up the script slightly * Remove eigvecs method for SymTridiagonal (#55903) The fallback method does the same, so this specialized method isn't necessary * add --trim option for generating smaller binaries (#55047) This adds a command line option `--trim` that builds images where code is only included if it is statically reachable from methods marked using the new function `entrypoint`. Compile-time errors are given for call sites that are too dynamic to allow trimming the call graph (however there is an `unsafe` option if you want to try building anyway to see what happens). The PR has two other components. One is changes to Base that generally allow more code to be compiled in this mode. These changes will either be merged in separate PRs or moved to a separate part of the workflow (where we will build a custom system image for this purpose). The branch is set up this way to make it easy to check out and try the functionality. The other component is everything in the `juliac/` directory, which implements a compiler driver script based on this new option, along with some examples and tests. This will eventually become a package "app" that depends on PackageCompiler and provides a CLI for all of this stuff, so it will not be merged here. To try an example: ``` julia contrib/juliac.jl --output-exe hello --trim test/trimming/hello.jl ``` When stripped the resulting executable is currently about 900kb on my machine. Also includes a lot of work by @topolarity --------- Co-authored-by: Gabriel Baraldi <[email protected]> Co-authored-by: Tim Holy <[email protected]> Co-authored-by: Cody Tapscott <[email protected]> * fix rawbigints OOB issues (#55917) Fixes issues introduced in #50691 and found in #55906: * use `@inbounds` and `@boundscheck` macros in rawbigints, for catching OOB with `--check-bounds=yes` * fix OOB in `truncate` * prevent loading other extensions when precompiling an extension (#55589) The current way of loading extensions when precompiling an extension very easily leads to cycles. For example, if you have more than one extension and you happen to transitively depend on the triggers of one of your extensions you will immediately hit a cycle where the extensions will try to load each other indefinitely. This is an issue because you cannot directly influence your transitive dependency graph so from this p.o.v the current system of loading extension is "unsound". The test added here checks this scenario and we can now precompile and load it without any warnings or issues. Would have made https://github.com/JuliaLang/julia/issues/55517 a non issue. Fixes https://github.com/JuliaLang/julia/issues/55557 --------- Co-authored-by: KristofferC <[email protected]> * TOML: Avoid type-pirating `Base.TOML.Parser` (#55892) Since stdlibs can be duplicated but Base never is, `Base.require_stdlib` makes type piracy even more complicated than it normally would be. To adapt, this changes `TOML.Parser` to be a type defined by the TOML stdlib, so that we can define methods on it without committing type-piracy and avoid problems like Pkg.jl#4017 Resolves https://github.com/JuliaLang/Pkg.jl/issues/4017#issuecomment-2377589989 * [FileWatching] fix PollingFileWatcher design and add workaround for a stat bug What started as an innocent fix for a stat bug on Apple (#48667) turned into a full blown investigation into the design problems with the libuv backend for PollingFileWatcher, and writing my own implementation of it instead which could avoid those singled-threaded concurrency bugs. * [FileWatching] fix FileMonitor similarly and improve pidfile reliability Previously pidfile used the same poll_interval as sleep to detect if this code made any concurrency mistakes, but we do not really need to do that once FileMonitor is fixed to be reliable in the presence of parallel concurrency (instead of using watch_file). * [FileWatching] reorganize file and add docs * Add `--trace-dispatch` (#55848) * relocation: account for trailing path separator in depot paths (#55355) Fixes #55340 * change compiler to be stackless (#55575) This change ensures the compiler uses very little stack, making it compatible with running on any arbitrary system stack size and depths much more reliably. It also could be further modified now to easily add various forms of pause-able/resumable inference, since there is no implicit state on the stack--everything is local and explicit now. Whereas before, less than 900 frames would crash in less than a second: ``` $ time ./julia -e 'f(::Val{N}) where {N} = N <= 0 ? 0 : f(Val(N - 1)); f(Val(1000))' Warning: detected a stack overflow; program state may be corrupted, so further execution might be unreliable. Internal error: during type inference of f(Base.Val{1000}) Encountered stack overflow. This might be caused by recursion over very long tuples or argument lists. [23763] signal 6: Abort trap: 6 in expression starting at none:1 __pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line) Allocations: 1 (Pool: 1; Big: 0); GC: 0 Abort trap: 6 real 0m0.233s user 0m0.165s sys 0m0.049s ```` Now: it is effectively unlimited, as long as you are willing to wait for it: ``` $ time ./julia -e 'f(::Val{N}) where {N} = N <= 0 ? 0 : f(Val(N - 1)); f(Val(50000))' info: inference of f(Base.Val{50000}) from f(Base.Val{N}) where {N} exceeding 2500 frames (may be slow). info: inference of f(Base.Val{50000}) from f(Base.Val{N}) where {N} exceeding 5000 frames (may be slow). info: inference of f(Base.Val{50000}) from f(Base.Val{N}) where {N} exceeding 10000 frames (may be slow). info: inference of f(Base.Val{50000}) from f(Base.Val{N}) where {N} exceeding 20000 frames (may be slow). info: inference of f(Base.Val{50000}) from f(Base.Val{N}) where {N} exceeding 40000 frames (may be slow). real 7m4.988s $ time ./julia -e 'f(::Val{N}) where {N} = N <= 0 ? 0 : f(Val(N - 1)); f(Val(1000))' real 0m0.214s user 0m0.164s sys 0m0.044s $ time ./julia -e '@noinline f(::Val{N}) where {N} = N <= 0 ? GC.safepoint() : f(Val(N - 1)); f(Val(5000))' info: inference of f(Base.Val{5000}) from f(Base.Val{N}) where {N} exceeding 2500 frames (may be slow). info: inference of f(Base.Val{5000}) from f(Base.Val{N}) where {N} exceeding 5000 frames (may be slow). real 0m8.609s user 0m8.358s sys 0m0.240s ``` * optimizer: simplify the finalizer inlining pass a bit (#55934) Minor adjustments have been made to the algorithm of the finalizer inlining pass. Previously, it required that the finalizer registration dominate all uses, but this is not always necessary as far as the finalizer inlining point dominates all the uses. So the check has been relaxed. Other minor fixes have been made as well, but their importance is low. * Limit `@inbounds` to indexing in the dual-iterator branch in `copyto_unaliased!` (#55919) This simplifies the `copyto_unalised!` implementation where the source and destination have different `IndexStyle`s, and limits the `@inbounds` to only the indexing operation. In particular, the iteration over `eachindex(dest)` is not marked as `@inbounds` anymore. This seems to help with performance when the destination uses Cartesian indexing. Reduced implementation of the branch: ```julia function copyto_proposed!(dest, src) axes(dest) == axes(src) || throw(ArgumentError("incompatible sizes")) iterdest, itersrc = eachindex(dest), eachindex(src) for (destind, srcind) in zip(iterdest, itersrc) @inbounds dest[destind] = src[srcind] end dest end function copyto_current!(dest, src) axes(dest) == axes(src) || throw(ArgumentError("incompatible sizes")) iterdest, itersrc = eachindex(dest), eachindex(src) ret = iterate(iterdest) @inbounds for a in src idx, state = ret::NTuple{2,Any} dest[idx] = a ret = iterate(iterdest, state) end dest end function copyto_current_limitinbounds!(dest, src) axes(dest) == axes(src) || throw(ArgumentError("incompatible sizes")) iterdest, itersrc = eachindex(dest), eachindex(src) ret = iterate(iterdest) for isrc in itersrc idx, state = ret::NTuple{2,Any} @inbounds dest[idx] = src[isrc] ret = iterate(iterdest, state) end dest end ``` ```julia julia> a = zeros(40000,4000); b = rand(size(a)...); julia> av = view(a, UnitRange.(axes(a))...); julia> @btime copyto_current!($av, $b); 617.704 ms (0 allocations: 0 bytes) julia> @btime copyto_current_limitinbounds!($av, $b); 304.146 ms (0 allocations: 0 bytes) julia> @btime copyto_proposed!($av, $b); 240.217 ms (0 allocations: 0 bytes) julia> versioninfo() Julia Version 1.12.0-DEV.1260 Commit 4a4ca9c8152 (2024-09-28 01:49 UTC) Build Info: Official https://julialang.org release Platform Info: OS: Linux (x86_64-linux-gnu) CPU: 8 × Intel(R) Core(TM) i5-10310U CPU @ 1.70GHz WORD_SIZE: 64 LLVM: libLLVM-18.1.7 (ORCJIT, skylake) Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores) Environment: JULIA_EDITOR = subl ``` I'm not quite certain why the proposed implementation here (`copyto_proposed!`) is even faster than `copyto_current_limitinbounds!`. In any case, `copyto_proposed!` is easier to read, so I'm not complaining. This fixes https://github.com/JuliaLang/julia/issues/53158 * Strong zero in Diagonal triple multiplication (#55927) Currently, triple multiplication with a `LinearAlgebra.BandedMatrix` sandwiched between two `Diagonal`s isn't associative, as this is implemented using broadcasting, which doesn't assume a strong zero, whereas the two-term matrix multiplication does. ```julia julia> D = Diagonal(StepRangeLen(NaN, 0, 3)); julia> B = Bidiagonal(1:3, 1:2, :U); julia> D * B * D 3×3 Matrix{Float64}: NaN NaN NaN NaN NaN NaN NaN NaN NaN julia> (D * B) * D 3×3 Bidiagonal{Float64, Vector{Float64}}: NaN NaN ⋅ ⋅ NaN NaN ⋅ ⋅ NaN julia> D * (B * D) 3×3 Bidiagonal{Float64, Vector{Float64}}: NaN NaN ⋅ ⋅ NaN NaN ⋅ ⋅ NaN ``` This PR ensures that the 3-term multiplication is evaluated as a sequence of two-term multiplications, which fixes this issue. This also improves performance, as only the bands need to be evaluated now. ```julia julia> D = Diagonal(1:1000); B = Bidiagonal(1:1000, 1:999, :U); julia> @btime $D * $B * $D; 656.364 μs (11 allocations: 7.63 MiB) # v"1.12.0-DEV.1262" 2.483 μs (12 allocations: 31.50 KiB) # This PR ``` * Fix dispatch on `alg` in Float16 Hermitian eigen (#55928) Currently, ```julia julia> using LinearAlgebra julia> A = Hermitian(reshape(Float16[1:16;], 4, 4)); julia> eigen(A).values |> typeof Vector{Float16} (alias for Array{Float16, 1}) julia> eigen(A, LinearAlgebra.QRIteration()).values |> typeof Vector{Float32} (alias for Array{Float32, 1}) ``` This PR moves the specialization on the `eltype` to an internal method, so that firstly all `alg`s dispatch to that method, and secondly, there are no ambiguities introduce by specializing the top-level `eigen`. The latter currently causes test failures in `StaticArrays` (https://github.com/JuliaArrays/StaticArrays.jl/actions/runs/11092206012/job/30816955210?pr=1279), and should be fixed by this PR. * Remove specialized `ishermitian` method for `Diagonal{<:Real}` (#55948) The fallback method for `Diagonal{<:Number}` handles this already by checking that the `diag` is real, so we don't need this additional specialization. * Fix logic in `?` docstring example (#55945) * fix `unwrap_macrocalls` (#55950) The implementation of `unwrap_macrocalls` has assumed that what `:macrocall` wraps is always an `Expr` object, but that is not necessarily correct: ```julia julia> Base.@assume_effects :nothrow @show 42 ERROR: LoadError: TypeError: in typeassert, expected Expr, got a value of type Int64 Stacktrace: [1] unwrap_macrocalls(ex::Expr) @ Base ./expr.jl:906 [2] var"@assume_effects"(__source__::LineNumberNode, __module__::Module, args::Vararg{Any}) @ Base ./expr.jl:756 in expression starting at REPL[1]:1 ``` This commit addresses this issue. * make faster BigFloats (#55906) We can coalesce the two required allocations for the MFPR BigFloat API design into one allocation, hopefully giving a easy performance boost. It would have been slightly easier and more efficient if MPFR BigFloat was already a VLA instead of containing a pointer here, but that does not prevent the optimization. * Add propagate_inbounds_meta to atomic genericmemory ops (#55902) `memoryref(mem, i)` will otherwise emit a boundscheck. ``` ; │ @ /home/vchuravy/WorkstealingQueues/src/CLL.jl:53 within `setindex_atomic!` @ genericmemory.jl:329 ; │┌ @ boot.jl:545 within `memoryref` %ptls_field = getelementptr inbounds i8, ptr %tls_pgcstack, i64 16 %ptls_load = load ptr, ptr %ptls_field, align 8 %"box::GenericMemoryRef" = call noalias nonnull align 8 dereferenceable(32) ptr @ijl_gc_small_alloc(ptr %ptls_load, i32 552, i32 32, i64 23456076646928) #9 %"box::GenericMemoryRef.tag_addr" = getelementptr inbounds i64, ptr %"box::GenericMemoryRef", i64 -1 store atomic i64 23456076646928, ptr %"box::GenericMemoryRef.tag_addr" unordered, align 8 store ptr %memoryref_data, ptr %"box::GenericMemoryRef", align 8 %.repack8 = getelementptr inbounds { ptr, ptr }, ptr %"box::GenericMemoryRef", i64 0, i32 1 store ptr %memoryref_mem, ptr %.repack8, align 8 call void @ijl_bounds_error_int(ptr nonnull %"box::GenericMemoryRef", i64 %7) unreachable ``` For the Julia code: ```julia function Base.setindex_atomic!(buf::WSBuffer{T}, order::Symbol, val::T, idx::Int64) where T @inbounds Base.setindex_atomic!(buf.buffer, order, val,((idx - 1) & buf.mask) + 1) end ``` from https://github.com/gbaraldi/WorkstealingQueues.jl/blob/0ebc57237cf0c90feedf99e4338577d04b67805b/src/CLL.jl#L41 * fix rounding mode in construction of `BigFloat` from pi (#55911) The default argument of the method was outdated, reading the global default rounding directly, bypassing the `ScopedValue` stuff. * fix `nonsetable_type_hint_handler` (#55962) The current implementation is wrong, causing it to display inappropriate hints like the following: ```julia julia> s = Some("foo"); julia> s[] = "bar" ERROR: MethodError: no method matching setindex!(::Some{String}, ::String) The function `setindex!` exists, but no method is defined for this combination of argument types. You attempted to index the type String, rather than an instance of the type. Make sure you create the type using its constructor: d = String([...]) rather than d = String Stacktrace: [1] top-level scope @ REPL[2]:1 ``` * REPL: make UndefVarError aware of imported modules (#55932) * fix test/staged.jl (#55967) In particular, the implementation of `overdub_generator54341` was dangerous. This fixes it up. * Explicitly store a module's location (#55963) Revise wants to know what file a module's `module` definition is in. Currently it does this by looking at the source location for the implicitly generated `eval` method. This is terrible for two reasons: 1. The method may not exist if the module is a baremodule (which is not particularly common, which is probably why we haven't seen it). 2. The fact that the implicitly generated `eval` method has this location information is an implementation detail that I'd like to get rid of (#55949). This PR adds explicit file/line info to `Module`, so that Revise doesn't have to use the hack anymore. * mergewith: add single argument example to docstring (#55964) I ran into this edge case. I though it should be documented. --------- Co-authored-by: Lilith Orion Hafner <[email protected]> * [build] avoid libedit linkage and align libccalllazy* SONAMEs (#55968) While building the 1.11.0-rc4 in Homebrew[^1] in preparation for 1.11.0 release (and to confirm Sequoia successfully builds) I noticed some odd linkage for our Linux builds, which included of: 1. LLVM libraries were linking to `libedit.so`, e.g. ``` Dynamic Section: NEEDED libedit.so.0 NEEDED libz.so.1 NEEDED libzstd.so.1 NEEDED libstdc++.so.6 NEEDED libm.so.6 NEEDED libgcc_s.so.1 NEEDED libc.so.6 NEEDED ld-linux-x86-64.so.2 SONAME libLLVM-16jl.so ``` CMakeCache.txt showed ``` //Use libedit if available. LLVM_ENABLE_LIBEDIT:BOOL=ON ``` Which might be overriding `HAVE_LIBEDIT` at https://github.com/JuliaLang/llvm-project/blob/julia-release/16.x/llvm/cmake/config-ix.cmake#L222-L225. So just added `LLVM_ENABLE_LIBEDIT` 2. Wasn't sure if there was a reason for this but `libccalllazy*` had mismatched SONAME: ```console ❯ objdump -p lib/julia/libccalllazy* | rg '\.so' lib/julia/libccalllazybar.so: file format elf64-x86-64 NEEDED ccalllazyfoo.so SONAME ccalllazybar.so lib/julia/libccalllazyfoo.so: file format elf64-x86-64 SONAME ccalllazyfoo.so ``` Modifying this, but can drop if intentional. --- [^1]: https://github.com/Homebrew/homebrew-core/pull/192116 * Add missing `copy!(::AbstractMatrix, ::UniformScaling)` method (#55970) Hi everyone! First PR to Julia here. It was noticed in a Slack thread yesterday that `copy!(A, I)` doesn't work, but `copyto!(A, I)` does. This PR adds the missing method for `copy!(::AbstractMatrix, ::UniformScaling)`, which simply defers to `copyto!`, and corresponding tests. I added a `compat` notice for Julia 1.12. --------- Co-authored-by: Lilith Orion Hafner <[email protected]> * Add forward progress update to NEWS.md (#54089) Closes #40009 which was left open because of the needs news tag. --------- Co-authored-by: Ian Butterworth <[email protected]> * Fix an intermittent test failure in `core` test (#55973) The test wants to assert that `Module` is not resolved in `Main`, but other tests do resolve this identifier, so the test can fail depending on test order (and I've been seeing such failures on CI recently). Fix that by running the test in a fresh subprocess. * fix comma logic in time_print (#55977) Minor formatting fix * optimizer: fix up the inlining algorithm to use correct `nargs`/`isva` (#55976) It appears that inlining.jl was not updated in JuliaLang/julia#54341. Specifically, using `nargs`/`isva` from `mi.def::Method` in `ir_prepare_inlining!` causes the following error to occur: ```julia function generate_lambda_ex(world::UInt, source::LineNumberNode, argnames, spnames, @nospecialize body) stub = Core.GeneratedFunctionStub(identity, Core.svec(argnames...), Core.svec(spnames...)) return stub(world, source, body) end function overdubbee54341(a, b) return a + b end const overdubee_codeinfo54341 = code_lowered(overdubbee54341, Tuple{Any, Any})[1] function overdub_generator54341(world::UInt, source::LineNumberNode, selftype, fargtypes) if length(fargtypes) != 2 return generate_lambda_ex(world, source, (:overdub54341, :args), (), :(error("Wrong number of arguments"))) else return copy(overdubee_codeinfo54341) end end @eval function overdub54341(args...) $(Expr(:meta, :generated, overdub_generator54341)) $(Expr(:meta, :generated_only)) end topfunc(x) = overdub54341(x, 2) ``` ```julia julia> topfunc(1) Internal error: during type inference of topfunc(Int64) Encountered unexpected error in runtime: BoundsError(a=Array{Any, 1}(dims=(2,), mem=Memory{Any}(8, 0x10632e780)[SSAValue(2), SSAValue(3), #<null>, #<null>, #<null>, #<null>, #<null>, #<null>]), i=(3,)) throw_boundserror at ./essentials.jl:14 getindex at ./essentials.jl:909 [inlined] ssa_substitute_op! at ./compiler/ssair/inlining.jl:1798 ssa_substitute_op! at ./compiler/ssair/inlining.jl:1852 ir_inline_item! at ./compiler/ssair/inlining.jl:386 ... ``` This commit updates the abstract interpretation and inlining algorithm to use the `nargs`/`isva` values held by `CodeInfo`. Similar modifications have also been made to EscapeAnalysis.jl. @nanosoldier `runbenchmarks("inference", vs=":master")` * Add `.zed` directory to `.gitignore` (#55974) Similar to the `vscode` config directory, we may ignore the `zed` directory as well. * typeintersect: reduce unneeded allocations from `merge_env` `merge_env` and `final_merge_env` could be skipped for emptiness test or if we know there's only 1 valid Union state. * typeintersect: trunc env before nested `intersect_all` if valid. This only covers the simplest cases. We might want a full dependence analysis and keep env length minimum in the future. * `@time` actually fix time report commas & add tests (#55982) https://github.com/JuliaLang/julia/pull/55977 looked simple but wasn't quite right because of a bad pattern in the lock conflicts report section. So fix and add tests. * adjust EA to JuliaLang/julia#52527 (#55986) `EnterNode.catch_dest` can now be `0` after the `try`/`catch` elision feature implemented in JuliaLang/julia#52527, and we actually need to adjust `EscapeAnalysis.compute_frameinfo` too. * Improvements to JITLink Seeing what this will look like, since it has a number of features (delayed compilation, concurrent compilation) that are starting to become important, so it would be nice to switch to only supporting one common implementation of memory management. Refs #50248 I am expecting https://github.com/llvm/llvm-project/issues/63236 may cause some problems, since we reconfigured some CI machines to minimize that issue, but it is still likely relevant. * rewrite catchjmp asm to use normal relocations instead of manual editing * add logic to prefer loading modules that are already loaded (#55908) Iterate over the list of existing loaded modules for PkgId whenever loading a new module for PkgId, so that we will use that existing build_id content if it otherwise passes the other stale_checks. * Apple: fix bus error on smaller readonly file in unix (#55859) Enables the fix for #28245 in #44354 for Apple now that the Julia bugs are fixed by #55641 and #55877. Closes #28245 * Add `Float16` to `Base.HWReal` (#55929) * docs: make mod an operator (#55988) * InteractiveUtils: add `@trace_compile` and `@trace_dispatch` (#55915) * Profile: document heap snapshot viewing tools (#55743) * [REPL] Fix #55850 by using `safe_realpath` instead of `abspath` in `projname` (#55851) * optimizer: enable load forwarding with the `finalizer` elision (#55991) When the finalizer elision pass is used, load forwarding is not performed currently, regardless of whether the pass succeeds or not. But this is not necessary, and by keeping the `setfield!` call, we can safely forward `getfield` even if finalizer elision is tried. * Avoid `stat`-ing stdlib path if it's unreadable (#55992) * doc: manual: cmd: fix Markdown in table entry for `--trim` (#55979) * Avoid conversions to `Float64` in non-literal powers of `Float16` (#55994) Co-authored-by: Alex Arslan <[email protected]> * Remove unreachable error branch in memset calls (and in repeat) (#55985) Some places use the pattern memset(A, v, length(A)), which requires a conversion UInt(length(A)). This is technically fallible, but can't actually fail when A is a Memory or Array. Remove the dead error branch by casting to UInt instead. Similarly, in repeat(x, r), r is first checked to be nonnegative, then converted to UInt, then used in multiple calls where it is converted to UInt each time. Here, only do it once. * fix up docstring of `mod` (#56000) * fix typos (#56008) these are all in markdown files Co-authored-by: spaette <[email protected]> * Vectorise random vectors of `Float16` (#55997) * Clarify `div` docstring for floating-point input (#55918) Closes #55837 This is a variant of the warning found in the `fld` docstring clarifying floating-point behaviour. * improve getproperty(Pairs) warnings (#55989) - Only call `depwarn` if the field is `itr` or `data`; otherwise let the field error happen as normal - Give a more specific deprecation warning. * Document type-piracy / type-leakage restrictions for `require_stdlib` (#56005) I was a recent offender in https://github.com/JuliaLang/Pkg.jl/issues/4017#issuecomment-2377589989 This PR tries to lay down some guidelines for the behavior that stdlibs and the callers of `require_stdlib` must adhere to to avoid "duplicate stdlib" bugs These bugs are particularly nasty because they are experienced semi-rarely and under pretty specific circumstances (they only occur when `require_stdlib` loads another copy of a stdlib, often in a particular order and/or with a particular state of your pre-compile / loading cache) so they may make it a long way through a pre-release cycle without an actionable bug report. * [LinearAlgebra] Remove unreliable doctests (#56011) The exact textual representation of the output of these doctests depend on the specific kernel used by the BLAS backend, and can vary between versions of OpenBLAS (as it did in #41973), or between different CPUs, which makes these doctests unreliable. Fix #55998. * cleanup functions of Hermitian matrices (#55951) The functions of Hermitian matrices are a bit of a mess. For example, if we have a Hermitian matrix `a` with negative eigenvalues, `a^0.5` doesn't produce the `Symmetric` wrapper, but `sqrt(a)` does. On the other hand, if we have a positive definite `b`, `b^0.5` will be `Hermitian`, but `sqrt(b)` will be `Symmetric`: ```julia using LinearAlgebra a = Hermitian([1.0 2.0;2.0 1.0]) a^0.5 sqrt(a) b = Hermitian([2.0 1.0; 1.0 2.0]) b^0.5 sqrt(b) ``` This sort of arbitrary assignment of wrappers happens with pretty much all functions defined there. There's also some oddities, such as `cis` being the only function defined for `SymTridiagonal`, even though all `eigen`-based functions work, and `cbrt` being the only function not defined for complex Hermitian matrices. I did a cleanup: I defined all functions for `SymTridiagonal` and `Hermitian{<:Complex}`, and always assigned the appropriate wrapper, preserving the input one when possible. There's an inconsistency remaining that I didn't fix, that only `sqrt` and `log` accept a tolerance argument, as changing that is probably breaking. There were also hardly any tests that I could find (only `exp`, `log`, `cis`, and `sqrt`). I'm happy to add them if it's desired. * Fix no-arg `ScopedValues.@with` within a scope (#56019) Fixes https://github.com/JuliaLang/julia/issues/56017 * LinearAlgebra: make matprod_dest public (#55537) Currently, in a matrix multiplication `A * B`, we use `B` to construct the destination. However, this may not produce the optimal destination type, and is essentially single-dispatch. Letting packages specialize `matprod_dest` would help us obtain the optimal type by dispatching on both the arguments. This may significantly improve performance in the matrix multiplication. As an example: ```julia julia> using LinearAlgebra, FillArrays, SparseArrays julia> F = Fill(3, 10, 10); julia> s = sprand(10, 10, 0.1); julia> @btime $F * $s; 15.225 μs (10 allocations: 4.14 KiB) julia> typeof(F * s) SparseMatrixCSC{Float64, Int64} julia> nnz(F * s) 80 julia> VERSION v"1.12.0-DEV.1074" ``` In this case, the destination is a sparse matrix with 80% of its elements filled and being set one-by-one, which is terrible for performance. Instead, if we specialize `matprod_dest` to return a dense destination, we may obtain ```julia julia> LinearAlgebra.matprod_dest(F::FillArrays.AbstractFill, S::SparseMatrixCSC, ::Type{T}) where {T} = Matrix{T}(undef, size(F,1), size(S,2)) julia> @btime $F * $s; 754.632 ns (2 allocations: 944 bytes) julia> typeof(F * s) Matrix{Float64} ``` Potentially, this may be improved further by specializing `mul!`, but this is a 20x improvement just by choosing the right destination. Since this is being made public, we may want to bikeshed on an appropriate name for the function. * Sockets: Warn when local network access not granted. (#56023) Works around https://github.com/JuliaLang/julia/issues/56022 * Update test due to switch to intel syntax by default in #48103 (#55993) * add require_lock call to maybe_loaded_precompile (#56027) If we expect this to be a public API (https://github.com/timholy/Revise.jl for some reason is trying to access this state), we should lock around it for consistency with the other similar functions. Needed for https://github.com/timholy/Revise.jl/pull/856 * fix `power_by_squaring`: use `promote` instead of type inference (#55634) Fixes #53504 Fixes #55633 * Don't show keymap `@error` for hints (#56041) It's too disruptive to show errors for hints. The error will still be shown if tab is pressed. Helps issues like https://github.com/JuliaLang/julia/issues/56037 * Refactoring to be considered before adding MMTk * Removing jl_gc_notify_image_load, since it's a new function and not part of the refactoring * Moving gc_enable code to gc-common.c * Addressing PR comments * Push resolution of merge conflict * Removing jl_gc_mark_queue_obj_explicit extern definition from scheduler.c * Don't need the getter function since it's possible to use jl_small_typeof directly * Remove extern from free_stack declaration in julia_internal.h * Putting everything that is common GC tls into gc-tls-common.h * Typo * Adding gc-tls-common.h to Makefile as a public header * Removing gc-tls-common fields from gc-tls-mmtk.h * Fix typo in sockets tests. (#56038) * EA: use `is_mutation_free_argtype` for the escapability check (#56028) EA has been using `isbitstype` for type-level escapability checks, but a better criterion (`is_mutation_free`) is available these days, so we would like to use that instead. * effects: fix `Base.@_noub_meta` (#56061) This had the incorrect number of arguments to `Expr(:purity, ...)` causing it to be silently ignored. * effects: improve `:noub_if_noinbounds` documentation (#56060) Just a small touch-up * Disallow assigning asymmetric values to SymTridiagonal (#56068) Currently, we can assign an asymmetric value to a `SymTridiagonal`, which goes against what `setindex!` is expected to do. This is because `SymTridiagonal` symmetrizes the values along the diagonal, so setting a diagonal entry to an asymmetric value would lead to a subsequent `getindex` producing a different result. ```julia julia> s = SMatrix{2,2}(1:4); julia> S = SymTridiagonal(fill(s,4), fill(s,3)) 4×4 SymTridiagonal{SMatrix{2, 2, Int64, 4}, Vector{SMatrix{2, 2, Int64, 4}}}: [1 3; 3 4] [1 3; 2 4] ⋅ ⋅ [1 2; 3 4] [1 3; 3 4] [1 3; 2 4] ⋅ ⋅ [1 2; 3 4] [1 3; 3 4] [1 3; 2 4] ⋅ ⋅ [1 2; 3 4] [1 3; 3 4] julia> S[1,1] = s 2×2 SMatrix{2, 2, Int64, 4} with indices SOneTo(2)×SOneTo(2): 1 3 2 4 julia> S[1,1] == s false julia> S[1,1] 2×2 Symmetric{Int64, SMatrix{2, 2, Int64, 4}} with indices SOneTo(2)×SOneTo(2): 1 3 3 4 ``` After this PR, ```julia julia> S[1,1] = s ERROR: ArgumentError: cannot set a diagonal entry of a SymTridiagonal to an asymmetric value ``` * Remove unused matrix type params in diag methods (#56048) These parameters are not used in the method, and are unnecessary for dispatch. * LinearAlgebra: diagzero for non-OneTo axes (#55252) Currently, the off-diagonal zeros for a block-`Diagonal` matrix is computed using `diagzero`, which calls `zeros` for the sizes of the elements. This returns an `Array`, unless one specializes `diagzero` for the custom `Diagonal` matrix type. This PR defines a `zeroslike` function that dispatches on the axes of the elements, which lets packages specialize on the axes to return custom `AbstractArray`s. Choosing to specialize on the `eltype` avoids the need to specialize on the container, and allows packages to return appropriate types for custom axis types. With this, ```julia julia> LinearAlgebra.zeroslike(::Type{S}, ax::Tuple{SOneTo, Vararg{SOneTo}}) where {S<:SMatrix} = SMatrix{map(length, ax)...}(ntuple(_->zero(eltype(S)), prod(length, ax))) julia> D = Diagonal(fill(SMatrix{2,3}(1:6), 2)) 2×2 Diagonal{SMatrix{2, 3, Int64, 6}, Vector{SMatrix{2, 3, Int64, 6}}}: [1 3 5; 2 4 6] ⋅ ⋅ [1 3 5; 2 4 6] julia> D[1,2] # now an SMatrix 2×3 SMatrix{2, 3, Int64, 6} with indices SOneTo(2)×SOneTo(3): 0 0 0 0 0 0 julia> LinearAlgebra.zeroslike(::Type{S}, ax::Tuple{SOneTo, Vararg{SOneTo}}) where {S<:MMatrix} = MMatrix{map(length, ax)...}(ntuple(_->zero(eltype(S)), prod(length, ax))) julia> D = Diagonal(fill(MMatrix{2,3}(1:6), 2)) 2×2 Diagonal{MMatrix{2, 3, Int64, 6}, Vector{MMatrix{2, 3, Int64, 6}}}: [1 3 5; 2 4 6] ⋅ ⋅ [1 3 5; 2 4 6] julia> D[1,2] # now an MMatrix 2×3 MMatrix{2, 3, Int64, 6} with indices SOneTo(2)×SOneTo(3): 0 0 0 0 0 0 ``` The reason this can't be the default behavior is that we are not guaranteed that there exists a `similar` method that accepts the combination of axes. This is why we have to fall back to using the sizes, unless a specialized method is provided by a package. One positive outcome of this is that indexing into such a block-diagonal matrix will now usually be type-stable, which mitigates https://github.com/JuliaLang/julia/issues/45535 to some extent (although it doesn't resolve the issue). I've also updated the `getindex` for `Bidiagonal` to use `diagzero`, instead of the similarly defined `bidiagzero` function that it was using. Structured block matrices may now use `diagzero` uniformly to generate the zero elements. * Multi-argument `gcdx(a, b, c...)` (#55935) Previously, `gcdx` only worked for two arguments - but the underlying idea extends to any (nonzero) number of arguments. Similarly, `gcd` already works for 1, 2, 3+ arguments. This PR implements the 1 and 3+ argument versions of `gcdx`, following the [wiki page](https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm#The_case_of_more_than_two_numbers) for the Extended Euclidean algorithm. * Refactoring to be considered before adding MMTk * Removing jl_gc_notify_image_load, since it's a new function and not part of the refactoring * Moving gc_enable code to gc-common.c * Addressing PR comments * Push resolution of merge conflict * Removing jl_gc_mark_queue_obj_explicit extern definition from scheduler.c * Don't need the getter function since it's possible to use jl_small_typeof directly * Remove extern from free_stack declaration in julia_internal.h * Putting everything that is common GC tls into gc-tls-common.h * Typo * Adding gc-tls-common.h to Makefile as a public header * Adding jl_full_sweep_reasons since timing.jl depends on it * Fixing issue with jl_full_sweep_reasons (missing constants) * fix `_growbeg!` unncessary resizing (#56029) This was very explicitly designed such that if there was a bunch of extra space at the end of the array, we would copy rather than allocating, but by making `newmemlen` be at least `overallocation(memlen)` rather than `overallocation(len)`, this branch was never hit. found by https://github.com/JuliaLang/julia/issues/56026 * REPL: hide any prints to stdio during `complete_line` (#55959) * teach llvm-alloc-helpers about `gc_loaded` (#56030) combined with https://github.com/JuliaLang/julia/pull/55913, the compiler is smart enough to fully remove ``` function f() m = Memory{Int}(undef, 3) @inbounds m[1] = 2 @inbounds m[2] = 2 @inbounds m[3] = 4 @inbounds return m[1] + m[2] + m[3] end ``` * mpfr: prevent changing precision (#56049) Changing precision requires reallocating the data field, which is better done by making a new BigFloat (since they are conceptually immutable anyways). Also do a bit a cleanup while here. Closes #56044 * stackwalk: fix jl_thread_suspend_and_get_state race (#56047) There was a missing re-assignment of old = -1; at the end of that loop which means in the ABA case, we accidentally actually acquire the lock on the thread despite not actually having stopped the thread; or in the counter-case, we try to run through this logic with old==-1 on the next iteration, and that isn't valid either (jl_thread_suspend_and_get_state should return failure and the loop will abort too early). Fix #56046 * irrationals: restrict assume effects annotations to known types (#55886) Other changes: * replace `:total` with the less powerful `:foldable` * add an `<:Integer` dispatch constraint on the `rationalize` method, closes #55872 * replace `Rational{<:Integer}` with just `Rational`, they're equal Other issues, related to `BigFloat` precision, are still present in irrationals.jl, to be fixed by followup PRs, including #55853. Fixes #55874 * update `hash` doc string: `widen` not required any more (#55867) Implementing `widen` isn't a requirement any more, since #26022. * Merge `diag` methods for triangular matrices (#56086) * slightly improve inference in precompilation code (#56084) Avoids the ``` 11: signature Tuple{typeof(convert), Type{String}, Any} triggered MethodInstance for Base.Precompilation.ExplicitEnv(::String) (84 children) ``` shown in https://github.com/JuliaLang/julia/issues/56080#issuecomment-2404765120 Co-authored-by: KristofferC <[email protected]> * avoid defining `convert(Vector{String}, ...)` in LibGit2 (#56082) This is a weird conversion function to define. Seems cleaner to use the iteration interface for this. Also avoids some invalidations (https://github.com/JuliaLang/julia/issues/56080#issuecomment-2404765120) Co-authored-by: KristofferC <[email protected]> * array: inline `convert` where possible (#56034) This improves a common scenario, where someone wants to `push!` a poorly-typed object onto a well-typed Vector. For example: ```julia const NT = @NamedTuple{x::Int,y::Any} foo(v::Vector{NT}, x::Int, @nospecialize(y)) = push!(v, (; x, y)) ``` The `(; x, y)` is slightly poorly-typed here. It could have any type for its `.y` field before it is converted inside the `push!` to a NamedTuple with `y::Any` Without this PR, the dispatch for this `push!` cannot be inferred: ```julia julia> code_typed(foo, (Vector{NT}, Int, Any))[1] CodeInfo( 1 ─ ... │ %4 = %new(%3, x, y)::NamedTuple{(:x, :y), <:Tuple{Int64, Any}} │ %5 = Main.push!(v, %4)::Vector{@NamedTuple{x::Int64, y}} └── return %5 ) => Vector{@NamedTuple{x::Int64, y}} ``` With this PR, the above dynamic call is fully statically resolved and inlined (and therefore `--trim` compatible) * Remove some unnecessary `real` specializations for structured matrices (#56083) The `real(::AbstractArray{<:Rea})` fallback method should handle these cases correctly. * Combine `diag` methods for `SymTridiagonal` (#56014) Currently, there are two branches, one for an `eltype` that is a `Number`, and the other that deals with generic `eltype`s. They do similar things, so we may combine these, and use branches wherever necessary to retain the performance. We also may replace explicit materialized arrays by generators in `copyto!`. Overall, this improves performance in `diag` for matrices of matrices, whereas the performance in the common case of matrices of numbers remains unchanged. ```julia julia> using StaticArrays, LinearAlgebra julia> s = SMatrix{2,2}(1:4); julia> S = SymTridiagonal(fill(s,100), fill(s,99)); julia> @btime diag($S); 1.292 μs (5 allocations: 7.16 KiB) # nightly, v"1.12.0-DEV.1317" 685.012 ns (3 allocations: 3.19 KiB) # This PR ``` This PR also allows computing the `diag` for more values of the band index `n`: ```julia julia> diag(S,99) 1-element Vector{SMatrix{2, 2, Int64, 4}}: [0 0; 0 0] ``` This would work as long as `getindex` works for the `SymTridiagonal` for that band, and the zero element may be converted to the `eltype`. * fix `Vararg{T,T} where T` crashing `code_typed` (#56081) Not sure this is the right place to fix this error, perhaps `match.spec_types` should always be a tuple of valid types? fixes #55916 --------- Co-authored-by: Jameson Nash <[email protected]> * [libblastrampoline_jll] Upgrade to v5.11.1 (#56094) v5.11.1 is a patch release with a couple of RISC-V fixes. * Revert "REPL: hide any prints to stdio during `complete_line`" (#56102) * Remove warning from c when binding is ambiguous (#56103) * make `Base.ANSIIterator` have a concrete field (#56088) Avoids the invalidation ``` backedges: 1: superseding sizeof(s::AbstractString) @ Base strings/basic.jl:177 with MethodInstance for sizeof(::AbstractString) (75 children) ``` shown in https://github.com/JuliaLang/julia/issues/56080#issuecomment-2404765120. Co-authored-by: KristofferC <[email protected]> * Subtype: some performance tuning. (#56007) The main motivation of this PR is to fix #55807. dc689fe8700f70f4a4e2dbaaf270f26b87e79e04 tries to remove the slow `may_contain_union_decision` check by re-organizing the code path. Now the fast path has been removed and most of its optimization has been integrated into the preserved slow path. Since the slow path stores all inner ∃ decisions on the outer most R stack, there might be overflow risk. aee69a41441b4306ba3ee5e845bc96cb45d9b327 should fix that concern. The reported MWE now becomes ```julia 0.000002 seconds 0.000040 seconds (105 allocations: 4.828 KiB, 52.00% compilation time) 0.000023 seconds (105 allocations: 4.828 KiB, 49.36% compilation time) 0.000026 seconds (105 allocations: 4.828 KiB, 50.38% compilation time) 0.000027 seconds (105 allocations: 4.828 KiB, 54.95% compilation time) 0.000019 seconds (106 allocations: 4.922 KiB, 49.73% compilation time) 0.000024 seconds (105 allocations: 4.828 KiB, 52.24% compilation time) ``` Local bench also shows that 72855cd slightly accelerates `OmniPackage.jl`'s loading ```julia julia> @time using OmniPackage # v1.11rc4 20.525278 seconds (25.36 M allocations: 1.606 GiB, 8.48% gc time, 12.89% compilation time: 77% of which was recompilation) # v1.11rc4+aee69a4+72855cd 19.527871 seconds (24.92 M allocations: 1.593 GiB, 8.88% gc time, 15.13% compilation time: 82% of which was recompilation) ``` * rearrange jl_delete_thread to be thread-safe (#56097) Prior to this, especially on macOS, the gc-safepoint here would cause the process to segfault as we had already freed the current_task state. Rearrange this code so that the GC interactions (except for the atomic store to current_task) are all handled before entering GC safe, and then signaling the thread is deleted (via setting current_task = NULL, published by jl_unlock_profile_wr to other threads) is last. ``` ERROR: Exception handler triggered on unmanaged thread. Process 53827 stopped * thread #5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008) frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt] 269 assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD); 270 jl_atomic_store_release(&ptls->gc_state, state); 271 if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE) -> 272 jl_gc_safepoint_(ptls); 273 return old_state; 274 } 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, Target 0: (julia) stopped. (lldb) up frame #1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt] 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, 276 int8_t state) 277 { -> 278 return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state)); 279 } 280 #ifdef __clang_gcanalyzer__ 281 // these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically) (lldb) frame #2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt] 534 ptls->root_task = NULL; 535 jl_free_thread_gc_state(ptls); 536 // then park in safe-region -> 537 (void)jl_gc_safe_enter(ptls); 538 } ``` (test incorporated into https://github.com/JuliaLang/julia/pull/55793) * OpenBLAS: Use dynamic architecture support on AArch64. (#56107) We already do so on Yggdrasil, so this just makes both source and binary builds behave similarly. Closes https://github.com/JuliaLang/julia/issues/56075 * IRShow: label builtin / intrinsic / dynamic calls in `code_typed` (#56036) This makes it much easier to spot dynamic dispatches * 🤖 [master] Bump the Pkg stdlib from 51d4910c1 to fbaa2e337 (#56124) * Fix type instability of closures capturing types (2) (#40985) Instead of closures lowering to `typeof` for the types of captured fields, this introduces a new function `_typeof_captured_variable` that returns `Type{T}` if `T` is a type (w/o free typevars). - replaces/closes #35970 - fixes #23618 --------- Co-authored-by: Takafumi Arakaki <[email protected]> Co-authored-by: Shuhei Kadowaki <[email protected]> * Remove debug error statement from Makefile. (#56127) * align markdown table (#56122) @gbaraldi `#51197` @spaette `#56008` fix innocuous malalignment of table after those pulls were merged * Improve IOBuffer docs (#56024) Based on the discussion in #55978, I have tried to clarify the documentation of `IOBuffer`. * Comment out url and fix typo in stackwalk.c (#56131) Introduced in #55623 * libgit2: Always use the bundled PCRE library. (#56129) This is how Yggdrasil builds the library. * Update JLL build versions (#56133) This commit encompasses the following changes: - Updating the JLL build version for Clang, dSFMT, GMP, LibUV, LibUnwind, LLD, LLVM, libLLVM, MbedTLS, MPFR, OpenBLAS, OpenLibm, p7zip, PCRE2, SuiteSparse, and Zlib. - Updating CompilerSupportLibraries to v1.2.0. The library versions contained in this release of CSL don't differ from v1.1.1, the only difference is that v1.2.0 includes FreeBSD AArch64. - Updating nghttp2 from 1.60.0 to 1.63.0. See [here](https://github.com/nghttp2/nghttp2/releases) for changes between these versions. - Adding `aarch64-unknown-freebsd` to the list of triplets to check when refreshing checksums. Note that dependencies that link to MbedTLS (Curl, LibSSH2, LibGit2) are excluded here. They'll be updated once a resolution is reached for the OpenSSL switching saga. Once that happens, FreeBSD AArch64 should be able to be built without any dependency source builds. * typo in `Compiler.Effects` doc string: `checkbounds` -> `boundscheck` (#56140) Follows up on #56060 * HISTORY: fix missing links (#56137) * OpenBLAS: Fix cross-compilation detection for source build. (#56139) We may be cross-compiling Linux-to-Linux, in which case `BUILD_OS` == `OS`, so look at `XC_HOST` to determine whether we're cross compiling. * `diag` for `BandedMatrix`es for off-limit bands (#56065) Currently, one can only obtain the `diag` for a `BandedMatrix` (such as a `Diagonal`) when the band index is bounded by the size of the matrix. This PR relaxes this requirement to match the behavior for arrays, where `diag` returns an empty vector for a large band index instead of throwing an error. ```julia julia> D = Diagonal(ones(4)) 4×4 Diagonal{Float64, Vector{Float64}}: 1.0 ⋅ ⋅ ⋅ ⋅ 1.0 ⋅ ⋅ ⋅ ⋅ 1.0 ⋅ ⋅ ⋅ ⋅ 1.0 julia> diag(D, 10) Float64[] julia> diag(Array(D), 10) Float64[] ``` Something similar for `SymTridiagonal` is being done in https://github.com/JuliaLang/julia/pull/56014 * Port progress bar improvements from Pkg (#56125) Includes changes from https://github.com/JuliaLang/Pkg.jl/pull/4038 and https://github.com/JuliaLang/Pkg.jl/pull/4044. Co-authored-by: Kristoffer Carlsson <[email protected]> * Add support for LLVM 19 (#55650) Co-authored-by: Zentrik <[email protected]> * 🤖 [master] Bump the Pkg stdlib from fbaa2e337 to 27c1b1ee5 (#56146) * HISTORY entry for deletion of `length(::Stateful)` (#55861) xref #47790 xref #51747 xref #54953 xref #55858 * ntuple: ensure eltype is always `Int` (#55901) Fixes #55790 * Improve remarks of the alloc opt pass slightly. (#55995) The Value printer LLVM uses just prints the kind of instruction so it just shows call. --------- Co-authored-by: Oscar Smith <[email protected]> * Implement Base.fd() for TCPSocket, UDPSocket, and TCPServer (#53721) This is quite handy if you want to pass off the file descriptor to a C library. I also added a warning to the `fd()` docstring to warn folks about duplicating the file descriptor first. * Fix `JULIA_CPU_TARGET` being propagated to workers precompiling stdlib pkgimages (#54093) Apparently (thanks ChatGPT) each line in a makefile is executed in a separate shell so adding an `export` line on one line does not propagate to the next line. * Merge tr methods for triangular matrices (#56154) Since the methods do identical things, we don't need multiple of these. * Reduce duplication in triangular indexing methods (#56152) This uses an orthogonal design to reduce code duplication in the indexing methods for triangular matrices. * update LLVM docs (#56162) dump with raw=true so you don't get random erorrs, and show how to run single modules. --------- Co-authored-by: Valentin Churavy <[email protected]> Co-authored-by: Mosè Giordano <[email protected]> Co-authored-by: Jameson Nash <[email protected]> * Fix zero elements for block-matrix kron involving Diagonal (#55941) Currently, it's assumed that the zero element is identical for the matrix, but this is not necessary if the elements are matrices themselves and have different sizes. This PR ensures that `kron` for a `Diagonal` has the correct zero elements. Current: ```julia julia> D = Diagonal(1:2) 2×2 Diagonal{Int64, UnitRange{Int64}}: 1 ⋅ ⋅ 2 julia> B = reshape([ones(2,2), ones(3,2), ones(2,3), ones(3,3)], 2, 2); julia> size.(kron(D, B)) 4×4 Matrix{Tuple{Int64, Int64}}: (2, 2) (2, 3) (2, 2) (2, 2) (3, 2) (3, 3) (2, 2) (2, 2) (2, 2) (2, 2) (2, 2) (2, 3) (2, 2) (2, 2) (3, 2) (3, 3) ``` This PR ```julia julia> size.(kron(D, B)) 4×4 Matrix{Tuple{Int64, Int64}}: (2, 2) (2, 3) (2, 2) (2, 3) (3, 2) (3, 3) (3, 2) (3, 3) (2, 2) (2, 3) (2, 2) (2, 3) (3, 2) (3, 3) (3, 2) (3, 3) ``` Note the differences e.g. in the `CartesianIndex(4,1)`, `CartesianIndex(3,2)` and `CartesianIndex(3,3)` elements. * Call `MulAddMul` instead of multiplication in _generic_matmatmul! (#56089) Fix https://github.com/JuliaLang/julia/issues/56085 by calling a newly created `MulAddMul` object that only wraps the `alpha` (with `beta` set to `false`). This avoids the explicit multiplication if `alpha` is known to be `isone`. * improve `allunique`'s type stability (#56161) Caught by https://github.com/aviatesk/JET.jl/issues/667. * Add invalidation barriers for `displaysize` and `implicit_typeinfo` (#56159) These are invalidated by our own stdlibs (Dates and REPL) unfortunately so we need to put this barrier in. This fix is _very_ un-satisfying, because it doesn't do anything to solve this problem for downstream libraries that use e.g. `displaysize`. To fix that, I think we need a way to make sure callers get these invalidation barriers by default... * Fix markdown list in installation.md (#56165) Documenter.jl requires all trailing list content to follow the same indentation as the header. So, in the current view (https://docs.julialang.org/en/v1/manual/installation/#Command-line-arguments) the list appears broken. * [Random] Add more comments and a helper function in Xoshiro code (#56144) Follow up to #55994 and #55997. This should basically be a non-functional change and I see no performance difference, but the comments and the definition of a helper function should make the code easier to follow (I initially struggled in #55997) and extend to other types. * add objects to concisely specify initialization PerProcess: once per process PerThread: once per thread id PerTask: once per task object * add precompile support for recording fields to change Somewhat generalizes our support for changing Ptr to C_NULL. Not particularly fast, since it is just using the builtins implementation of setfield, and delaying the actual stores, but it should suffice. * improve OncePer implementation Address reviewer feedback, add more fixes and more tests, rename to add Once prefix. * fix use-after-free in test (detected in win32 CI) * Make loading work when stdlib deps are missing in the manifest (#56148) Closes https://github.com/JuliaLang/julia/issues/56109 Simulating a bad manifest by having `LibGit2_jll` missing as a dep of `LibGit2` in my default env, say because the manifest was generated by a different julia version or different master julia commit. ## This PR, it just works ``` julia> using Revise julia> ``` i.e. ``` % JULIA_DEBUG=loading ./julia --startup-file=no julia> using Revise ... ┌ Debug: Stdlib LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] is trying to load `LibGit2_jll` │ which is not listed as a dep in the load path manifests, so resorting to search │ in the stdlib Project.tomls for true deps └ @ Base loading.jl:387 ┌ Debug: LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] indeed depends on LibGit2_jll in project /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/LibGit2/Project.toml └ @ Base loading.jl:395 ... julia> ``` ## Master ``` julia> using Revise Info Given Revise was explicitly requested, output will be shown live ERROR: LoadError: ArgumentError: Package LibGit2 does not have LibGit2_jll in its dependencies: - Note that the following manifests in the load path were resolved with a potentially different DEV version of the current version, which may be the cause of the error. Try to re-resolve them in the current version, or consider deleting them if that fails: /Users/ian/.julia/environments/v1.12/Manifest.toml - You may have a partially installed environment. Try `Pkg.instantiate()` to ensure all packages in the environment are installed. - Or, if you have LibGit2 checked out for development and have added LibGit2_jll as a dependency but haven't updated your primary environment's manifest file, try `Pkg.resolve()`. - Otherwise you may need to report an issue with LibGit2 ... ``` * Remove llvm-muladd pass and move it's functionality to to llvm-simdloop (#55802) Closes https://github.com/JuliaLang/julia/issues/55785 I'm not sure if we want to backport this like this. Because that removes some functionality (the pass itself). So LLVM.jl and friends might need annoying version code. We can maybe keep the code there and just not run the pass in a backport. * Fix implicit `convert(String, ...)` in several places (#56174) This removes several `convert(String, ...)` from this code, which really shouldn't be something we invalidate on in the first place (see https://github.com/JuliaLang/julia/issues/56173) but this is still an improvement in code quality so let's take it. * Change annotations to use a NamedTuple (#55741) Due to popular demand, the type of annotations is to be changed from a `Tuple{UnitRange{Int}, Pair{Symbol, Any}}` to a `NamedTuple{(:region, :label, :value), Tuple{UnitRange{Int}, Symbol, Any}}`. This requires the expected code churn to `strings/annotated.jl`, and some changes to the StyledStrings and JuliaSyntaxHighlighting libraries. Closes #55249 and closes #55245. * Getting rid of mmtk_julia.c in the binding and moving it to gc-mmtk.c * Trying to organize and label the code in gc-mmtk.c * Remove redundant `convert` in `_setindex!` (#56178) Follow up to #56034, ref: https://github.com/JuliaLang/julia/pull/56034#discussion_r1798573573. --------- Co-authored-by: Cody Tapscott <[email protected]> * Improve type inference of Artifacts.jl (#56118) This also has some changes that move platform selection to compile time together with https://github.com/JuliaPackaging/JLLWrappers.jl/commit/45cc04963f3c99d4eb902f97528fe16fc37002cc, move the platform selection to compile time. (this helps juliac a ton) * Initial support for RISC-V (#56105) Rebase and extension of @alexfanqi's initial work on porting Julia to RISC-V. Requires LLVM 19. Tested on a VisionFive2, built with: ```make MARCH := rv64gc_zba_zbb MCPU := sifive-u74 USE_BINARYBUILDER:=0 DEPS_GIT = llvm override LLVM_VER=19.1.1 override LLVM_BRANCH=julia-release/19.x override LLVM_SHA1=julia-release/19.x ``` ```julia-repl ❯ ./julia _ _ _ _(_)_ | Documentation: https://docs.julialang.org (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 1.12.0-DEV.1374 (2024-10-14) _/ |\__'_|_|_|\__'_| | riscv/25092a3982* (fork: 1 commits, 0 days) |__/ | julia> versioninfo(; verbose=true) Julia Version 1.12.0-DEV.1374 Commit 25092a3982* (2024-10-14 09:57 UTC) Platform Info: …

@Compiler

Fixes https://buildkite.com/julialang/julia-master/builds/46446#0195f712-1844-4e81-8b16-27b953fedcd3/899-1778 ``` | Error in testset Profile: | Test Failed at /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/stdlib/v1.13/Profile/test/runtests.jl:231 | Expression: occursin("@Compiler" * slash, str) | Evaluated: occursin("@Compiler/", "Overhead ╎ [+additional indent] Count File:Line Function\n=========================================================\n ╎9 @juliasrc/task.c:1249 start_task\n ╎ 9 @juliasrc/julia.h:2353 jl_apply\n ╎ 9 @juliasrc/gf.c:3693 ijl_apply_generic\n ╎ 9 @juliasrc/gf.c:3493 _jl_invoke\n ╎ 9 [unknown stackframe]\n ╎ 9 @Distributed/src/process_messages.jl:287 (::Distributed.var\"#handle_msg##2#handle_msg##3\"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()\n ╎ ╎ 9 @Distributed/src/process_messages.jl:70 run_work_thunk(thunk::Distributed.var\"#handle_msg##4#handle_msg##5\"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)\n ╎ ╎ 9 @Distributed/src/process_messages.jl:287 (::Distributed.var\"#handle_msg##4#handle_msg##5\"{Distributed.CallMsg{:call_fetch}})()\n ╎ ╎ 9 @juliasrc/builtins.c:841 jl_f__apply_iterate\n ╎ ╎ 9 @juliasrc/julia.h:2353 jl_apply\n ╎ ╎ 9 @juliasrc/gf.c:3693 ijl_apply_generic\n ╎ ╎ ╎ 9 @juliasrc/gf.c:3493 _jl_invoke\n ╎ ╎ ╎ 9 @Base/Base_compiler.jl:223 kwcall(::@NamedTuple{seed::UInt128}, ::typeof(invokelatest), ::Function, ::String, ::Vararg{String})\n ╎ ╎ ╎ 9 @juliasrc/builtins.c:841 jl_f__apply_iterate\n ╎ ╎ ╎ 9 @juliasrc/julia.h:2353 jl_apply\n ╎ ╎ ╎ 9 @juliasrc/gf.c:3693 ijl_apply_generic\n ╎ ╎ ╎ ╎ 9 @juliasrc/gf.c:3493 _jl_invoke\n ╎ ╎ ╎ ╎ 9 @juliasrc/builtins.c:853 jl_f_invokelatest\n ╎ ╎ ╎ ╎ 9 @juliasrc/julia.h:2353 jl_apply\n ╎ ╎ ╎ ╎ 9 @juliasrc/gf.c:3693 ijl_apply_generic\n ╎ ╎ ╎ ╎ 9 @juliasrc/gf.c:3493 _jl_invoke\n ╎ ╎ ╎ ╎ ╎ 9 [unknown stackframe]\n ╎ ╎ ╎ ╎ ╎ 9 /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/test/testdefs.jl:7 kwcall(::@NamedTuple{seed::UInt128}, ::typeof(runtests), name::String, path::String)\n ╎ ╎ ╎ ╎ ╎ 9 /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/test/testdefs.jl:7 runtests\n ╎ ╎ ╎ ╎ ╎ 9 /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/test/testdefs.jl:13 runtests(name::String, path::String, isolate::Bool; seed::UInt128)\n ╎ ╎ ╎ ╎ ╎ 9 @Base/env.jl:265 withenv(f::var\"#4#5\"{UInt128, String, String, Bool, Bool}, keyvals::Pair{String, Bool})\n ╎ ╎ ╎ ╎ ╎ ╎ 9 /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/test/testdefs.jl:27 (::var\"#4#5\"{UInt128, String, String, Bool, Bool})()\n ╎ ╎ ╎ ╎ ╎ ╎ 9 @Base/timing.jl:621 macro expansion\n ╎ ╎ ╎ ╎ ╎ ╎ 9 /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/test/testdefs.jl:29 macro expansion\n ╎ ╎ ╎ ╎ ╎ ╎ 9 @Test/src/Test.jl:1835 macro expansion\n ╎ ╎ ╎ ╎ ╎ ╎ 9 /cache/build/tester-amdci5-10/julialang/julia-master/julia-7c9af464cc/share/julia/test/testdefs.jl:37 macro expansion\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @Base/Base.jl:303 include(mod::Module, _path::String)\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @Base/loading.jl:2925 _include(mapexpr::Function, mod::Module, _path::String)\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/gf.c:3693 ijl_apply_generic\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/gf.c:3493 _jl_invoke\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @Base/loading.jl:2865 include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @Base/boot.jl:489 eval(m::Module, e::Any)\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/toplevel.c:1095 ijl_toplevel_eval_in\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/toplevel.c:1050 ijl_toplevel_eval\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/toplevel.c:978 jl_toplevel_eval_flex\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/toplevel.c:1038 jl_toplevel_eval_flex\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:897 jl_interpret_toplevel_thunk\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:557 eval_body\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:557 eval_body\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:557 eval_body\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:692 eval_body\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:193 eval_stmt_value\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:242 eval_value\n ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ ╎ 9 @juliasrc/interpreter.c:124 do_call\n ╎ ╎ ╎ ╎ ... ```

@threads

Use an atomic fetch and add to fix a data race in `Module()` identified by tsan: ``` ./usr/bin/julia -t4,0 --gcthreads=1 -e 'Threads.@threads for i=1:100 Module() end' ================== WARNING: ThreadSanitizer: data race (pid=5575) Write of size 4 at 0xffff9bf9bd28 by thread T9: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) mmtk#2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) mmtk#3 <null> <null> (0xffff76a21164) mmtk#4 <null> <null> (0xffff76a1f074) mmtk#5 <null> <null> (0xffff76a1f0c4) mmtk#6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) mmtk#7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) mmtk#8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) mmtk#9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Previous write of size 4 at 0xffff9bf9bd28 by thread T10: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) mmtk#2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) mmtk#3 <null> <null> (0xffff76a21164) mmtk#4 <null> <null> (0xffff76a1f074) mmtk#5 <null> <null> (0xffff76a1f0c4) mmtk#6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) mmtk#7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) mmtk#8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) mmtk#9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Location is global 'jl_new_module__.mcounter' of size 4 at 0xffff9bf9bd28 (libjulia-internal.so.1.13+0x3dbd28) ```

@noinline

…aLang#58893) When invoking any "functor-like", such as a closure: ```julia bar(x) = @noinline ((y)->x+y)(x) ``` our IR printing was not showing the arg0 invoked, even when it is required to determine which MethodInstance this is invoking. Before: ```julia julia> @code_typed optimize=true bar(1) CodeInfo( 1 ─ %1 = %new(var"#bar#mmtk#2#bar#mmtk#3"{Int64}, x)::var"#bar#mmtk#2#bar#mmtk#3"{Int64} │ %2 = invoke %1(x::Int64)::Int64 └── return %2 ) => Int64 ``` After: ```julia julia> @code_typed optimize=true bar(1) CodeInfo( 1 ─ %1 = %new(var"#bar#mmtk#2#bar#mmtk#3"{Int64}, x)::var"#bar#mmtk#2#bar#mmtk#3"{Int64} │ %2 = invoke (%1::var"#bar#mmtk#2#bar#mmtk#3"{Int64})(x::Int64)::Int64 └── return %2 ) => Int64 ```

Simplify `workqueue_for`. While not strictly necessary, the acquire load in `getindex(once::OncePerThread{T,F}, tid::Integer)` makes ThreadSanitizer happy. With the existing implementation, we get false positives whenever a thread other than the one that originally allocated the array reads it: ``` ================== WARNING: ThreadSanitizer: data race (pid=6819) Atomic read of size 8 at 0xffff86bec058 by main thread: #0 getproperty Base_compiler.jl:57 (sys.so+0x113b478) #1 julia_pushNOT._1925 task.jl:868 (sys.so+0x113b478) mmtk#2 julia_enq_work_1896 task.jl:969 (sys.so+0x5cd218) mmtk#3 schedule task.jl:983 (sys.so+0x892294) mmtk#4 macro expansion threadingconstructs.jl:522 (sys.so+0x892294) mmtk#5 julia_start_profile_listener_60681 Base.jl:355 (sys.so+0x892294) mmtk#6 julia___init___60641 Base.jl:392 (sys.so+0x1178dc) mmtk#7 jfptr___init___60642 <null> (sys.so+0x118134) mmtk#8 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4) mmtk#9 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4) mmtk#10 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0xbba74) mmtk#11 jl_module_run_initializer /home/user/c/julia/src/toplevel.c:68:13 (libjulia-internal.so.1.13+0xbba74) mmtk#12 _finish_jl_init_ /home/user/c/julia/src/init.c:632:13 (libjulia-internal.so.1.13+0x9c0fc) mmtk#13 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4) mmtk#14 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8) mmtk#15 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934) mmtk#16 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20) Previous write of size 8 at 0xffff86bec058 by thread T2: #0 IntrusiveLinkedListSynchronized task.jl:863 (sys.so+0x78d220) #1 macro expansion task.jl:932 (sys.so+0x78d220) mmtk#2 macro expansion lock.jl:376 (sys.so+0x78d220) mmtk#3 julia_workqueue_for_1933 task.jl:924 (sys.so+0x78d220) mmtk#4 julia_wait_2048 task.jl:1204 (sys.so+0x6255ac) mmtk#5 julia_task_done_hook_49205 task.jl:839 (sys.so+0x128fdc0) mmtk#6 jfptr_task_done_hook_49206 <null> (sys.so+0x902218) mmtk#7 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4) mmtk#8 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4) mmtk#9 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9c79c) mmtk#10 jl_finish_task /home/user/c/julia/src/task.c:345:13 (libjulia-internal.so.1.13+0x9c79c) mmtk#11 jl_threadfun /home/user/c/julia/src/scheduler.c:122:5 (libjulia-internal.so.1.13+0xe7db8) Thread T2 (tid=6824, running) created by main thread at: #0 pthread_create <null> (julia+0x85f88) #1 uv_thread_create_ex /workspace/srcdir/libuv/src/unix/thread.c:172 (libjulia-internal.so.1.13+0x1a8d70) mmtk#2 _finish_jl_init_ /home/user/c/julia/src/init.c:618:5 (libjulia-internal.so.1.13+0x9c010) mmtk#3 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4) mmtk#4 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8) mmtk#5 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934) mmtk#6 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20) SUMMARY: ThreadSanitizer: data race Base_compiler.jl:57 in getproperty ================== ```

* Increment state conditionally in `CartesianIndices` iteration (#58742) Fixes https://github.com/JuliaLang/julia/issues/53430 ```julia julia> a = rand(100,100); b = similar(a); av = view(a, axes(a)...); bv = view(b, axes(b)...); bv2 = view(b, UnitRange.(axes(b))...); julia> @btime copyto!($bv2, $av); # slow, indices are UnitRanges 12.352 μs (0 allocations: 0 bytes) # master, v"1.13.0-DEV.745" 1.662 μs (0 allocations: 0 bytes) # this PR julia> @btime copyto!($bv, $av); # reference 1.733 μs (0 allocations: 0 bytes) ``` The performances become comparable after this PR. I've also renamed the second `I` to `Itail`, as the two variables represent different quantities. * 🤖 [master] Bump the Distributed stdlib from 51e5297 to 3679026 (#58748) Stdlib: Distributed URL: https://github.com/JuliaLang/Distributed.jl Stdlib branch: master Julia branch: master Old commit: 51e5297 New commit: 3679026 Julia version: 1.13.0-DEV Distributed version: 1.11.0(Does not match) Bump invoked by: @DilumAluthge Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: https://github.com/JuliaLang/Distributed.jl/compare/51e52978481835413d15b589919aba80dd85f890...3679026d7b510befdedfa8c6497e3cb032f9cea1 ``` $ git log --oneline 51e5297..3679026 3679026 Merge pull request #137 from JuliaLang/dpa/dont-use-link-local 875cd5a Rewrite the code to be a bit more explicit 2a6ee53 Non-link-local IP4 > non-link-local IP6 > link-local IP4 > link-local IP6 c0e9eb4 Factor functionality out into separate `choose_bind_addr()` function 86cbb8a Add explanation 0b7288c Worker: Bind to the first non-link-local IPv4 address ff8689a Merge pull request #131 from JuliaLang/spawnat-docs ba3c843 Document that `@spawnat :any` doesn't do load-balancing ``` Co-authored-by: DilumAluthge <[email protected]> * devdocs: contributing: fix headings (#58749) In particular, it seems like Documenter takes the level-one heading to define the page title. So the page titles were missing in the TOC before this change. * Work around LLVM JITLink stack overflow issue. (#58579) The JITLinker recurses for every symbol in the list so limit the size of the list This is kind of ugly. Also 1000 might be too large, we don't want to go too small because that wastes memory and 1000 was fine locally for the things I tested. Fixes https://github.com/JuliaLang/julia/issues/58229 * bump Compiler.jl version to 0.1.1 (#58744) As the latest version of BaseCompiler.jl will be bumped to v0.1.1 after JuliaRegistries/General#132990. * REPL: fix typo and potential `UndefVarError` (#58761) Detected by the new LS diagnostics:) * fix fallback code path in `take!(::IOBuffer)` method (#58762) JET told me that the `data` local variable was inparticular is undefined at this point. After reviewing this code, I think this code path is unreachable actually since `bytesavailable(io::IOBuffer)` returns `0` when `io` has been closed. So it's probably better to make it clear. * Fix multi-threading docs typo (#58770) * help bounds checking to be eliminated for `getindex(::Memory, ::Int)` (#58754) Second try for PR #58741. This moves the `getindex(::Memory, ::Int)` bounds check to Julia, which is how it's already done for `getindex(::Array, ::Int)`, so I guess it's correct. Also deduplicate the bounds checking code while at it. * Define textwidth for overlong chars (#58602) Previously, this would error. There is no guarantee of how terminals render overlong encodings. Some terminals does not print them at all, and some print "�". Here, we set a textwidth of 1, conservatively. Refs #58593 * Add MethodError hints for functions in other modules (#58715) When a MethodError occurs, check if functions with the same name exist in other modules (particularly those of the argument types). This helps users discover that they may need to import a function or ensure multiple functions are the same generic function. - For Base functions: suggests importing (e.g., "You may have intended to import Base.length") - For other modules: suggests they may be intended as the same generic function - Shows all matches from relevant modules in sorted order - Uses modulesof! to properly handle all type structures including unions Fixes #58682 * Fix markdown bullet list in variables-and-scoping.md (#58771) * CONTRIBUTING.md: Ask folks to disclose AI-written PRs (#58666) * Convert julia-repl blocks to jldoctest format (#58594) Convert appropriate julia-repl code blocks to jldoctest format to enable automatic testing. In addition, this introduces a new `nodoctest = "reason"` pattern to annotate code blocks that are deliberate not doctested, so future readers will know not to try. Many code blocks are converted, in particular: - Manual pages: arrays.md, asynchronous-programming.md, functions.md, integers-and-floating-point-numbers.md, metaprogramming.md, multi-threading.md, performance-tips.md, variables.md, variables-and-scoping.md - Base documentation: abstractarray.jl, bitarray.jl, expr.jl, file.jl, float.jl, iddict.jl, path.jl, scopedvalues.md, sort.md - Standard library: Dates/conversions.jl, Random/RNGs.jl, Sockets/addrinfo.jl Key changes: - Add filters for non-deterministic output (timing, paths, memory addresses) - Add setup/teardown for filesystem operations - Fix parentmodule(M) usage in expr.jl for doctest compatibility - Document double escaping requirement for regex filters in docstrings - Update AGENTS.md with test running instructions Note: Some julia-repl blocks were intentionally left unchanged when they demonstrate language internals subject to change or contain non-deterministic output that cannot be properly filtered. Refs #56921 --------- Co-authored-by: Keno Fischer <[email protected]> Co-authored-by: Claude <[email protected]> * adds the `nth` function for iterables (#56580) Hi, I've turned the open ended issue #54454 into an actual PR. Tangentially related to #10092 ? This PR introduces the `nth(itr, n)` function to iterators to give a `getindex` type of behaviour. I've tried my best to optimize as much as possible by specializing on different types of iterators. In the spirit of iterators any OOB access returns `nothing`. (edit: instead of throwing an error, i.e. `first(itr, n)` and `last(itr, n)`) here is the comparison of running the testsuite (~22 different iterators) using generic `nth` and specialized `nth`: ```julia @btime begin for (itr, n, _) in $testset _fallback_nth(itr, n) end end 117.750 μs (366 allocations: 17.88 KiB) @btime begin for (itr, n, _) in $testset nth(itr, n) end end 24.250 μs (341 allocations: 16.70 KiB) ``` --------- Co-authored-by: adienes <[email protected]> Co-authored-by: Steven G. Johnson <[email protected]> Co-authored-by: Dilum Aluthge <[email protected]> * refine IR model queries (#58661) - `jl_isa_ast_node` was missing `enter`/`leave` nodes. - `Core.IR` exports mistakenly included a function `memoryref`. - `Base.IR`, and `quoted` were not public or documented. - Add julia function `isa_ast_node` to improve accuracy of `quoted`. - Change `==` on AST nodes to check egal equality of any constants in the IR / AST, and make hashing consistent with that change. This helpfully allows determining that `x + 1` and `x + 1.0` are not equivalent, exchangeable operations. If you need to compare any two objects for semantic equality, you may need to first wrap them with `x = Base.isa_ast_node(x) ? x : QuoteNode(x)` to resolve the ambiguity of whether the comparison is of the semantics or value. - Handle `undef` fields in Phi/PhiC node equality and hashing * fix showing types after removing using Core (#58773) PR #57357 changed the default using list, but only changed some of the places where the `show` code handled that. This led to duplicate (confusing) printing, since both Core. and Base. prefixes are dropped. Fix #58772 * inform compiler about local variable definedness (#58778) JET's new analysis pass now detects local variables that may be undefined, which has revealed such issues in several functions within Base (JuliaLang/julia#58762). This commit addresses local variables whose definedness the compiler cannot properly determine, primarily in functions reachable from JET's test suite. No functional changes are made. * better effects for `iterate` for `Memory` and `Array` (#58755) * Test: Hide REPL internals in backtraces (#58732) * Update docs for various type predicates (#58774) Makes the description for `isdispatchtuple` accurate, adds a docstring for `iskindtype` and `isconcretedispatch`, and adds notes to the docs for `isconcretetype` and `isabstracttype` explaining why they aren't antonyms. * Test: show context when a let testset errors (#58727) * [libblastrampoline_jll] Upgrade to v5.13.1 (#58775) ### Check list Version numbers: - [x] `deps/libblastrampoline.version`: `LIBNAME_VER`, `LIBNAME_BRANCH`, `LIBNAME_SHA1` and `LIBNAME_JLL_VER` - [x] `stdlib/libblastrampoline_jll/Project.toml`: `version` Checksum: - [x] `deps/checksums/libblastrampoline` * 🤖 [master] Bump the Pkg stdlib from 5577f68d6 to e3d456127 (#58781) Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: 5577f68d6 New commit: e3d456127 Julia version: 1.13.0-DEV Pkg version: 1.13.0 Bump invoked by: @KristofferC Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: https://github.com/JuliaLang/Pkg.jl/compare/5577f68d612139693282c037d070f515bf160d1b...e3d4561272fc029e9a5f940fe101ba4570fa875d ``` $ git log --oneline 5577f68d6..e3d456127 e3d456127 add update function to apps and fix a bug when adding an already installed app (#4263) cae9ce02a Fix historical stdlib fixup if `Pkg` is in the Manifest (#4264) a42046240 don't use tree hash from manifest if the path is set from sources (#4260) a94a6bcae fix dev taking when the app is already installed (#4259) 313fddccb Internals: Add fallback `Base.show(::IO, ::RegistryInstance)` method (#4251) ``` Co-authored-by: KristofferC <[email protected]> * prevent unnecessary repeated squaring calculation (#58720) * LibGit2: Update to 1.9.1 (#58731) * Unify `_checkbounds_array` into `checkbounds` and use it in more places (#58785) Ref: https://github.com/JuliaLang/julia/pull/58755#discussion_r2158944282. --------- Co-authored-by: Matt Bauman <[email protected]> Co-authored-by: Matt Bauman <[email protected]> * Chained hash pipelining in array hashing (#58252) the proposed switch in https://github.com/JuliaLang/julia/pull/57509 from `3h - hash_finalizer(x)` to `hash_finalizer(3h -x)` should increase the hash quality of chained hashes, as the expanded expression goes from something like `sum((-3)^k * hash(x) for k in ...)` to a non-simplifiable composition this does have the unfortunate impact of long chains of hashes getting a bit slower as there is more data dependency and the CPU cannot work on the next element's hash before combining the previous one (I think --- I'm not particularly an expert on this low level stuff). As far as I know this only really impacts `AbstractArray` so, I've implemented a proposal that does some unrolling / pipelining manually to recover `AbstractArray` hashing performance. in fact, it's quite a lot faster now for most lengths. I tuned the thresholds (8 accumulators, certain length breakpoints) by hand on my own machine. * Require all tuples in eachindex to have the same length. (#48125) Potential fix for #47898 --------- Co-authored-by: navdeep rana <[email protected]> Co-authored-by: Oscar Smith <[email protected]> Co-authored-by: Jerry Ling <[email protected]> Co-authored-by: Andy Dienes <[email protected]> * trailing dimensions in eachslice (#58791) fixes https://github.com/JuliaLang/julia/issues/51692 * Allow underscore (unused) args in presence of kwargs (#58803) Admittedly fixed because I thought I introduced this bug recently, but actually, fix #32727. `f(_; kw) = 1` should now lower in a similar way to `f(::Any; kw) = 1`, where we use a gensym for the first argument. Not in this PR, but TODO: `nospecialize` underscore-only names * codegen: slightly optimize gc-frame allocation (#58794) Try to avoid allocating frames for some very simple function that only have the safepoint on entry and don't define any values themselves. * codegen: ensure safepoint functions can read the pgcstack (#58804) This needs to be readOnly over all memory, since GC could read anything (especially pgcstack), and which is not just argmem:read, but also the pointer accessed from argmem that is read from. Fix #58801 Note that this is thought not to be a problem for CleanupWriteBarriers, since while that does read the previously-inaccessibleMemOnly state, these functions are not marked nosync, so as long as the global state can be read, it also must be assumed that it might observe another thread has written to any global state. * Revert code changes from "strengthen assume_effects doc" PR (#58289) Reverts only the functional changes from JuliaLang/julia#58254, not the docs. Accessing this field here assumes that the counter valid is numeric and relevant to the current inference frame, neither of which is intended to be true, as we continue to add interfaces to execute methods outside of their current specific implementation with a monotonic world counter (e.g. with invoke on a Method, with precompile files, with external MethodTables, or with static compilation). * build: Error when attempting to set USECLANG/USEGCC (#58795) Way back in the good old days, these used to switch between GCC and Clang. I guess these days we always auto-switch based on the CC value. If you try to directly set USECLANG, things get into a bad state. Give a better error message for that case. * build: Add --no-same-owner to TAR (#58796) tar changes behavior when the current uid is 0 to try to also restore owner uids/gids (if recorded). It is possible for the uid to be 0 in single-uid environments like user namespace sandboxes, in which case the attempt to change the uid/gid fails. Of course ideally, the tars would have been created non-archival (so that the uid/gid wasn't recorded in the first place), but we get source tars from various places, so we can't guarantee this. To make sure we don't run into trouble, manually add the --no-same-owner flag to disable this behavior. * Add `cfunction` support for `--trim` (#58812) * fix error message for `eachindex(::Vararg{Tuple})` (#58811) Make the error message in case of mismatch less confusing and consistent with the error message for arrays. While at it, also made other changes of the same line of source code: * use function composition instead of an anonymous closure * expand the one-liner into a multiline `if` --------- Co-authored-by: Andy Dienes <[email protected]> * use more canonical way to check binding existence (#58809) * Add `trim_mode` parameter to JIT type-inference entrypoint (#58817) Resolves https://github.com/JuliaLang/julia/issues/58786. I think this is only a partial fix, since we can still end up loading code from pkgimages that has been poorly inferred due to running without these `InferenceParams`. However, many of the common scenarios (such as JLL's depending on each other) seem to be OK since we have a targeted heuristic that adds `__init__()` to a pkgimage only if the module has inference enabled. * codegen: gc wb for atomic FCA stores (#58792) Need to re-load the correct `r` since issetfield skips the intcast, resulting in no gc wb for the FCA. Fix #58760 * codegen: relaxed jl_tls_states_t.safepoint load (#58828) Every function with a safepoint causes spurious thread sanitizer warnings without this change. Codegen is unaffected, except when we build with `ThreadSanitizerPass`. * bpart: Properly track methods with invalidated source after require_world (#58830) There are three categories of methods we need to worry about during staticdata validation: 1. New methods added to existing generic functions 2. New methods added to new generic functions 3. Existing methods that now have new CodeInstances In each of these cases, we need to check whether any of the implicit binding edges from the method's source was invalidated. Currently, we handle this for 1 and 2 by explicitly scanning the method on load. However, we were not tracking it for case 3. Fix that by using an extra bit in did_scan_method that gets set when we see an existing method getting invalidated, so we know that we need to drop the corresponding CodeInstances during load. Fixes #58346 * Limit --help and --help-hidden to 100 character line length (#58835) Just fixing the command line description to make sure it is not more than 100 characters wide as discussed with @oscardssmith in PR #54066 and PR #53759. I also added a test to make sure that nothing more than 100 characters is inserted. Thank you. * libuv: Mark `(un)preserve_handle` as `@nospecialize` (#58844) These functions only worry about object identity, so there's no need for them to specialize them on their type. * add METHOD_SIG_LATEST_ONLY optimization to MethodInstance too (#58825) Add the same optimization from Method to MethodInstance, although the performance gain seems to be negligible in my specific testing, there doesn't seem any likely downside to adding one caching bit to avoid some recomputations. * Encode fully_covers=false edges using negative of method count This change allows edges that don't fully cover their method matches to be properly tracked through serialization. When fully_covers is false (indicating incomplete method coverage), we encode the method count as negative in the edges array to signal that compactly. * move trim patches to separate files, only load if trimming (#58826) fixes part of #58458 * gf: Add METHOD_SIG_LATEST_HAS_NOTMORESPECIFIC dispatch status bit This commit introduces a new dispatch status bit to track when a method has other methods that are not more specific than it, enabling better optimization decisions during method dispatch. Key changes: 1. Add METHOD_SIG_LATEST_HAS_NOTMORESPECIFIC bit to track methods with non-morespecific intersections 2. Add corresponding METHOD_SIG_PRECOMPILE_HAS_NOTMORESPECIFIC bit for precompiled methods 3. Refactor method insertion logic: - Remove morespec_unknown enum state, compute all morespec values upfront - Convert enum morespec_options to simple boolean logic (1/0) - Change 'only' from boolean to 'dispatch_bits' bitmask - Move dispatch status updates before early continues in the loop * optimize verify_call again * juliac: Add rudimentary Windows support (#57481) This was essentially working as-is, except for our reliance on a C compiler. Not sure how we feel about having an `Artifacts.toml` floating around our `contrib` folder, but I'm not aware of an alternative other than moving `juliac.jl` to a subdirectory. * fix null comparisons for non-standard address spaces (#58837) Co-authored-by: Jameson Nash <[email protected]> * debuginfo: Memoize object symbol lookup (#58851) Supersedes https://github.com/JuliaLang/julia/pull/58355. Resolves https://github.com/JuliaLang/julia/issues/58326. On this PR: ```julia julia> @btime lgamma(2.0) ┌ Warning: `lgamma(x::Real)` is deprecated, use `(logabsgamma(x))[1]` instead. │ caller = var"##core#283"() at execution.jl:598 └ @ Core ~/.julia/packages/BenchmarkTools/1i1mY/src/execution.jl:598 47.730 μs (105 allocations: 13.24 KiB) ``` On `nightly`: ```julia julia> @btime lgamma(2.0) ┌ Warning: `lgamma(x::Real)` is deprecated, use `(logabsgamma(x))[1]` instead. │ caller = var"##core#283"() at execution.jl:598 └ @ Core ~/.julia/packages/BenchmarkTools/1i1mY/src/execution.jl:598 26.856 ms (89 allocations: 11.32 KiB) ``` * bpart: Skip inserting image backedges while we're generating a pkgimage (#58843) Should speed up deeply nested precompiles by skipping unnecessary work here. PR is against #58830 to avoid conflicts, but semantically independent. * Re-add old function name for backward compatibility in init (#58860) While julia has no C-API backwards compatibility guarantees this is simple enough to add. Fixes #58859 * trimming: Add `_uv_hook_close` support (#58871) Resolves https://github.com/JuliaLang/julia/issues/58862. Since this hook is called internally by the runtime, `--trim` was not aware of the callee edge required here. * Don't `@inbounds` AbstractArray's iterate method; optimize `checkbounds` instead (#58793) Split off from #58785, this simplifies `iterate` and removes the `@inbounds` call that was added in https://github.com/JuliaLang/julia/pull/58635. It achieves the same (or better!) performance, however, by targeting optimizations in `checkbounds` and — in particular — the construction of a linear `eachindex` (against which the bounds are checked). --------- Co-authored-by: Mosè Giordano <[email protected]> * aotcompile: Fix early-exit if CI not found for `cfunction` (#58722) As written, this was accidentally skipping all the subsequent `cfuncs` that need adapters. * zero-index get/setindex(::ReinterpretArray) require a length of 1 (#58814) fix https://github.com/JuliaLang/julia/issues/58232 o3 helped me understand the existing implementations but code is mine --------- Co-authored-by: Matt Bauman <[email protected]> * Add `Base.isprecompilable` (#58805) Alternative to https://github.com/JuliaLang/julia/pull/58146. We want to compile a subset of the possible specializations of a function. To this end, we have a number of manually written `precompile` statements. Creating this list is, unfortunately, error-prone, and the list is also liable to going stale. Thus we'd like to validate each `precompile` statement in the list. The simple answer is, of course, to actually run the `precompile`s, and we naturally do so, but this takes time. We would like a relatively quick way to check the validity of a `precompile` statement. This is a dev-loop optimization, to allow us to check "is-precompilable" in unit tests. We can't use `hasmethod` as it has both false positives (too loose): ```julia julia> hasmethod(sum, (AbstractVector,)) true julia> precompile(sum, (AbstractVector,)) false julia> Base.isprecompilable(sum, (AbstractVector,)) # <- this PR false ``` and also false negatives (too strict): ```julia julia> bar(@nospecialize(x::AbstractVector{Int})) = 42 bar (generic function with 1 method) julia> hasmethod(bar, (AbstractVector,)) false julia> precompile(bar, (AbstractVector,)) true julia> Base.isprecompilable(bar, (AbstractVector,)) # <- this PR true ``` We can't use `hasmethod && isconcretetype` as it has false negatives (too strict): ```julia julia> has_concrete_method(f, argtypes) = all(isconcretetype, argtypes) && hasmethod(f, argtypes) has_concrete_method (generic function with 1 method) julia> has_concrete_method(bar, (AbstractVector,)) false julia> has_concrete_method(convert, (Type{Int}, Int32)) false julia> precompile(convert, (Type{Int}, Int32)) true julia> Base.isprecompilable(convert, (Type{Int}, Int32)) # <- this PR true ``` `Base.isprecompilable` is essentially `precompile` without the actual compilation. * Add a `similar` method for `Type{<:CodeUnits}` (#57826) Currently, `similar(::CodeUnits)` works as expected by going through the generic `AbstractArray` method. However, the fallback method hit by `similar(::Type{<:CodeUnits}, dims)` does not work, as it assumes the existence of a constructor that accepts an `UndefInitializer`. This can be made to work by defining a corresponding `similar` method that returns an `Array`. One could make a case that this is a bugfix since it was arguably a bug that this method didn't work given that `CodeUnits` is an `AbstractArray` subtype and the other `similar` methods work. If anybody buys that argument, it could be nice to backport this; it came up in some internal code that uses Arrow.jl and JSON3.jl together. * 🤖 [master] Bump the Pkg stdlib from e3d456127 to 109eaea66 (#58858) Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: master Julia branch: master Old commit: e3d456127 New commit: 109eaea66 Julia version: 1.13.0-DEV Pkg version: 1.13.0 Bump invoked by: @KristofferC Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: https://github.com/JuliaLang/Pkg.jl/compare/e3d4561272fc029e9a5f940fe101ba4570fa875d...109eaea66a0adb0ad8fa497e64913eadc2248ad1 ``` $ git log --oneline e3d456127..109eaea66 109eaea66 Various app improvements (#4278) 25c2390ed feat(apps): Add support for multiple apps per package via submodules (#4277) c78b40b35 copy the app project instead of wrapping it (#4276) d2e61025b Fix leading whitespace in REPL commands with comma-separated packages (#4274) e02bcabd7 Registry: Properly pass down `depot` (#4268) e9a055240 fix what project file to look at when package without path but with a subdir is devved by name (#4271) 8b1f0b9ff prompt for confirmation before removing compat entry (#4254) eefbef649 feat(errors): Improve error message for incorrect package UUID (#4270) 4d1c6b0a3 explain no reg installed when no reg installed (#4261) ``` Co-authored-by: KristofferC <[email protected]> * fix a few tiny JET linter issues (#58869) * Fix data race in jl_new_module__ (#58880) Use an atomic fetch and add to fix a data race in `Module()` identified by tsan: ``` ./usr/bin/julia -t4,0 --gcthreads=1 -e 'Threads.@threads for i=1:100 Module() end' ================== WARNING: ThreadSanitizer: data race (pid=5575) Write of size 4 at 0xffff9bf9bd28 by thread T9: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Previous write of size 4 at 0xffff9bf9bd28 by thread T10: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Location is global 'jl_new_module__.mcounter' of size 4 at 0xffff9bf9bd28 (libjulia-internal.so.1.13+0x3dbd28) ``` * fix trailing indices stackoverflow in reinterpreted array (#58293) would fix https://github.com/JuliaLang/julia/issues/57170, fix https://github.com/JuliaLang/julia/issues/54623 @nanosoldier `runbenchmarks("array", vs=":master")` * Add missing module qualifier (#58877) A very simple fix addressing the following bug: ```julia Validation: Error During Test at REPL[61]:1 Got exception outside of a @test #=ERROR showing exception stack=# UndefVarError: `get_ci_mi` not defined in `Base.StackTraces` Suggestion: check for spelling errors or missing imports. Hint: a global variable of this name also exists in Base. - Also declared public in Compiler (loaded but not imported in Main). Stacktrace: [1] show_custom_spec_sig(io::IOContext{IOBuffer}, owner::Any, linfo::Core.CodeInstance, frame::Base.StackTraces.StackFrame) @ Base.StackTraces ./stacktraces.jl:293 [2] show_spec_linfo(io::IOContext{IOBuffer}, frame::Base.StackTraces.StackFrame) @ Base.StackTraces ./stacktraces.jl:278 [3] print_stackframe(io::IOContext{IOBuffer}, i::Int64, frame::Base.StackTraces.StackFrame, n::Int64, ndigits_max::Int64, modulecolor::Symbol; prefix::Nothing) @ Base ./errorshow.jl:786 ``` AFAIK this occurs when printing a stacktrace from a `CodeInstance` that has a non-default owner. * OpenSSL: Update to 3.5.1 (#58876) Update the stdlib OpenSSL to 3.5.1. This is a candidate for backporting to Julia 1.12 if there is another beta release. * `setindex!(::ReinterpretArray, v)` needs to convert before reinterpreting (#58867) Found in https://github.com/JuliaLang/julia/pull/58814#discussion_r2169155093. Previously, in a very limited situation (a zero-dimensional reinterpret that reinterprets between primitive types that was setindex!'ed with zero indices), we omitted the `convert`. I believe this was an unintentional oversight, and hopefully nobody is depending on this behavior. * Support `debuginfo` context option in IRShow for `IRCode`/`IncrementalCompact` (#58642) This allows us to get complete source information during printing for `IRCode` and `IncrementalCompact`, same as we do by default with `CodeInfo`. The user previously had to do: ```julia Compiler.IRShow.show_ir(stdout, ir, Compiler.IRShow.default_config(ir; verbose_linetable=true)) ``` and now, they only need to do: ```julia show(IOContext(stdout, :debuginfo => :source), ir) ``` * Add offset in `hvncat` dimension calculation to fix issue with 0-length elements in first dimension (#58881) * fix `setindex(::ReinterpretArray,...)` for zero-d arrays (#58868) by copying the way getindex works. Found in https://github.com/JuliaLang/julia/pull/58814#discussion_r2178243259 --------- Co-authored-by: Andy Dienes <[email protected]> * add back `to_power_type` to `deprecated.jl` since some packages call it (#58886) Co-authored-by: KristofferC <[email protected]> * Pkg: Allow configuring can_fancyprint(io::IO) using IOContext (#58887) * Make `Base.donotdelete` public (#55774) I rely on `Base.donotdelete` in [Chairmarks.jl](https://chairmarks.lilithhafner.com) and I'd like it to be public. I imagine that other benchmarking tools also rely on it. It's been around since 1.8 (see also: #55773) and I think we should commit to keeping it functional for the rest of 1.x. * Add link to video in profiling manual (#58896) * Stop documenting that `permute!` is "in-place"; it isn't and never has been non-allocating (#58902) * faster iteration over a `Flatten` of heterogenous iterators (#58522) seems to help in many cases. would fix the precise MWE given in https://github.com/JuliaLang/julia/issues/52552, but does not necessarily fix comprehensively all perf issues of all heterogenous flattens. but, may as well be better when it's possible setup: ``` julia> using BenchmarkTools julia> A = rand(Int, 100000); B = 1:100000; julia> function g(it) s = 0 for i in it s += i end s end ``` before: ``` julia> @btime g($(Iterators.flatten((A, B)))) 12.461 ms (698979 allocations: 18.29 MiB) julia> @btime g($(Iterators.flatten(i for i in (A, B)))) 12.393 ms (698979 allocations: 18.29 MiB) julia> @btime g($(Iterators.flatten([A, B]))) 15.115 ms (999494 allocations: 25.93 MiB) julia> @btime g($(Iterators.flatten((A, Iterators.flatten((A, B)))))) 82.585 ms (2997964 allocations: 106.78 MiB) ``` after: ``` julia> @btime g($(Iterators.flatten((A, B)))) 135.958 μs (2 allocations: 64 bytes) julia> @btime g($(Iterators.flatten(i for i in (A, B)))) 149.500 μs (2 allocations: 64 bytes) julia> @btime g($(Iterators.flatten([A, B]))) 17.130 ms (999498 allocations: 25.93 MiB) julia> @btime g($(Iterators.flatten((A, Iterators.flatten((A, B)))))) 13.716 ms (398983 allocations: 10.67 MiB) ``` * Make `hypot` docs example more type stable (#58918) * Markdown: Make `Table`/`LaTeX` objects subtypes of `MarkdownElement` (#58916) These objects satisfy the requirements of the `MarkdownElement` interface (such as implementing `Markdown.plain`), so they should be subtypes of `MarkdownElement`. This is convenient when defining functions for `MarkdownElement` in other packages. * Support "Functor-like" `code_typed` invocation (#57911) This lets you easily inspect IR associated with "Functor-like" methods: ```julia julia> (f::Foo)(offset::Float64) = f.x + f.y + offset julia> code_typed((Foo, Float64)) 1-element Vector{Any}: CodeInfo( 1 ─ %1 = Base.getfield(f, :x)::Int64 │ %2 = Base.getfield(f, :y)::Int64 │ %3 = Base.add_int(%1, %2)::Int64 │ %4 = Base.sitofp(Float64, %3)::Float64 │ %5 = Base.add_float(%4, offset)::Float64 └── return %5 ) => Float64 ``` This is just a small convenience over `code_typed_by_type`, but I'm in support of it (even though it technically changes the meaning of, e.g., `code_typed((1, 2))` which without this PR inspects `(::Tuple{Int,Int})(::Vararg{Any})` We should probably update all of our reflection machinery (`code_llvm`, `code_lowered`, `methodinstance`, etc.) to support this "non-arg0" style as well, but I wanted to open this first to make sure folks like it. * IRShow: Print arg0 type when necessary to disambiguate `invoke` (#58893) When invoking any "functor-like", such as a closure: ```julia bar(x) = @noinline ((y)->x+y)(x) ``` our IR printing was not showing the arg0 invoked, even when it is required to determine which MethodInstance this is invoking. Before: ```julia julia> @code_typed optimize=true bar(1) CodeInfo( 1 ─ %1 = %new(var"#bar##2#bar##3"{Int64}, x)::var"#bar##2#bar##3"{Int64} │ %2 = invoke %1(x::Int64)::Int64 └── return %2 ) => Int64 ``` After: ```julia julia> @code_typed optimize=true bar(1) CodeInfo( 1 ─ %1 = %new(var"#bar##2#bar##3"{Int64}, x)::var"#bar##2#bar##3"{Int64} │ %2 = invoke (%1::var"#bar##2#bar##3"{Int64})(x::Int64)::Int64 └── return %2 ) => Int64 ``` * Support "functors" for code reflection utilities (#58891) As a follow-up to https://github.com/JuliaLang/julia/pull/57911, this updates: - `Base.method_instance` - `Base.method_instances` - `Base.code_ircode` - `Base.code_lowered` - `InteractiveUtils.code_llvm` - `InteractiveUtils.code_native` - `InteractiveUtils.code_warntype` to support "functor" invocations. e.g. `code_llvm((Foo, Int, Int))` which corresponds to `(::Foo)(::Int, ::Int)` * Prevent data races in invalidate_code_for_globalref! * Fix type instability in invalidate_code_for_globalref! * Add the fact that functions ending with `!` may allocate to the FAQ (#58904) I've run into this question several times, that might count as "frequently asked". * Economy mode REPL: run the event loop with jl_uv_flush (#58926) `ios_flush` won't wait for the `jl_static_show` from the previous evaluation to complete, resulting in the output being interleaved with subsequent REPL outputs. Anything that produces a lot of output will trigger it, like `Core.GlobalMethods.defs`. * Fix grammar, typos, and formatting issues in docstrings (#58944) Co-authored-by: Claude <[email protected]> * Fix nthreadpools size in JLOptions (#58937) * NFC: Remove duplicate `julia-src-%` dependency in makefile (#58947) * Improve error message for missing dependencies in packages (#58878) * Make current_terminfo a OncePerProcess (#58854) There seems to be no reason to always load this unconditionally - especially since it's in the critical startup path. If we never print colored output or our IO is not a TTY, we don't need to load this at all. While we're at it, remove the `term_type` argument to `ttyhascolor`, which didn't work as advertised anyway, since it still looked at the current_terminfo. If clients want to do a full TermInfo check, they can do that explicitly. (Written by Claude Code) * chore: remove redundant words in comment (#58955) * add a precompile workload to TOML (#58949) * 🤖 [master] Bump the NetworkOptions stdlib from c090626 to 532992f (#58882) Co-authored-by: DilumAluthge <[email protected]> * remove excessive code from trim script (#58853) Co-authored-by: gbaraldi <[email protected]> * Add juliac Artifacts.toml in Makefile (#58936) * staticdata: Don't discard inlineable code that inference may need (#58842) See https://github.com/JuliaLang/julia/issues/58841#issuecomment-3014833096. We were accidentally discarding inferred code during staticdata preparation that we would need immediately afterwards to satisfy inlining requests during code generation for the system image. This was resulting in spurious extra compilation at the first inference after sysimage reload. Additionally it was likely causing various unnecessary dispatch slow paths in the generated inference code. Fixes #58841. * clear up `isdone` docstring (#58958) I got pretty confused on my first reading of this docstring because for some reason I thought it was saying that `isdone(itr, state) == missing` implied that it was true that `iterate(itr, state) === nothing` (aka that `state` is indeed final). which of course is wrong and doesn't make sense, but it's still how I read it. I think the new docstring is a bit more explicit. * shield `_artifact_str` function behind a world age barrier (#58957) We already do this for `require` in Base loading, it probably makes sense to do this here as well, as invalidating this function easily adds +1s in load time for a jll. Avoids the big load time penalty from loading IntelOpenMP_jll in https://github.com/JuliaLang/julia/issues/57436#issuecomment-3052258775. Before: ``` julia> @time using ModelingToolkit 6.546844 seconds (16.09 M allocations: 938.530 MiB, 11.13% gc time, 16.35% compilation time: 12% of which was recompilation) ``` After: ``` julia> @time using ModelingToolkit 5.637914 seconds (8.26 M allocations: 533.694 MiB, 11.47% gc time, 3.11% compilation time: 17% of which was recompilation) ``` --------- Co-authored-by: KristofferC <[email protected]> Co-authored-by: Cody Tapscott <[email protected]> * doc: Fix grammar, typos, and formatting issues across documentation (#58932) Co-authored-by: Claude <[email protected]> * Replace Base.Workqueues with a OncePerThread (#58941) Simplify `workqueue_for`. While not strictly necessary, the acquire load in `getindex(once::OncePerThread{T,F}, tid::Integer)` makes ThreadSanitizer happy. With the existing implementation, we get false positives whenever a thread other than the one that originally allocated the array reads it: ``` ================== WARNING: ThreadSanitizer: data race (pid=6819) Atomic read of size 8 at 0xffff86bec058 by main thread: #0 getproperty Base_compiler.jl:57 (sys.so+0x113b478) #1 julia_pushNOT._1925 task.jl:868 (sys.so+0x113b478) #2 julia_enq_work_1896 task.jl:969 (sys.so+0x5cd218) #3 schedule task.jl:983 (sys.so+0x892294) #4 macro expansion threadingconstructs.jl:522 (sys.so+0x892294) #5 julia_start_profile_listener_60681 Base.jl:355 (sys.so+0x892294) #6 julia___init___60641 Base.jl:392 (sys.so+0x1178dc) #7 jfptr___init___60642 <null> (sys.so+0x118134) #8 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4) #9 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4) #10 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0xbba74) #11 jl_module_run_initializer /home/user/c/julia/src/toplevel.c:68:13 (libjulia-internal.so.1.13+0xbba74) #12 _finish_jl_init_ /home/user/c/julia/src/init.c:632:13 (libjulia-internal.so.1.13+0x9c0fc) #13 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4) #14 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8) #15 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934) #16 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20) Previous write of size 8 at 0xffff86bec058 by thread T2: #0 IntrusiveLinkedListSynchronized task.jl:863 (sys.so+0x78d220) #1 macro expansion task.jl:932 (sys.so+0x78d220) #2 macro expansion lock.jl:376 (sys.so+0x78d220) #3 julia_workqueue_for_1933 task.jl:924 (sys.so+0x78d220) #4 julia_wait_2048 task.jl:1204 (sys.so+0x6255ac) #5 julia_task_done_hook_49205 task.jl:839 (sys.so+0x128fdc0) #6 jfptr_task_done_hook_49206 <null> (sys.so+0x902218) #7 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4) #8 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4) #9 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9c79c) #10 jl_finish_task /home/user/c/julia/src/task.c:345:13 (libjulia-internal.so.1.13+0x9c79c) #11 jl_threadfun /home/user/c/julia/src/scheduler.c:122:5 (libjulia-internal.so.1.13+0xe7db8) Thread T2 (tid=6824, running) created by main thread at: #0 pthread_create <null> (julia+0x85f88) #1 uv_thread_create_ex /workspace/srcdir/libuv/src/unix/thread.c:172 (libjulia-internal.so.1.13+0x1a8d70) #2 _finish_jl_init_ /home/user/c/julia/src/init.c:618:5 (libjulia-internal.so.1.13+0x9c010) #3 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4) #4 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8) #5 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934) #6 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20) SUMMARY: ThreadSanitizer: data race Base_compiler.jl:57 in getproperty ================== ``` * Fix `hygienic-scope`s in inner macro expansions (#58965) Changes from https://github.com/JuliaLang/julia/pull/43151, github just didn't want me to re-open it. As discussed on slack, any `hygienic-scope` within an outer `hygienic-scope` can read and write variables in the outer one, so it's not particularly hygienic. The result is that we can't safely nest macro calls unless they know the contents of all inner macro calls. Should fix #48910. Co-authored-by: Michiel Dral <[email protected]> * remove comment from julia-syntax that is no longer true (#58964) The code this referred to was removed by c6c3d72d1cbddb3d27e0df0e739bb27dd709a413 * expand memoryrefnew capabilities (#58768) The goal here is 2-fold. Firstly, this should let us simplify the boundscheck (not yet implimented), but this also should reduce Julia IR side a bit. * Add news entry and update docstring for #58727 (#58973) * Fix alignment of failed precompile jobs on CI (#58971) * bpart: Tweak `isdefinedglobal` on backdated constant (#58976) In d2cc06193ef4161e4ac161bd4b5b57a51686a89a and prior commits, we made backdated access a conditional error (if depwarns are enabled or in generators). However, we did not touch `isdefinedglobal`. This resulted in the common pattern `isdefinedglobal(m, s) && getglobal(m, s)` to sometimes error. In particular, this could be observed when attempting to print a type from inside a generated function before that type's definition age. Additionally, I think the usage there, which used `invokelatest` on each of the two queries is problematic because it is racy, since the two `invokelatest` calls may be looking at different world ages. This makes two tweaks: 1. Makes `isdefinedglobal` consistent with `getglobal` in that it now returns false if `getglobal` would throw due to the above referenced restriction. 2. Removes the implicit `invokelatest` in _isself in the show code. Instead, it will use the current world. I considered having it use the exception age when used for MethodErrors. However, because this is used for printing it matters more how the object can be accessed *now* rather than how it could have been accessed in the past. * Fix precompilepkgs warn loaded setting (#58978) * specify that `Iterators.rest` must be given a valid `state` (#58962) ~currently `Iterators.rest(1:2, 3)` creates an infinite loop. after this PR it would be an `ArgumentError`~ docs only now * stdlib/Dates: Fix doctest regex to handle DateTime with 0 microseconds (#58981) The `now(UTC)` doctest can fail when the DateTime has exactly 0 milliseconds, as the output format omits the fractional seconds entirely (e.g., "2023-01-04T10:52:24" instead of "2023-01-04T10:52:24.000"). Update the regex filter to make the milliseconds portion optional by using `(\\.\\d{3})?` instead of `\\.\\d{3}`. Fixes CI failure: https://buildkite.com/julialang/julia-master/builds/49144#0197fd72-d1c6-44d6-9c59-5f548ab98f04 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Keno Fischer <[email protected]> Co-authored-by: Claude <[email protected]> * Fix unique for range wrappers with zero step (#51004) The current implementation assumes that the vector indexing `r[begin:begin]` is specialized to return a range, which isn't the case by default. As a consequence, ```julia julia> struct MyStepRangeLen{T,R} <: AbstractRange{T} x :: R end julia> MyStepRangeLen(s::StepRangeLen{T}) where {T} = MyStepRangeLen{T,typeof(s)}(s) MyStepRangeLen julia> Base.first(s::MyStepRangeLen) = first(s.x) julia> Base.last(s::MyStepRangeLen) = last(s.x) julia> Base.length(s::MyStepRangeLen) = length(s.x) julia> Base.step(s::MyStepRangeLen) = step(s.x) julia> r = MyStepRangeLen(StepRangeLen(1,0,4)) 1:0:1 julia> unique(r) ERROR: MethodError: Cannot `convert` an object of type Vector{Int64} to an object of type MyStepRangeLen{Int64, Int64, StepRangeLen{Int64, Int64, Int64, Int64}} [...] ``` This PR fixes this by using constructing a `UnitRange` instead of the indexing operation. After this, we obtain ```julia julia> unique(r) 1:1:1 ``` In principle, the `step` should be preserved, but `range(r[begin]::Int, step=step(r), length=length(r))` appears to error at present, as it tries to construct a `StepRange` instead of a `StepRangeLen`. This fix isn't perfect as it assumes that the conversion from a `UnitRange` _is_ defined, which is also not the case by default. For example, the following still won't work: ```julia julia> struct MyRange <: AbstractRange{Int} end julia> Base.first(x::MyRange) = 1 julia> Base.last(x::MyRange) = 1 julia> Base.length(x::MyRange) = 3 julia> Base.step(x::MyRange) = 0 julia> unique(MyRange()) ERROR: MethodError: no method matching MyRange(::UnitRange{Int64}) [...] ``` In fact, if the indexing `MyRange()[begin:begin]` has been specialized but the conversion from a `UnitRange` isn't, then this is actually a regression. I'm unsure if such pathological cases are common, though. The reason the first example works is that the conversion for a range wrapper is defined implicitly if the parent type supports conversion from a `UnitRange`. * Docs: add GC user docs (#58733) Co-authored-by: Andy Dienes <[email protected]> Co-authored-by: Gabriel Baraldi <[email protected]> Co-authored-by: Diogo Netto <[email protected]> * 🤖 [master] Bump the Pkg stdlib from 109eaea66 to b85e29428 (#58991) Co-authored-by: IanButterworth <[email protected]> * Add one-argument `argtypes` methods to source reflection functions (#58925) Follow-up to https://github.com/JuliaLang/julia/pull/58891#issuecomment-3036419509, extending the feature to `which`, `functionloc`, `edit` and `less`. * Test: Add compiler hint for `ts` variable definedness in `@testset for` (#58989) Helps the new language server avoid reporting unused variable reports. * trimming: explictly add Libdl dep for test/trimming/basic_jll.jl (#58990) * win/msys2: Automatically switch msys2 symlinks mode for LLVM (#58988) As noted in https://github.com/JuliaLang/julia/issues/54981#issuecomment-2336444226, msys2 currently fails to untar an llvm source build. Fix that by setting the appropriate environment variable to switch the symlinks mode. * Fix order of MSYS rules (#58999) git-external changes the LLVM_SRC_DIR variable, so the target-specific variable applies to the wrong target if defined before it - didn't notice in local testing because I had accidentally switched the variable globally earlier for testing - but showed up on a fresh build. * msys2: Recommend correct cmake package (#59001) msys2 ships 2 different cmake packages, one built natively (with mingw prefix in the package name) and one built against the posix emulation environment. The posix emulation one does not work because it will detect unix-style paths, which it then writes into files that native tools process. Unlike during command invocation (where the msys2 runtime library does path translation), when paths are written to files, they are written verbatim. The practical result of this is that e.g. the LLVM build will fail with a mysterious libz link failure (as e.g. reported in #54981). This is our fault, because our built instructions tell the user to install the wrong one. Fix all that by 1. Correcting the build instructions to install the correct cmake 2. Detecting if the wrong cmake is installed and advising the correct one 3. Fixing an issue where the native CMake did not like our CMAKE_C_COMPILER setting. With all this, CMake runs correctly under msys2 with USE_BINARYBUILDER_LLVM=0. * feat(REPL): Added `active_module` context to numbered REPL (#59000) * optimize `length(::OrdinalRange)` for large bit-ints (#58864) Split from #58793, this coalesces nearly all the branches in `length`, allowing it to inline and generally perform much better while retaining the exact same functionality. --------- Co-authored-by: N5N3 <[email protected]> * Fix LLVM TaskDispatcher implementation issues (#58950) Fixes #58229 (LLVM JITLink stack overflow issue) I tried submitting this promise/future implementation upstream (https://github.com/llvm/llvm-project/compare/main...vtjnash:llvm-project:jn/cowait-jit) so that I would not need to duplicate nearly as much code here to fix this bug, but upstream is currently opposed to fixing this bug and instead insists it is preferable for each downstream project to implement this fix themselves adding extra maintenance burden for us for now. Sigh. * Improve --trace-dispatch coverage: emit in "full-cache" fast path as well. (#59012) This PR moves the `--trace-dispatch` logging inside `jl_lookup_generic_` from only the `cache miss case` to also logging it inside the `no method was found in the associative cache, check the full cache` case. This PR logs the data from inside each of the two slow-path cases. * MozillaCACerts: Update to 2025-07-15 (#59010) * Fix use-after-free in FileWatching (#59017) We observe an abort on Windows on Revise master CI, where a free'd handle is passed to jl_close_uv. The root cause is that uv_fseventscb_file called uvfinalize earlier, but did not set the handle to NULL, so when the actual finalizer ran later, it would see corrupted state. * Roll up msys2/clang/windows build fixes (#59003) This rolls up everything I had to change to get a successful source build of Julia under msys2. It's a misc collection of msys2, clang and other fixes. With this, I can use the following Make.user: ``` USE_SYSTEM_CSL=1 USE_BINARYBUILDER_LLVM=0 CC=clang CXX=clang++ FC=gfortran ``` The default USE_SYSTEM_CSL is broken due to #56840 With USE_SYSTEM_CSL=1, LLVM is broken due to #57021 Clang is required because gcc can't do an LLVM source build due to known export symbol size limits (ref JuliaPackaging/Yggdrasil#11652). That said, if we address the ABI issues in #56840, the default Make.user should build again (with BB-provided LLVM). * Fix tar command (#59026) Scheduled build failing with ``` cd [buildroot]/deps/srccache/ && /usr/bin/tar --no-same-owner -xfz [buildroot]/deps/srccache/libunwind-1.8.2.tar.gz /usr/bin/tar: z: Cannot open: No such file or directory ``` Issue probably introduced in https://github.com/JuliaLang/julia/pull/58796. According to chatgpt this will fix it * Add 'sysimage' keyword for `JULIA_CPU_TARGET` to match (or extend) the sysimage target (#58970) * add `@__FUNCTION__` and `Expr(:thisfunction)` as generic function self-reference (#58940) This PR adds `@__FUNCTION__` to match the naming conventions of existing reflection macros (`@__MODULE__`, `@__FILE__`, etc.). --------- Co-authored-by: Jeff Bezanson <[email protected]> * Bugfix: Use Base.aligned_sizeof instead of sizeof in Mmap.mmap (#58998) fix #58982 * Fix PR reference in NEWS (#59046) * 🤖 [master] Bump the LibCURL stdlib from a65b64f to 038790a (#59038) Co-authored-by: IanButterworth <[email protected]> * 🤖 [master] Bump the DelimitedFiles stdlib from db79c84 to a982d5c (#59036) Co-authored-by: IanButterworth <[email protected]> * 🤖 [master] Bump the SHA stdlib from 4451e13 to 169a336 (#59041) Co-authored-by: IanButterworth <[email protected]> * 🤖 [master] Bump the Pkg stdlib from b85e29428 to 38d2b366a (#59040) Co-authored-by: IanButterworth <[email protected]> * 🤖 [master] Bump the Statistics stdlib from 77bd570 to 22dee82 (#59043) Co-authored-by: IanButterworth <[email protected]> * Expand JULIA_CPU_TARGET docs (#58968) * 🤖 [master] Bump the LinearAlgebra stdlib from 3e4d569 to 2c3fe9b (#59039) Co-authored-by: IanButterworth <[email protected]> Co-authored-by: Ian Butterworth <[email protected]> * 🤖 [master] Bump the SparseArrays stdlib from 6d072a8 to 30201ab (#59042) * 🤖 [master] Bump the JuliaSyntaxHighlighting stdlib from f803fb0 to b666d3c (#59037) * stored method interference graph (#58948) Store full method interference relationship graph in interferences field of Method to avoid expensive morespecific calls during dispatch. This provides significant performance improvements: - Replace method comparisons with precomputed interference lookup. - Optimize ml_matches minmax computation using interference lookups. - Optimize sort_mlmatches for large return sets by iterating over interferences instead of all matching methods. - Add method_morespecific_via_interferences in both C and Julia. This representation may exclude some edges that are implied by transitivity since sort_mlmatches will ensure the correct result by following strong edges. Ambiguous edges are guaranteed to be checkable without recursion. Also fix a variety of bugs along the way: - Builtins signature would cause them to try to discard all other methods during `sort_mlmatches`. - Some ambiguities were over-estimated, which now are improved upon. - Setting lim==-1 now gives the same limited list of methods as lim>0, since that is actually faster now than attempting to give the unsorted list. This provides a better fix to #53814 than #57837 and fixes #58766. - Reverts recent METHOD_SIG_LATEST_HAS_NOTMORESPECIFIC attempt (though not the whole commit), since I found a significant problem with any usage of that bit during testing: it only tracks methods that intersect with a target, but new methods do not necessarily intersect with any existing target. This provides a decent performance improvement to `methods` calls, which implies a decent speed up to package loading also (e.g. ModelingToolkit loads in about 4 seconds instead of 5 seconds). * build/llvm: Remove bash-specific curly expansion (#59058) Fixes #59050 * build: More msys2 fixes (#59028) * remove a testset from MMAP that might cause CI to now fail on Windows (#59062) * Use a dedicated parameter attribute to identify the gstack arg. (#59059) Otherwise, on systems without SwitfCC support (i.e. RISC-V) `getPGCstack` may return null, disabling the final GC pass. * skip unnecessary alias-check in `collect(::AbstractArray)` from `copyto!` (#55748) As discussed on Slack with @MasonProtter & @jakobnissen, `collect` currently does a usually cheap - but sometimes expensive - aliasing check (via `unalias`->`mightalias`->`dataid` -> `objectid`) before copying contents over; this check is unnecessary, however, since the source array is newly created and cannot possibly alias the input. This PR fixes that by swapping from `copyto!` to `copyto_unaliased!` in the `_collect_indices` implementations where the swap is straightforward (e.g., it is not so straightforward for the fallback `_collect_indices(indsA, A)`, so I skipped it there). This improves the following example substantially: ```jl struct GarbageVector{N} <: AbstractVector{Int} v :: Vector{Int} garbage :: NTuple{N, Int} end GarbageVector{N}(v::Vector{Int}) where N = GarbageVector{N}(v, ntuple(identity, Val(N))) Base.getindex(gv::GarbageVector, i::Int) = gv.v[i] Base.size(gv::GarbageVector) = size(gv.v) using BenchmarkTools v = rand(Int, 10) gv = GarbageVector{100}(v) @btime collect($v); # 30 ns (v1.10.4) -> 30 ns (PR) @btime collect($gv); # 179 ns (v1.10.4) -> 30 ns (PR) ``` Relatedly, it seems the fact that `mightalias` is comparing immutable contents as well - and hence slowing down the `unalias` check for the above `GarbageVector` via a slow `objectid` on tuples - is suboptimal. I don't know how to fix that though, so I'd like to leave that outside this PR. (Probably related to https://github.com/JuliaLang/julia/pull/26237) Co-authored-by: Matt Bauman <[email protected]> * Fix and update Revise manifest (#59077) * 🤖 [master] Bump the Pkg stdlib from 38d2b366a to 542ca0caf (#59083) Co-authored-by: IanButterworth <[email protected]> * Do not needlessly disable CPU features. (#59080) On QEMU's RISC-V cpu, LLVM's `getHostCPUFeatures` reports: ``` +zksed,+zkne,+zksh,+zfh,+zfhmin,+zacas,+v,+f,+c,+zvknha,+a,+zfa,+ztso,+zicond,+zihintntl,+zvbb,+zvksh,+zvkg,+zbkb,+zvkned,+zvbc,+zbb,+zvfhmin,+zbkc,+d,+i,+zknh,+zicboz,+zbs,+zvksed,+zbc,+zba,+zvknhb,+zknd,+zvkt,+zbkx,+zkt,+zvfh,+zvkb,+m ``` We change that to: ``` +zksed,+zkne,+zksh,+zfh,+zfhmin,+zacas,+v,+f,+c,+zvknha,+a,+zfa,+ztso,+zicond,+zihintntl,+zvbb,+zvksh,+zvkg,+zbkb,+zvkned,+zvbc,+zbb,+zvfhmin,+zbkc,+d,+i,+zknh,+zicboz,+zbs,+zvksed,+zbc,+zba,+zvknhb,+zknd,+zvkt,+zbkx,+zkt,+zvfh,+zvkb,+m,-zcmop,-zca,-zcd,-zcb,-zve64d,-zve64x,-zve64f,-zawrs,-zve32x,-zimop,-zihintpause,-zcf,-zve32f ``` i.e. we add `-zcmop,-zca,-zcd,-zcb,-zve64d,-zve64x,-zve64f,-zawrs,-zve32x,-zimop,-zihintpause,-zcf,-zve32f`, disabling stuff `zve*` after first enabling `v` (which includes `zvl*b`). That's not valid: ``` LLVM ERROR: 'zvl*b' requires 'v' or 'zve*' extension to also be specified ``` ... so disable this post-processing of LLVM feature sets and trust what it spits out. AFAICT this only matters for the fallback path of `processor.cpp`, so shouldn't impact most users. * build: Also pass -fno-strict-aliasing for C++ (#59066) As diagnosed by Andrew Pinski (https://github.com/JuliaLang/julia/issues/58466#issuecomment-3105141193), we are not respecting strict aliasing currently. We turn this off for C, but the flag appears to be missing for C++. Looks like it's been that way ever since that flag was first added to our build system (#484). We should probably consider running TypeSanitizer over our code base to see if we can make our code correct under strict aliasing as compilers are increasingly taking advantage of it. Fixes #58466 * Fix typo in `include`'s docstring (#59055) * results.json: Fix repo paths so links to github work (#59090) * Update RISC-V building docs. (#59088) We have pre-built binaries for RISC-V now. * Test: improve type stabilities (#59082) Also simplifies code a bit, by removing unnecessary branches. * LibCURL_jll: New version 8.15.0 (#59057) Note that CURL 8.15.0 does not support using Secure Transport on MacOS any more. This PR thus switches CURL to using OpenSSL on MacOS. --------- Co-authored-by: Mosè Giordano <[email protected]> * Switch RISC-V to large model on LLVM 20 (#57865) Co-authored-by: Tim Besard <[email protected]> * Support complex numbers in eps (#21858) This came up in https://github.com/JuliaMath/IterativeSolvers.jl/pull/113#issuecomment-301273365 . JuliaDiffEq and IterativeSolvers.jl have to make sure that the real-type is pulled out in order for `eps` to work: ```julia eps(real(typeof(b))) ``` This detail can make many algorithms with tolerances that are written generically that would otherwise work with complex numbers error. This PR proposes to do just that trick, so that way `eps(1.0 + 1.0im)` returns machine epsilon for a Float64 (and generally works for `AbstractFloat` of course). --------- Co-authored-by: Steven G. Johnson <[email protected]> * 🤖 [master] Bump the Pkg stdlib from 542ca0caf to d94f8a1d9 (#59093) Co-authored-by: IanButterworth <[email protected]> * add array element mutex offset in print and gc (#58997) The layout, printing, and gc logic need to correctly offset and align the inset fields to account for the per-element mutex of an atomic array with large elements. Fix #58993 * Fix typo in tests introduced by #21858 (#59102) That [2017 PR](https://github.com/JuliaLang/julia/pull/21858) used very old types and had a semantic merge conflict. * Fix msys symlink override rule (#59101) The `export VAR=VAL` is syntax, so it can't be expanded. Fixes #59096 * inference: Make test indepdent of the `Complex` method table (#59105) * Add uptime to CI test info (#59107) * Fix rounding when converting Rational to BigFloat (#59063) * make ReinterpretArray more Offset-safe (#58898) * remove extraneous function included in #21858 (#59109) Removes an apparently extraneous function accidentally included in #21858, as noted in https://github.com/JuliaLang/julia/pull/21858/files#r2233250284. * [REPL] Handle empty completion, keywords better (#59045) When the context is empty, (like "<TAB><TAB>"), return only names local to the module (fixes #58931). If the cursor is on something that "looks like" an identifier, like a boolean or one of the keywords, treat it as if it was one for completion purposes. Typing a keyword and hitting tab no longer returns the completions for the empty input (fixes #58309, #58832). * Add builtin function name to add methods error (#59112) ``` julia> Base.throw(x::Int) = 1 ERROR: cannot add methods to builtin function `throw` Stacktrace: [1] top-level scope @ REPL[1]:1 ``` * better error in juliac for defining main inside a new module (#59106) This is more helpful if the script you try to compile defines a module containing main instead of defining it at …

@inbounds

… of `ReshapedArray` (JuliaLang#43518) This performance difference was found when working on JuliaLang#42736. Currently, our `ReshapedArray` use stride based `MultiplicativeInverse` to speed up index transformation. For example, for `a::AbstractArray{T,3}` and `b = vec(a)`, the index transformation is equivalent to: ```julia offset = i - 1 # b[i] d1, r1 = divrem(offset, stride(a, 3)) # stride(a, 3) = size(a, 1) * size(a, 2) d2, r2 = divrem(r1, stride(a, 2)) # stride(a, 2) = size(a, 1) CartesianIndex(r2 + 1, d2 +1, d1 + 1) # a has one-based axes ``` (All the `stride` is replaced with a `MultiplicativeInverse` to accelerate `divrem`) This PR wants to replace the above machinery with: ```julia offset = i - 1 d1, r1 = divrem(offset, size(a, 1)) d2, r2 = divrem(d1, size(a, 2)) CartesianIndex(r1 + 1, r2 +1, d2 + 1) ``` For random access, they should have the same computational cost. But for sequential access, like `sum(b)`, `size` based transformation seems faster. To avoid bottleneck from IO, use `reshape(::CartesianIndices, x...)` to benchmark: ```julia f(x) = let r = 0 for i in eachindex(x) @inbounds r |= +(x[i].I...) end r end a = CartesianIndices((99,100,101)); @Btime f(vec($a)); mmtk#2.766 ms --> 2.591 ms @Btime f(reshape($a,990,1010)); mmtk#3.412 ms --> 2.626 ms @Btime f(reshape($a,33,300,101)); mmtk#3.422 ms --> 2.342 ms ``` I haven't looked into the reason for this performance difference. Beside acceleration, this also makes it possible to reuse the `MultiplicativeInverse` in some cases (like JuliaLang#42736). So I think it might be useful? --------- Co-authored-by: Andy Dienes <[email protected]> Co-authored-by: Andy Dienes <[email protected]>

udesou and others added 2 commits January 24, 2023 05:50

Adapting Julia to work with MMTK

1a250ba

Updating Julia to latest master

ee516db

udesou mentioned this pull request Jan 25, 2023

Update/julia master mmtk/mmtk-julia#18

Merged

udesou requested a review from qinsoon January 25, 2023 10:52

qinsoon approved these changes Jan 31, 2023

View reviewed changes

qinsoon merged commit e8ba639 into mmtk:master Jan 31, 2023

qinsoon pushed a commit to qinsoon/julia that referenced this pull request Feb 2, 2023

Update/julia master (mmtk#2)

3748508

This PR updates the binding to the latest Julia master (up to this commit: 134f3e7).

kpamnany pushed a commit that referenced this pull request Mar 13, 2023

Update/julia master (#2)

be136ce

This PR updates the binding to the latest Julia master (up to this commit: 134f3e7).

kpamnany pushed a commit that referenced this pull request Mar 15, 2023

Update/julia master (#2)

28228aa

This PR updates the binding to the latest Julia master (up to this commit: 134f3e7).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update/julia master #2

Update/julia master #2

Uh oh!

udesou commented Jan 25, 2023

Uh oh!

qinsoon left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Update/julia master #2

Update/julia master #2

Uh oh!

Conversation

udesou commented Jan 25, 2023

Uh oh!

qinsoon left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants