Setup for using Banded Structures in BVP

[avik-pal]

• Upstream Banded QR Piracy Proper handling for BandedMatrices LinearSolve.jl#393
• Adjoint / Transpose Banded Matrices proper support Add dispatches for Transpose and Adjoint for Banded Matrices JuliaArrays/ArrayInterface.jl#422
• Support for MIRK Methods
• Support for Multiple Shooting
  • Implement Reordering for MS
• Change defaults for linear solve (banded default selection doesn't work) Proper handling for BandedMatrices LinearSolve.jl#393
• CUDA / AMDGPU scalar indexing patch for Banded Matrices
• Component Arrays

[ChrisRackauckas]

Conflicts

[avik-pal]

@ChrisRackauckas should we go ahead with this? If using CUDA / AMDGPU then we throw a fairly obvious error. Supporting it properly needs some custom cuda kernels (the state of CUSPARSE seems weird with most operations sending the arrays back to CPU)

[ChrisRackauckas]

Supporting it properly needs some custom cuda kernels (the state of CUSPARSE seems weird with most operations sending the arrays back to CPU)

Yes. GPU needs a lot more. It needs custom kernel generation for how to construct the Jacobian, and then special CUSPARSE routines where we use the ilu0 (which is missing ldiv!'s). It needs more work, let's do that part separately.

[avik-pal]

The CUDA part will unfortunately be a massive blocker if we want to Neural BVPs 😞

[ChrisRackauckas]

At least MultipleShooting is ready for CUDA?

[avik-pal]

Need to test, but most likely no. It also specializes heavily on sparsity patterns

[avik-pal]

For the purposes of Neural BVP if we avoid passing in the jacobian function and just use GMRES we should be good to go (I think)

[ChrisRackauckas]

Yes that should be the case. Jacobian-free should be fine. Anything with a Jacobian involved will take more work (but be better overall). But that's why I had it as a separate project 😅