Integrate SciMLOperators with rest of the ecosystem

[vpuri3]

Previous attempts to integrate SciMLOperators into the ecosystem broke everything. This was because we coupled together two separate tasks:

1. supporting SciMLOperators
2. deprecating AbstractDiffEqOperator, DiffEqLinearOperator, AbstractDiffEqCompositeOperator, and the concrete types.

So we were never able to get anywhere because errors (in downstream packages) from the deprecations in step 2 were too many to handle. This time we shall do the integration step-by-step.

• Define all operators and traits in SciMLOperators
• Have SciMLBase reexport SciMLOperators. Do NOT deprecate diffeqoperator family
• Ditto for DiffEqBase.jl
• Add support for SciMLOperators in downstream packages.
• Replaces subtypes of *DiffEqOperators to AbstractSciMLOperators. Do NOT remove support for DiffEqOperators. Order below:
• LinearSolve.jl - make preconditioners AbstractSciMLOperators
• Sundials.jl
• DiffEqOperators.jl
• SparseDiffTools.jl - make AD operators AbstractSciMLOperators
• DiffEqSensitivity.jl - use SparseDiffTools AD operators in SteadyStateAdjoint. Use adjoint(A) if islinear(A) = true, has_adjoint(A) = true.
• OrdinaryDiffEq.jl - form WOperator just by composing AbstractSciMLOperator: W = 1/gamma * M - J or the opp
• StochasticDiffEq.jl
• Once SciMLOperators are supported everywhere, remove DiffEqOperators starting from downstream packages to upstream packages.
• Finally deprecate AbstractDiffEqOperator family in SciMLBase.jl
• remove DiffEqOperators from docs

EDITS -

TODOs

SciMLBase.jl

deprecate all diffeqoperator subtypes

LinearSolve.jl

• update preconditioner docs
• deprecations:

const ComposePreconditioner = SciMLOperators.ComposedOperator
@deprecate ComposePreconditioner SciMLOperators.ComposedOperator

const InvPreconditioner = SciMLOperators.InvertedOperator
@deprecate InvPreconditioner SciMLOperators.InvertedOperator

• in defaultalg selection, as well as in init_cacheval, OperatorAssumptions{nothing} behaves exactly like OperatorAssumptions{true}. So let's remove the option to put in nothing.
• in src/common.jl, set_A also modify cache.OperatorAssumptions.
• in src/default.jl, remove method defaultalg(A::Diagonal, b, ::OperatorAssumptions{false}) because diagonal matrices are always square
• In src/factorization.jl, DiagonalFactorization should ideally preinvert the D.diag in init_cacheval, and then mul! in SciMLBase.solve. Create a separate PR
• replace IterativeSolvers: Identity in OrdinaryDiffEq with IdentityOperator.

DiffEqBase.jl

In test/basic_operator_tests.jl, test/affine_operator_tests.jl, there are tests for DiffEqOperators. Let's leave that as it is till DiffEqOps are remove from downstream packages.

SparseDiffTools.jl

• Add OrdinaryDiffEq.jl - InterfaceII to downstream tests. Same for SciMLSensitivity.jl when we get done with that.
• Add JacVec ODE tests
• autodiff shouldn't be boolean logic but multi-valued logic through types choices, like AutoZygote(). change the autodiff choices to be non-boolean logic and Zygote an extension package in order to release.
• release v2 Release v2 JuliaDiff/SparseDiffTools.jl#230

# JacVec OrdinaryDiffEq itnegration test

function lorenz(du, u, p, t)
    du[1] = 10.0(u[2] - u[1])
    du[2] = u[1] * (28.0 - u[3]) - u[2]
    du[3] = u[1] * u[2] - (8 / 3) * u[3]
end
u0 = [1.0; 0.0; 0.0]
tspan = (0.0, 100.0)

ff1 = ODEFunction(lorenz, jac_prototype = JacVec(lorenz, u0))
ff2 = ODEFunction(lorenz, jac_prototype = JacVec(lorenz, u0, autodiff=false))

for ff in [ff1, ff2]
    prob = ODEProblem(ff, u0, tspan)
    @test solve(prob, TRBDF2()).retcode == :Success
    @test solve(prob, TRBDF2(linsolve = KrylovJL_GMRES())).retcode == :Success
    @test solve(prob, Exprb32()).retcode == :Success
    @test solve(prob, Rosenbrock23()).retcode == :Success
    @test solve(prob, Rosenbrock23(linsolve = KrylovJL_GMRES())).retcode == :Success
end

[vpuri3]

CC: @ChrisRackauckas, @xtalax

[xtalax]

Ready to help, but not sure how helpful I can be towards operator translation as I don't fully understand their requirements

[vpuri3]

@ChrisRackauckas LMK when you register sparsedifftools v2.0, and I'll start with SciMLSensitivity, OrdinaryDiffEq

[ChrisRackauckas]

It should be hopefully alter this week. It needs JuliaDiff/SparseDiffTools.jl#215 to be completed first, but I can help with the bits there. Note that the ArrayInterface overhaul "comes first" and may clog up the pipeline this week, hopefully that moves fast.

[vpuri3]

@gaurav-arya do you wanna take a stab at OrdinaryDiffeq

[gaurav-arya]

yes, happy to since I've already looked at a lot of the operator code there. Would need a merge of #143 though

[vpuri3]

there are many things to do in OrdinaryDiffEq. while the kwargs PR is in review can you do a small PR in OrdinaryDiffEq to support Sparsedifftools v2? that's blocking the scimlsensitivity pr.

[gaurav-arya]

Sounds good, I'll get to it Friday/Saturday

[ChrisRackauckas]

I had to pull back on OrdinaryDiffEq.jl for now because it needs to update update_coefficients! for the arguments change. And Sundials.jl is getting a test failure due to the change of JacVec (it doesn't depend on SparseDiffTools but does use JacVec in a test).

[gaurav-arya]

For the arguments change, are you referring to #143 or something else? Because #143 should be totally backwards compatible -- it gives SciMLOperators the flexibility to accept keyword arguments, but only in special situations when they expect to receive them from the caller.