Split CUDA language support into separate package

[maleadt]

Both @wsmoses and @ejmeitz asked if it would be possible to depend on only the CUDA kernel language support (and presumably the ability to execute CUDA code), and not the whole CUDA package including all of the libraries. This actually seems like a reasonably thing, e.g. introducing CUDALang.jl or something, but it's nontrivial:

• Move more stuff to GPUToolbox
• Remove the dependence of non-library submodules on CUDA.jl (e.g. CUDAdrv/CUPTI can't rely on APIUtils)
• Make CUDAdrv its own package
• Make CUPTI its own package
• Make CUDALang its own package, depending on CUDAdrv and CUPTI

[jonas-schulze]

This sounds a bit like the list of packages you had moved away from (CUDAnative.jl, CUDAapi.jl, CUDAdrv.jl, etc.); at least to a novice like me. What would be the benefit? Do I then need to decide again, what level of abstraction requires what package to use an NVIDIA GPU from Julia?

For some of my packages, I like to trade convenience (on my workstation) for efficiency (on the CI runner without a GPU) by defining my tests like this:

using CUDA
using Tests

tests = [
    ("Vector", [1, 2, 3], 6),
    ("Iterable", 1:3, 6),
]

if CUDA.functional()
    push!(tests, ("CuVector", CuVector([1, 2, 3]), 6))
end

@testset "summation ($desc)" for (desc, input, expected) in tests
    @test sum(input) === expected
end

Would the roadmap described in this issue help with reducing the overhead of a setup like the one I just described?

[maleadt]

The idea would be that CUDA.jl remains the parent package, but alternative back-ends (like Legate or Reactant) which can consume CUDA kernels but don't need the compiler or library deps can do so without the humongous CUDA.jl dependency.

That said, I'm not fully convinced either, and don't have the time to work on this anyway, so this presumably won't be happening anytime soon.