Add support for solver cuPDLPx

[mal84emma]

Changes proposed in this Pull Request

Add support for GPU-accelerated solver cuPDLPx

Note: due to the lower numerical precision of this solver, some of the equality assertions in the tests have been replaced with np.isclose
Also note: this solver does not support QP or MIP, and currently the file IO is not available through the Python API (will add when later versions support it)

Checklist

• Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
• Unit tests for new features were added (if applicable).
• A note for the release notes doc/release_notes.rst of the upcoming release is included.
• I consent to the release of this PR's code under the MIT license.

[fneum]

Looks pretty good!

The higher tolerances in this PR largely apply to all solvers. It could be an option to introduce a constant that sets the tolerance based on whether it's a GPU or CPU-based solver.

[mal84emma]

Great point :) I've just added global variables to the tests that set the acceptable relative solution tolerances for CPU and GPU based solvers.

[FabianHofmann]

@mal84emma wonderful, I hope you don't mind me pushing a final small commit for fixing minor stuff

[mal84emma]

@FabianHofmann Fix looks great, very happy with it, thanks for catching!

[FabianHofmann]

@mal84emma could you do me two favors?

1. the installation of the cupdlpx solver seems broken and it is hard to understand why (see CI), do you have a clue?
2. we recently introduced the solver registry (see Feature/solver features registry #528 ) which I would also like to leverage in the test scripts to avoid things like if solver not in feasible_mip_solvers: would you mind looking into this?

[mal84emma]

@FabianHofmann absolutely, delighted to!

1. Reading the CI logs, the underlying issue is Failed to find nvcc. from cmake.
   I'm not super familiar with CI workflows, but my best guess is that the machines that the workflows are being tested on don't have GPUs, and so trying to install cuPDLPx will always fail on them.
   This is probably the major caveat of the GPU accelerated solvers. Trying to get a working CUDA/NVCC install is pretty painful. I've found I need to manually install CUDA and build the cuPDLPx binary locally before doing the pip install.
   I'm not sure if it's possible to remove it from the CI workflow and have an experimental/hard dev warning?

2. This looks much neater, very happy to give it a look in. But to be upfront I have my PhD viva today and I'm starting a job soon, so it might be a while before I get the time to make the edits.

[ollie-bell]

FWIW cuOpt has already implemented the core concepts outlined in the cupdlpx paper, and is more mature and better maintained as a library. Similarly, HiGHS has also committed to releasing a built in GPU-enabled PDLP solver in the coming months (and the implementation is well under way) which will also implement core concepts from the cupdlpx paper.

For HiGHS it goes without saying that from a CI perspective, nothing would need to change. cuOpt still suffers from the problem of needing a compatible GPU at runtime, though. They are aware of this being an issue and seem open to having tests for modelling languages in their own repo, see NVIDIA/cuopt#155

[mal84emma]

cuOpt is a great shout, I hadn't realised they'd implemented the cuPDLPx improvements already.
I'll try to find some time to look at adding support for it (at some point). Help would always appreciated if you're interested.

I think there's a lot of cool developments on the horizon for GPU accelerated LP (e.g. GPU IPMs), and it'd be awesome to have the new solver options available as they come out.
(I've been getting some seriously impressive results benchmarking cuPDLPx on some A100s - faster solves than CPU Gurobi barrier!)

[ollie-bell]

(e.g. GPU IPMs)

cuOpt also has recently released a GPU barrier solver. I haven't done any benchmarking with it myself.