Is it not possible to `load` transformer model on cpu only

[Lianowar]

What happened?

I need to run inference of BERT4Rec on a CPU-only instance, but I can't.
When I try to load the fitted model, I get a PyTorch error.

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

This occurs in __setstate__ at rectools/models/nn/transformers/base.py:596, and it’s currently impossible to pass the map_location parameter when loading the model.
Is exists some workaround there?

Expected behavior

No response

Relevant logs and/or screenshots

No response

Operating System

ubuntu

Python Version

3.11

RecTools version

0.13.0

[blondered]

Hi!
Thank you for highlighting this.
We ourselves usually use pytorch lightning checkpoints in practice for saving and loading models. E.g. you can create a callback that saves model checkpoint on last epoch during training and use this checkpoint with load_from_checkpoint method and pass map_location there.

[Lianowar]

Is it ready for production use?
As I understand, when using load_from_checkpoint, I can't use the recommend method before fitting the model again.
And I don’t want to fit the model every time the server starts :)

[blondered]

We do have an issue that loaded model can't be saved without fitting again due to pytorch-lightning specific behaviour. But we didn't have any problems with recommending.
Which problems do you face in this scenario?