[Bug]: RuntimeError: Library libcublas.so.12 is not found or cannot be loaded

[alexandremasanes]

Issue

When trying to translate a manga page with offline translation I get the following error :
RuntimeError: Library libcublas.so.12 is not found or cannot be loaded

Running ldconfig -p | grep libcublas outputs this :
libcublasLt.so.12 (libc6,x86-64) => /opt/cuda/lib64/libcublasLt.so.12 libcublasLt.so (libc6,x86-64) => /opt/cuda/lib64/libcublasLt.so libcublas.so.12 (libc6,x86-64) => /opt/cuda/lib64/libcublas.so.12 libcublas.so (libc6,x86-64) => /opt/cuda/lib64/libcublas.so

I do have libnvidia-container-tool and extra/libnvidia-container installed.

I start the webserver using docker-compose -f demo/doc/docker-compose-web-with-gpu.yml up

Command Line Arguments

Console logs

[alexandremasanes]

I fixed this error by updating the compose file to this :

version: "3.9"

services:
  manga_image_translator:
    image: zyddnys/manga-image-translator:main
    container_name: manga_image_translator_gpu
    entrypoint: python
    command: server/main.py --verbose --start-instance --host=0.0.0.0 --port=5003 --use-gpu
    volumes:
      - ./../../result:/app/result
      - ./../../server/main.py:/app/server/main.py
      - ./../../server/instance.py:/app/server/instance.py
      # Mount host CUDA libraries so PyTorch can find them
      - /usr/local/cuda/lib64:/usr/local/cuda/lib64:ro
    ports:
      - 5003:5003
    ipc: host
    # Use NVIDIA GPU runtime
    runtime: nvidia
    environment:
      NVIDIA_VISIBLE_DEVICES: all
      NVIDIA_DRIVER_CAPABILITIES: compute,utility
      LD_LIBRARY_PATH: /usr/local/cuda/lib64
      # baidu  
      BAIDU_APP_ID: ''  
      BAIDU_SECRET_KEY: ''  
      # youdao  
      YOUDAO_APP_KEY: ''  
      YOUDAO_SECRET_KEY: ''  
      # deepl  
      DEEPL_AUTH_KEY: ''  
      # openai/ChatGPT  
      OPENAI_API_KEY: ''
      OPENAI_API_BASE: ''
      OPENAI_MODEL: ''
      OPENAI_HTTP_PROXY: ''
      # glossary
      OPENAI_GLOSSARY_PATH: './dict/mit_glossary.txt'
      # Groq  
      GROQ_API_KEY: ''  
      GROQ_MODEL: 'mixtral-8x7b-32768'  
      # sakura  
      SAKURA_API_BASE: 'http://127.0.0.1:8080/v1'  
      SAKURA_VERSION: '0.9'  
      SAKURA_DICT_PATH: './sakura_dict.txt'  
      # Caiyun  
      CAIYUN_TOKEN: ''  
      # Gemini  
      GEMINI_API_KEY: ''  
      GEMINI_MODEL: 'gemini-1.5-flash-002'  
      # DeepSeek  
      DEEPSEEK_API_KEY: ''  
      DEEPSEEK_API_BASE: 'https://api.deepseek.com'  
      DEEPSEEK_MODEL: 'deepseek-chat'  
      # Custom OpenAI compatible API  
      CUSTOM_OPENAI_API_KEY: 'ollama'  
      CUSTOM_OPENAI_API_BASE: 'http://localhost:11434/v1'  
      CUSTOM_OPENAI_MODEL: ''  
      CUSTOM_OPENAI_MODEL_CONF: ''  
    command: >
      bash -c "echo '/usr/local/cuda/lib64' > /etc/ld.so.conf.d/cuda.conf &&
               ldconfig &&
               python server/main.py --verbose --start-instance --host=0.0.0.0 --port=5003 --use-gpu"

[frederik-uni]

Haven’t tested it but why not LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
python server/main.py --verbose --start-instance --host=0.0.0.0 --port=5003 --use-gpu

Seems cleaner to me and should do almost the same