Error: Exception: Failure fetching models from Azure Foundry catalog

[leestott]

OpenVinoExecution provider Exception on Intel Devices

Error

🟢 Service is Started on http://127.0.0.1:53483/, PID 1304!
🕖 Downloading and registering OpenVINOExecutionProvider [1/2]... Exception: Failure fetching models from Azure Foundry catalog
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..

Repo:

Tried two versions of foundry local unistallled and reinstalled foundry local app.
Deleted .foundry user users\

Versions Tested

foundry --version
0.8.119

foundry --version
0.8.117

log

foundry cache list
🟢 Service is Started on http://127.0.0.1:55604/, PID 15472!
Models cached on device:
Alias Model ID
🕛 Downloading and registering OpenVINOExecutionProvider [1/2]... Exception: Failure fetching models from Azure Foundry catalog
Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
[13:12:34 ERR] LogException
Microsoft.AI.Foundry.Local.Common.FLException: Failure fetching models from Azure Foundry catalog
---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
at System.Net.Sockets.NetworkStream.Read(Span1) + 0x59 --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Read(Span1) + 0xf1
at System.Net.Http.HttpConnection.d__82.MoveNext() + 0xde
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.Fill() + 0x8b
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.Read(Span1) + 0xbd at System.Net.Http.HttpBaseStream.Read(Byte[], Int32, Int32) + 0x51 at System.IO.StreamReader.ReadBuffer() + 0x37 at System.IO.StreamReader.get_EndOfStream() + 0x34 at Microsoft.AI.Foundry.Local.Common.EPManagement.<InitializeEPsAsync>d__6.MoveNext() + 0xb7c --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Common.EPManagement.<InitializeEPsAsync>d__6.MoveNext() + 0x1138 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<GetAutoRegisterEPStatus>d__23.MoveNext() + 0x73b --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<InitModelsAsync>d__22.MoveNext() + 0x3cf --- End of inner exception stack trace --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<InitModelsAsync>d__22.MoveNext() + 0xe7b --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<Models>d__8.MoveNext() + 0x53 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<GetModelAsync>d__16.MoveNext() + 0x63 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Commands.CacheListCommand.<<Create>b__2_0>d.MoveNext() + 0x4e1 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Common.CommandActionFactory.<>c__DisplayClass0_01.<b__0>d.MoveNext() + 0x1e7
--- End of stack trace from previous location ---
at System.CommandLine.NamingConventionBinder.CommandHandler.d__66.MoveNext() + 0x50
--- End of stack trace from previous location ---
at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.d__11.MoveNext() + 0x61
--- End of stack trace from previous location ---
at System.CommandLine.Invocation.InvocationPipeline.d__0.MoveNext() + 0x1cd
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.Program.

d__1.MoveNext() + 0x4e4

Log from .foundry

at Microsoft.AI.Foundry.Local.Common.ServiceManagement.d__9.MoveNext() + 0x5f
2026-01-29 13:17:31.743 +00:00 [INF] Starting service <C:\Program Files\WindowsApps\Microsoft.FoundryLocal_0.8.119.102_x64__8wekyb3d8bbwe\Inference.Service.Agent.exe --urls="http://127.0.0.1:0/" --OpenAIServiceSettings:ModelDirPath="C:\Users******.foundry\cache\models" --JsonRpcServer:Run=true --JsonRpcServer:PipeName="inference_agent" --Logging:LogLevel:Default="Information">
2026-01-29 13:17:31.920 +00:00 [INF] Service endpoints are not yet bound after 1 tries, waiting to retry...
2026-01-29 13:17:31.926 +00:00 [INF] Now listening on: http://127.0.0.1:53483
2026-01-29 13:17:31.926 +00:00 [INF] Application started. Press Ctrl+C to shut down.
2026-01-29 13:17:31.926 +00:00 [INF] Hosting environment: Production
2026-01-29 13:17:31.927 +00:00 [INF] Content root path: C:\Program Files\WindowsApps\Microsoft.FoundryLocal_0.8.119.102_x64__8wekyb3d8bbwe
2026-01-29 13:17:32.002 +00:00 [INF] Provider 1/2: OpenVINOExecutionProvider is in NotReady state. Downloading and ensuring provider is ready.
2026-01-29 13:17:32.049 +00:00 [INF] Provider OpenVINOExecutionProvider download and ensuring ready attempt: Success
2026-01-29 13:17:32.072 +00:00 [INF] Provider 1/2: OpenVINOExecutionProvider is now in Ready state. Now registering provider.
2026-01-29 13:17:32.433 +00:00 [INF] Found service endpoints: http://127.0.0.1:53483
2026-01-29 13:17:32.433 +00:00 [INF] Service is started on http://127.0.0.1:53483/, PID 1304!
2026-01-29 13:17:32.434 +00:00 [INF] Command:ModelInit Status:Success Direct:True Time:938ms
2026-01-29 13:17:32.434 +00:00 [INF] Command:ServiceStart Status:Failure Direct:False Time:939ms
2026-01-29 13:17:32.434 +00:00 [INF] Checking EP autoregistration status...
2026-01-29 13:17:32.463 +00:00 [INF] UserAgent:foundry-local-CLI/0.8.119 Command:ServiceStatus Status:Success Direct:True Time:0ms
2026-01-29 13:17:32.463 +00:00 [INF] Processing EP autoregistration status...
2026-01-29 13:17:32.463 +00:00 [INF] Command:ServiceStatus Status:Success Direct:False Time:29ms
2026-01-29 13:17:32.466 +00:00 [INF] UserAgent:foundry-local-CLI/0.8.119 Command:AutoRegisterCertifiedEps Status:Success Direct:True Time:0ms
2026-01-29 13:17:32.467 +00:00 [INF] Reporting progress of running EP download
2026-01-29 13:17:35.952 +00:00 [INF] Stream disconnected
2026-01-29 13:17:35.954 +00:00 [INF] Command:ServiceAutoRegister Status:Failure Direct:False Time:3491ms
2026-01-29 13:17:35.954 +00:00 [INF] Command:ModelList Status:Failure Direct:True Time:4459ms
2026-01-29 13:17:35.955 +00:00 [ERR] LogException
Microsoft.AI.Foundry.Local.Common.FLException: Failure fetching models from Azure Foundry catalog
---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
at System.Net.Sockets.NetworkStream.Read(Span1) + 0x59 --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Read(Span1) + 0xf1
at System.Net.Http.HttpConnection.d__82.MoveNext() + 0xde
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.Fill() + 0x8b
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.Read(Span1) + 0xbd at System.Net.Http.HttpBaseStream.Read(Byte[], Int32, Int32) + 0x51 at System.IO.StreamReader.ReadBuffer() + 0x37 at System.IO.StreamReader.get_EndOfStream() + 0x34 at Microsoft.AI.Foundry.Local.Common.EPManagement.<InitializeEPsAsync>d__6.MoveNext() + 0xb7c --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Common.EPManagement.<InitializeEPsAsync>d__6.MoveNext() + 0x1138 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<GetAutoRegisterEPStatus>d__24.MoveNext() + 0x73b --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<InitModelsAsync>d__23.MoveNext() + 0x3cf --- End of inner exception stack trace --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<InitModelsAsync>d__23.MoveNext() + 0xe7b --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<Models>d__8.MoveNext() + 0x53 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Catalog.AzureFoundryCatalog.<DisplayModelInfoAsync>d__22.MoveNext() + 0x5f --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Commands.ModelListCommand.<<Create>b__2_0>d.MoveNext() + 0x167 --- End of stack trace from previous location --- at Microsoft.AI.Foundry.Local.Common.CommandActionFactory.<>c__DisplayClass0_01.<b__0>d.MoveNext() + 0x1e7
--- End of stack trace from previous location ---
at System.CommandLine.NamingConventionBinder.CommandHandler.d__66.MoveNext() + 0x50
--- End of stack trace from previous location ---
at System.CommandLine.NamingConventionBinder.ModelBindingCommandHandler.d__11.MoveNext() + 0x61
--- End of stack trace from previous location ---
at System.CommandLine.Invocation.InvocationPipeline.d__0.MoveNext() + 0x1cd
--- End of stack trace from previous location ---
at Microsoft.AI.Foundry.Local.Program.

d__1.MoveNext() + 0x4cb

[leestott]

Confirming this is only effecting intel devices

ARM Devices running Qualcomm are working correctly

[HamidOna13]

Experiencing the same issue on a machine with Intel CPU + NVIDIA GPU (Windows, Foundry 0.8.119). The service starts but crashes during OpenVINO EP download, even though the intended runtime is CUDA/TensorRT. Previously cached GPU models can't be loaded because the EP auto-registration failure kills the entire process before reaching model loading. Could EP registration failures for one provider be non-fatal so other available providers can still be used?

[leestott]

I have tried

Get-AppxPackage -AllUsers "WindowsWorkload.EP*" | Remove-AppxPackage

Get-AppxPackage -AllUsers ".EP." | Select-Object -ExpandProperty PackageFullName

Get-AppxPackage -AllUsers "*.EP.*" | Select-Object -ExpandProperty PackageFullName MicrosoftCorporationII.WinML.Intel.OpenVINO.EP.1.8_1.8.63.0_x64__8wekyb3d8bbwe MicrosoftCorporationII.WinML.NVIDIA.TRT-RTX.EP.1.8_1.8.22.0_x64__8wekyb3d8bbwe

Then try foundry service init.

❯ foundry service init 🟢 Service is Started on http://127.0.0.1:63530/, PID 11904! 🕗 Downloading and registering OpenVINOExecutionProvider [1/2]... An unexpected error occurred. Exception: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. An existing connection was forcibly closed by the remote host.

[leestott]

Also tried this

Run this script in Powershell Admin User to clean up the installed EP.

foundry service stop

Get-AppxPackage -AllUsers ".EP." | Select-Object -ExpandProperty PackageFullName

Get-AppxPackage -AllUsers ".EP." | ForEach-Object { try { remove-AppxPackage -Package $.PackageFullName -AllUsers Write-Host "Removed package: $($.PackageFullName)" } catch { Write-Warning "Failed to remove package: ( .PackageFullName). Error: $" } } remove-appxpackage uup://product/Windows.Workload.ExecutionProvider.TRT.amd64 remove-appxpackage uup://product/Windows.Workload.ExecutionProvider.OpenVino.amd64 remove-appxpackage uup://product/Windows.Workload.ExecutionProvider.QNN.arm64 remove-appxpackage uup://product/Windows.Workload.ExecutionProvider.VITISAI.amd64

❯ foundry service init 🟢 Service is Started on http://127.0.0.1:62002/, PID 33852! 🕖 Downloading and registering OpenVINOExecutionProvider [1/2]... An unexpected error occurred. Exception: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. An existing connection was forcibly closed by the remote host.

Created windows feedback

Windows Feedback Hub issue, with the category (press Windows F and select category "Developer Platform > Windows Machine Learning"

[leestott]

Confirming Windows Version
OS Name Microsoft Windows 11 Enterprise
Version 10.0.26200 Build 26200

[leestott]

Fix -FOUND

Run the following command in Windows Powershell Admin Mode

Get-AppxPackage -Name "*OpenVINO*" | Remove-AppxPackage

foundry model list

Models are now shown

Alias                          Device     Task               File Size    License      Model ID
-----------------------------------------------------------------------------------------------
phi-4                          GPU        chat-completion    8.37 GB      MIT          Phi-4-cuda-gpu:1
                               GPU        chat-completion    8.37 GB      MIT          Phi-4-trtrtx-gpu:1
                               GPU        chat-completion    8.37 GB      MIT          Phi-4-generic-gpu:1
                               CPU        chat-completion    10.16 GB     MIT          Phi-4-generic-cpu:1
----------------------------------------------------------------------------------------------------------
phi-3.5-mini                   GPU        chat-completion    2.13 GB      MIT          Phi-3.5-mini-instruct-cuda-gpu:1
                               GPU        chat-completion    2.13 GB      MIT          phi-3.5-mini-instruct-trtrtx-gpu:1
                               GPU        chat-completion    2.16 GB      MIT          Phi-3.5-mini-instruct-generic-gpu:1
                               CPU        chat-completion    2.53 GB      MIT          Phi-3.5-mini-instruct-generic-cpu:1
--------------------------------------------------------------------------------------------------------------------------
phi-3-mini-128k                GPU        chat-completion    2.13 GB      MIT          Phi-3-mini-128k-instruct-cuda-gpu:1
                               GPU        chat-completion    2.13 GB      MIT          phi-3-mini-128k-instruct-trtrtx-gpu:1
                               GPU        chat-completion    2.13 GB      MIT          Phi-3-mini-128k-instruct-generic-gpu:1
                               CPU        chat-completion    2.54 GB      MIT          Phi-3-mini-128k-instruct-generic-cpu:2
-----------------------------------------------------------------------------------------------------------------------------
phi-3-mini-4k                  GPU        chat-completion    2.13 GB      MIT          Phi-3-mini-4k-instruct-cuda-gpu:1
                               GPU        chat-completion    2.13 GB      MIT          phi-3-mini-4k-instruct-trtrtx-gpu:1
                               GPU        chat-completion    2.13 GB      MIT          Phi-3-mini-4k-instruct-generic-gpu:1
                               CPU        chat-completion    2.53 GB      MIT          Phi-3-mini-4k-instruct-generic-cpu:2
---------------------------------------------------------------------------------------------------------------------------
mistral-7b-v0.2                GPU        chat-completion    3.98 GB      apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-cuda-gpu:1
                               GPU        chat-completion    3.98 GB      Apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-trtrtx-gpu:1
                               GPU        chat-completion    4.07 GB      apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-generic-gpu:1
                               CPU        chat-completion    4.07 GB      apache-2.0   mistralai-Mistral-7B-Instruct-v0-2-generic-cpu:2
---------------------------------------------------------------------------------------------------------------------------------------
deepseek-r1-14b                GPU        chat-completion    9.83 GB      MIT          deepseek-r1-distill-qwen-14b-cuda-gpu:3
                               GPU        chat-completion    9.83 GB      MIT          deepseek-r1-distill-qwen-14b-trtrtx-gpu:1
                               GPU        chat-completion    10.27 GB     MIT          deepseek-r1-distill-qwen-14b-generic-gpu:3
                               CPU        chat-completion    11.51 GB     MIT          deepseek-r1-distill-qwen-14b-generic-cpu:3
---------------------------------------------------------------------------------------------------------------------------------
deepseek-r1-7b                 GPU        chat-completion    5.28 GB      MIT          deepseek-r1-distill-qwen-7b-cuda-gpu:3
                               GPU        chat-completion    5.58 GB      MIT          deepseek-r1-distill-qwen-7b-generic-gpu:3
                               CPU        chat-completion    6.43 GB      MIT          deepseek-r1-distill-qwen-7b-generic-cpu:3
                               GPU        chat-completion    5.28 GB      MIT          DeepSeek-R1-Distill-Qwen-7B-trtrtx-gpu:1
-------------------------------------------------------------------------------------------------------------------------------
whisper-base                   GPU        automatic-speech-recognition 0.19 GB      apache-2.0   openai-whisper-base-cuda-gpu:1
                               CPU        automatic-speech-recognition 0.37 GB      apache-2.0   openai-whisper-base-generic-cpu:1
------------------------------------------------------------------------------------------------------------------------
whisper-medium                 GPU        automatic-speech-recognition 1.53 GB      apache-2.0   openai-whisper-medium-cuda-gpu:1
                               CPU        automatic-speech-recognition 3.05 GB      apache-2.0   openai-whisper-medium-generic-cpu:1
--------------------------------------------------------------------------------------------------------------------------
whisper-small                  GPU        automatic-speech-recognition 0.53 GB      apache-2.0   openai-whisper-small-cuda-gpu:1
                               CPU        automatic-speech-recognition 1.05 GB      apache-2.0   openai-whisper-small-generic-cpu:1
-------------------------------------------------------------------------------------------------------------------------
whisper-tiny                   GPU        automatic-speech-recognition 0.11 GB      apache-2.0   openai-whisper-tiny-cuda-gpu:1
                               CPU        automatic-speech-recognition 0.22 GB      apache-2.0   openai-whisper-tiny-generic-cpu:2
------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-0.5b             GPU        chat-completion    0.52 GB      apache-2.0   qwen2.5-coder-0.5b-instruct-cuda-gpu:4
                               GPU        chat-completion    0.52 GB      MIT          qwen2.5-coder-0.5b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    0.52 GB      apache-2.0   qwen2.5-coder-0.5b-instruct-generic-gpu:4
                               CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-coder-0.5b-instruct-generic-cpu:4
--------------------------------------------------------------------------------------------------------------------------------
phi-4-mini-reasoning           GPU        chat-completion    3.15 GB      MIT          Phi-4-mini-reasoning-cuda-gpu:3
                               GPU        chat-completion    3.15 GB      MIT          Phi-4-mini-reasoning-generic-gpu:3
                               CPU        chat-completion    4.52 GB      MIT          Phi-4-mini-reasoning-generic-cpu:3
-------------------------------------------------------------------------------------------------------------------------
qwen2.5-0.5b                   GPU        chat-completion    0.52 GB      apache-2.0   qwen2.5-0.5b-instruct-cuda-gpu:4
                               GPU        chat-completion    0.52 GB      MIT          qwen2.5-0.5b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    0.68 GB      apache-2.0   qwen2.5-0.5b-instruct-generic-gpu:4
                               CPU        chat-completion    0.80 GB      apache-2.0   qwen2.5-0.5b-instruct-generic-cpu:4
--------------------------------------------------------------------------------------------------------------------------
qwen2.5-1.5b                   GPU        chat-completion    1.25 GB      apache-2.0   qwen2.5-1.5b-instruct-cuda-gpu:4
                               GPU        chat-completion    1.25 GB      MIT          qwen2.5-1.5b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    1.51 GB      apache-2.0   qwen2.5-1.5b-instruct-generic-gpu:4
                               CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-1.5b-instruct-generic-cpu:4
--------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-1.5b             GPU        chat-completion    1.25 GB      apache-2.0   qwen2.5-coder-1.5b-instruct-cuda-gpu:4
                               GPU        chat-completion    1.25 GB      MIT          qwen2.5-coder-1.5b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    1.25 GB      apache-2.0   qwen2.5-coder-1.5b-instruct-generic-gpu:4
                               CPU        chat-completion    1.78 GB      apache-2.0   qwen2.5-coder-1.5b-instruct-generic-cpu:4
--------------------------------------------------------------------------------------------------------------------------------
phi-4-mini                     GPU        chat-completion    3.60 GB      MIT          Phi-4-mini-instruct-cuda-gpu:5
                               GPU        chat-completion    3.72 GB      MIT          Phi-4-mini-instruct-generic-gpu:5
                               CPU        chat-completion    4.80 GB      MIT          Phi-4-mini-instruct-generic-cpu:5
------------------------------------------------------------------------------------------------------------------------
qwen2.5-14b                    GPU        chat-completion    8.79 GB      apache-2.0   qwen2.5-14b-instruct-cuda-gpu:4
                               GPU        chat-completion    8.79 GB      MIT          qwen2.5-14b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    9.30 GB      apache-2.0   qwen2.5-14b-instruct-generic-gpu:4
                               CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-14b-instruct-generic-cpu:4
-------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-14b              GPU        chat-completion    8.79 GB      apache-2.0   qwen2.5-coder-14b-instruct-cuda-gpu:4
                               GPU        chat-completion    8.79 GB      MIT          qwen2.5-coder-14b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    8.79 GB      apache-2.0   qwen2.5-coder-14b-instruct-generic-gpu:4
                               CPU        chat-completion    11.06 GB     apache-2.0   qwen2.5-coder-14b-instruct-generic-cpu:4
-------------------------------------------------------------------------------------------------------------------------------
qwen2.5-coder-7b               GPU        chat-completion    4.73 GB      apache-2.0   qwen2.5-coder-7b-instruct-cuda-gpu:4
                               GPU        chat-completion    4.73 GB      MIT          qwen2.5-coder-7b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    4.73 GB      apache-2.0   qwen2.5-coder-7b-instruct-generic-gpu:4
                               CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-coder-7b-instruct-generic-cpu:4
------------------------------------------------------------------------------------------------------------------------------
qwen2.5-7b                     GPU        chat-completion    4.73 GB      apache-2.0   qwen2.5-7b-instruct-cuda-gpu:4
                               GPU        chat-completion    5.49 GB      MIT          qwen2.5-7b-instruct-trtrtx-gpu:2
                               GPU        chat-completion    5.20 GB      apache-2.0   qwen2.5-7b-instruct-generic-gpu:4
                               CPU        chat-completion    6.16 GB      apache-2.0   qwen2.5-7b-instruct-generic-cpu:4
------------------------------------------------------------------------------------------------------------------------
whisper-large-v3-turbo         GPU        automatic-speech-recognition 8.79 GB      apache-2.0   openai-whisper-large-v3-turbo-cuda-gpu:2
                               CPU        automatic-speech-recognition 8.79 GB      apache-2.0   openai-whisper-large-v3-turbo-generic-cpu:2
----------------------------------------------------------------------------------------------------------------------------------
gpt-oss-20b                    CPU        chat-completion    12.26 GB     MIT          gpt-oss-20b-generic-cpu:1
                               GPU        chat-completion    9.65 GB      apache-2.0   gpt-oss-20b-cuda-gpu:1
-------------------------------------------------------------------------------------------------------------
deepseek-r1-1.5b               GPU        chat-completion    1.43 GB      MIT          DeepSeek-R1-Distill-Qwen-1.5B-trtrtx-gpu:1