---
source: https://iinventedlight.com/library/notes/hf-hub-inference-client.md
title: "InferenceClient: one client for providers, endpoints, local servers"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "Apache-2.0 (huggingface/transformers, huggingface_hub, diffusers docs; LICENSE verified in each repo); prose restates cited source lines in own words"
license_family: hf-apache
attribution: "Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs, Run Inference on servers, Apache-2.0 (huggingface docs repos); Hugging Face docs, Environment variables, Apache-2.0 (huggingface docs repos)."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-09
status: published
checked_on: 2026-11-09
as_of: 2026-11-09
confidence: medium
untested: "No code was run for this note; only the fetched documentation pages were read."
category: coding-references
topic_area: "code-ai-eng: Hugging Face"
tags: [hugging-face, huggingface-hub, inference, inference-providers]
client: Librarian
sources:
  - id: S1
    title: "Run Inference on servers"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/huggingface_hub/guides/inference
    accessed: 2026-11-09
    license: "Apache-2.0 (huggingface docs repos)"
  - id: S2
    title: "Environment variables"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables
    accessed: 2026-11-09
    license: "Apache-2.0 (huggingface docs repos)"
---
# InferenceClient: one client for providers, endpoints, local servers

**Answer:** `huggingface_hub` gives one unified interface to run inference on Hub-hosted models across multiple services. `InferenceClient` is a Python client making HTTP calls to these APIs.

## Three targets

1. Inference Providers: unified access to hundreds of models through serverless inference partners.
2. Inference Endpoints: dedicated, fully managed deployments on a cloud provider of your choice.
3. Local endpoints: local inference servers like llama.cpp, Ollama, vLLM, LiteLLM, or TGI.

## Model and provider ids

With a provider, pass the model's Hugging Face Hub id, not the id the third-party provider uses. The default provider value is "auto", which selects the first available provider sorted by the user's order in settings.

## Chat

`chat_completion` generates a response from a list of messages; the output follows the OpenAI specification.

## Auth

`HF_TOKEN` overrides the token stored on the machine.

## Artifacts

- [Topic packet: huggingface](https://iinventedlight.com/library/packets/huggingface.json)

## See also

- [hf-hub-download-files](https://iinventedlight.com/library/notes/hf-hub-download-files/)
- [hf-transformers-pipeline](https://iinventedlight.com/library/notes/hf-transformers-pipeline/)

## Claims

- id: C1
  claim: InferenceClient makes HTTP calls to HF APIs.
  source: S1
  evidence: "is a Python client making HTTP calls to our APIs."

- id: C2
  claim: Model ids are Hub ids, not provider ids.
  source: S1
  evidence: "The model id must be the id of the model on the Hugging Face Hub"

- id: C3
  claim: Local servers like Ollama and vLLM are supported.
  source: S1
  evidence: "llama.cpp, Ollama, vLLM, LiteLLM, or Text Generation Inference (TGI)"

- id: C4
  claim: HF_TOKEN overrides the stored token.
  source: S2
  evidence: "overwrite the token stored on the machine"

## Sources

- [S1] Run Inference on servers. Hugging Face docs. https://huggingface.co/docs/huggingface_hub/guides/inference . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

- [S2] Environment variables. Hugging Face docs. https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.


License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs (huggingface/transformers, huggingface/huggingface_hub, huggingface/diffusers on GitHub), Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0). Text was condensed and changed from the originals.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).