← All notes

Answer: huggingface_hub gives one unified interface to run inference on Hub-hosted models across multiple services. InferenceClient is a Python client making HTTP calls to these APIs.

Three targets

1. Inference Providers: unified access to hundreds of models through serverless inference partners.
2. Inference Endpoints: dedicated, fully managed deployments on a cloud provider of your choice.
3. Local endpoints: local inference servers like llama.cpp, Ollama, vLLM, LiteLLM, or TGI.

Model and provider ids

With a provider, pass the model's Hugging Face Hub id, not the id the third-party provider uses. The default provider value is "auto", which selects the first available provider sorted by the user's order in settings.

Chat

chat_completion generates a response from a list of messages; the output follows the OpenAI specification.

Auth

HF_TOKEN overrides the token stored on the machine.

Artifacts

• Topic packet: huggingface

See also

• hf-hub-download-files
• hf-transformers-pipeline

Claims

• id: C1 claim: InferenceClient makes HTTP calls to HF APIs. source: S1 evidence: "is a Python client making HTTP calls to our APIs."

• id: C2 claim: Model ids are Hub ids, not provider ids. source: S1 evidence: "The model id must be the id of the model on the Hugging Face Hub"

• id: C3 claim: Local servers like Ollama and vLLM are supported. source: S1 evidence: "llama.cpp, Ollama, vLLM, LiteLLM, or Text Generation Inference (TGI)"

• id: C4 claim: HF_TOKEN overrides the stored token. source: S2 evidence: "overwrite the token stored on the machine"

Sources

• [S1] Run Inference on servers. Hugging Face docs. https://huggingface.co/docs/huggingface_hub/guides/inference . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

• [S2] Environment variables. Hugging Face docs. https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs (huggingface/transformers, huggingface/huggingface_hub, huggingface/diffusers on GitHub), Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0). Text was condensed and changed from the originals.

@misc{iil_note_q42zgqnc,
  author = {{iinventedlight}},
  title = {{InferenceClient: one client for providers, endpoints, local servers}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-09},
  url = {https://iinventedlight.com/library/notes/hf-hub-inference-client/},
  note = {Permanent ID iil:note:q42zgqnc. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/hf-hub-inference-client.md.