Hugging Face
Bundles contain custom components that support specific third-party integrations with Langflow.
The components in the Hugging Face bundle require access to Hugging Face APIs.
For more information about Hugging Face features and functionality used by Hugging Face components, see the Hugging Face documentation.
Hugging Face text generation
The Hugging Face component generates text using a specified model by sending requests to the Hugging Face API, which is a hosted inference API for models hosted on Hugging Face. Authentication is required.
This component can output either a Model Response (Message) or a Language Model (LanguageModel).
Specifically, the Language Model output is an instance of HuggingFaceHub configured according to the component's parameters.
Use the Language Model output when you want to use a Hugging Face model as the LLM for another LLM-driven component, such as a Language Model or Smart Function component.
For more information, see Language Model components.
Hugging Face text generation parameters
Many Hugging Face component input parameters are hidden by default in the visual editor. You can toggle parameters through the Controls in the component's header menu.
| Name | Type | Description |
|---|---|---|
| model_id | String | Input parameter. The model ID from Hugging Face Hub. For example, "gpt2", "facebook/bart-large". |
| huggingfacehub_api_token | SecretString | Input parameter. Your Hugging Face API token for authentication. |
| temperature | Float | Input parameter. Controls randomness in the output. Range: [0.0, 1.0]. Default: 0.7. |
| max_new_tokens | Integer | Input parameter. Maximum number of tokens to generate. Default: 512. |
| top_p | Float | Input parameter. Nucleus sampling parameter. Range: [0.0, 1.0]. Default: 0.95. |
| top_k | Integer | Input parameter. Top-k sampling parameter. Default: 50. |
| model_kwargs | Dictionary | Input parameter. Additional keyword arguments to pass to the model. |
Hugging Face Embeddings Inference
Use the Hugging Face Embeddings Inference component to create embeddings with Hugging Face's hosted models or with your own locally hosted models.
The component generates embeddings using Hugging Face Inference API models. Authentication is required when not using a local model.
For more information about using embedding model components in flows, see Embedding Model components and Use a local Hugging Face embeddings model.
Hugging Face Embeddings Inference parameters
| Name | Display Name | Info |
|---|---|---|
| API Key | API Key | Input parameter. Your Hugging Face API token for accessing the Hugging Face Inference API, if required. Local inference models do not require an API key. |
| API URL | API URL | Input parameter. The URL of the Hugging Face Inference API. |
| Model Name | Model Name | Input parameter. The name of the model to use for embeddings. |
Use a local Hugging Face embeddings model
To connect the local Hugging Face model to the Hugging Face Embeddings Inference component and use it in a flow, follow these steps:
-
For this example, create a flow from the Vector Store RAG template.
-
Replace the two OpenAI Embeddings components with Hugging Face Embeddings Inference components.
Make sure to reconnect the Embedding Model ports from each embedding model component to its corresponding Astra DB vector store component.
-
Configure the Astra DB vector store components to connect to your Astra organization, or replace both Astra DB vector store components with other vector store components.
-
Connect each Hugging Face Embeddings Inference component to your local inference model:
- Inference Endpoint: Enter the URL of your local inference model.
- API Key: Can be empty for local inference.
- Model Name: Enter the name of your local inference model if it isn't automatically detected.
-
To test the flow, click Playground, and then enter some text to generate an embedding.
