Build an LLM App with Tool Use using Llama 3.1 Running Locally

Cutting-edge AI apps don’t always require the cloud. Today we are combining local Llama 3.1 with tool-use, making it possible to interact with real-world data sources while the AI itself runs locally.

In this tutorial, we’ll guide you through building a local Llama 3.1-powered assistant that integrates tools like Yahoo Finance for stock data and SerpAPI for web searches. While the assistant itself operates locally, it will access real-time data through these external APIs.

🎁 $50 worth AI Bonus Content at the end!

What We’re Building

This Streamlit app demonstrates tool-use with the local Llama3 model using Ollama. The AI Assistant uses the tool that the user selects.

This assistant will:

• Use Streamlit for an easy-to-use chat interface

• Use Phidata for AI assistant and tools

• Use the local Llama 3.1 model via Ollama as the model

• Provide dynamic tool selection for YFinance (for stock data) and SerpAPI (for web searches).

• Display which tools are currently enabled in a sidebar.

Users can select which tools (YFinance and/or SerpAPI) they want the assistant to use via checkboxes in the sidebar. The app initializes or updates the assistant based on the selected tools. Users can ask questions through a chat input, and the assistant responds using the enabled tools.

Prerequisites

Before we begin, make sure you have:

1. Python installed on your machine (version 3.7 or higher is recommended)

2. Basic familiarity with Python programming

3. Your SerpAPI API key (sign up here).

4. A code editor of your choice (we recommend VSCode or PyCharm for their excellent Python support)

Step-by-Step Instructions

Step 1: Setting Up the Environment

First, let's get our development environment ready:

1. Clone the GitHub repository:

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git

🌟 Don't forget to star the opensource repo to show your support.

2. Go to the local_llama3.1_tool_use folder:

cd local-llama3-tool-use

3. Install the required dependencies:

pip install -r requirements.txt

4. Set up your SerpAPI API key:

export SERPAPI_API_KEY=your_api_key_here

Step 2: Creating the Streamlit App

Now that the environment is set, let’s create our Streamlit app. Create a new file llama3_tool_use.py and add the following code:

• Import Required Libraries: At the top of your file, add

  • Streamlit for the web interface
  • Phidata for AI assistant and tools
  • Ollama for local LLM integration

import streamlit as st
import os
from phi.assistant import Assistant
from phi.llm.ollama import Ollama
from phi.tools.yfinance import YFinanceTools
from phi.tools.serpapi_tools import SerpApiTools

• Set up the Streamlit app and check for API key: It will configure the Streamlit page and ensure the SERPAPI_API_KEY is set

st.set_page_config(page_title="Llama-3 Tool Use", page_icon="🦙")

# Ensure SERPAPI_API_KEY is set
if 'SERPAPI_API_KEY' not in os.environ:
    st.error("Please set the SERPAPI_API_KEY environment variable.")
    st.stop()

• Define the assistant creation function: This will create an Assistant with Llama-3.1 model and configure tools and settings for the assistant

def get_assistant(tools):
    return Assistant(
        name="llama3_assistant",
        llm=Ollama(model="llama3"),
        tools=tools,
        description="You are a helpful assistant that can access specific tools based on user selection.",
        show_tool_calls=True,
        debug_mode=True,
        # This setting adds the current datetime to the instructions
        add_datetime_to_instructions=True,

    )

• Set Up the Streamlit App: Streamlit lets you create user interface with just python code. For this app we will:
  • Adds a title and description to the app
  • Provides instructions for users

st.title("🦙 Local Llama-3 Tool Use")
st.markdown("""
This app demonstrates function calling with the local Llama3 model using Ollama.
Select tools in the sidebar and ask relevant questions!
""")

• Create a sidebar for tool selection: Users can select tools via checkboxes and this will initialize tools based on user selection.

st.sidebar.title("Tool Selection")
use_yfinance = st.sidebar.checkbox("YFinance (Stock Data)", value=True)
use_serpapi = st.sidebar.checkbox("SerpAPI (Web Search)", value=True)

# Initialize or update the assistant based on selected tools
tools = []
if use_yfinance:
    tools.append(YFinanceTools(stock_price=True, company_info=True))
if use_serpapi:
    tools.append(SerpApiTools())

• Initialize or update the assistant: This will create or update the assistant when tools change, and reset the chat history on tool change.

if "assistant" not in st.session_state or st.session_state.get("tools") != tools:
    st.session_state.assistant = get_assistant(tools)
    st.session_state.tools = tools
    st.session_state.messages = []

• Display current tool status and chat interface: This will show the enabled/ disabled status of tools and display the chat history.

st.sidebar.markdown("### Current Tools:")
st.sidebar.markdown(f"- YFinance: {'Enabled' if use_yfinance else 'Disabled'}")
st.sidebar.markdown(f"- SerpAPI: {'Enabled' if use_serpapi else 'Disabled'}")

# Chat interface
for message in st.session_state.get("messages", []):
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

• Implement chat input and response generation: This will
  • Capture user input
  • Generate and display the assistant's response
  • Update chat history

if prompt := st.chat_input("Ask a question based on the enabled tools"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        response_container = st.empty()
        response = ""
        for chunk in st.session_state.assistant.run(prompt):
            response += chunk
            response_container.write(response + "▌")
        response_container.write(response)
    st.session_state.messages.append({"role": "assistant", "content": response})

Sidebar instructions:

st.sidebar.markdown("""
### How to use:
1. Select the tools you want to use in the sidebar
2. Ask questions related to the enabled tools
3. The assistant will use only the selected tools to answe
### Note:
Make sure you have set the SERPAPI_API_KEY environment variable to use the SerpAPI tool.
""")

st.sidebar.markdown("""
### Sample questions:
- YFinance: "What's the current price of AAPL?"
- SerpAPI: "What are the latest developments in AI?"
- Both: "Compare TSLA stock price with recent news about Tesla's performance"
""")

Step 3: Running the App

With our code in place, it's time to launch the app.

• Start the Streamlit App: In your terminal, navigate to the project folder, and run the following command

streamlit run llama3_tool_use.py

• Access Your AI Assistant: Streamlit will provide a local URL (typically http://localhost:8501). Open this in your web browser, select the tools you want the assistant to use, and have fun!

Working Application Demo

Conclusion

Great work! You’ve built a powerful AI assistant using Llama 3 locally, integrating real-time tools like YFinance and SerpAPI. This setup can be the foundation for a wide range of applications.

For the next steps, you could enhance the app by incorporating more data sources, such as APIs for cryptocurrency or sports statistics. Alternatively, explore adding voice interface capabilities to your app, allowing users to interact with the assistant using speech.

We share hands-on tutorials like this 2-3 times a week, to help you stay ahead in the world of AI. If you're serious about levelling up your AI skills and staying ahead of the curve, subscribe now and be the first to access our latest tutorials.

Bonus worth $50 💵💰

Share this newsletter on your social channels and tag Unwind AI (X, LinkedIn, Threads, Facebook) to get AI resource pack worth $50 for FREE. Valid for limited time only!