Build an AI RAG Agent with Web Access using GPT-4o

Today we’re building a smart AI agent that not only retrieves answers from PDFs but also searches the web in real time—all with minimal code. In this tutorial, we’ll walk through how to create a Retrieval-Augmented Generation (RAG) agent that uses GPT-4o for intelligent querying. Your agent will tap into a PDF-based knowledge base and perform web searches using DuckDuckGo, providing rich insights through a sleek playground interface.

Using Phidata, a framework designed for building agent-based systems, we’ll streamline the entire setup. You’ll combine tools like LanceDB for vector-based searches, PDF knowledge embedding, and interactive browsing. The result? A powerful AI assistant ready to handle complex queries with ease.

🎁 $50 worth AI Bonus Content at the end!

What We’re Building

This script will demonstrate how to build a RAG agent with web access using GPT-4o in just 15 lines of Python code. The agent uses a PDF knowledge base and has the ability to search the web using DuckDuckGo.

Features

• Creates a RAG agent using GPT-4o

• Incorporates a PDF-based knowledge base

• Uses LanceDB as the vector database for efficient similarity search

• Includes web search capability through DuckDuckGo

• Provides a playground interface for easy interaction

Prerequisites

Before we begin, make sure you have:

1. Python installed on your machine (version 3.7 or higher is recommended)

2. Your OpenAI API Key

3. Basic familiarity with Python programming

4. A code editor of your choice (we recommend VS Code or PyCharm for their excellent Python support)

Step-by-Step Instructions

Setting Up the Environment

First, let's get our development environment ready:

1. Clone the GitHub repository:

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git

🌟 Don't forget to star the opensource repo to show your support.

2. Go to the agentic_rag folder:

cd agentic_rag

3. Install the required dependencies:

pip install -r requirements.txt

3. Get your OpenAI API Key: Sign up for an OpenAI account (or the LLM provider of your choice) and obtain your API key. Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY='your-api-key-here'

Creating the App

Let’s create our app. Create a new file rag_agent.py and add the following code:

1. Import Required Libraries: 
   • Phidata for agent creation, knowledge base, and vector database

   • OpenAI chat model integration

   • Playground for interactive demo

   • DuckDuckGo tool for web searches

   • Export your OpenAI API key to use GPT-4o

from phi.agent import Agent
from phi.model.openai import OpenAIChat
from phi.knowledge.pdf import PDFUrlKnowledgeBase
from phi.vectordb.lancedb import LanceDb, SearchType
from phi.playground import Playground, serve_playground_app
from phi.tools.duckduckgo import DuckDuckGo

2. Let's set up our knowledge base using a PDF:

   • Creates a knowledge base from a PDF URL (Can use multiple URLs)

   • Uses LanceDB as the vector database

   • Loads the knowledge base

db_uri = "tmp/lancedb"
# Create a knowledge base from a PDF
knowledge_base = PDFUrlKnowledgeBase(
    urls=["https://phi-public.s3.amazonaws.com/recipes/ThaiRecipes.pdf"],
    # Use LanceDB as the vector database
    vector_db=LanceDb(table_name="recipes", uri=db_uri, search_type=SearchType.vector),
)
# Load the knowledge base: Comment out after first run
knowledge_base.load(upsert=True)

3. Setup the RAG Agent with tools and knowledge base:

   • Uses OpenAI's GPT-4o as the LLM

   • Incorporates DuckDuckGo as a tool

   • Integrates the PDF knowledge base

   • Enables markdown output and tool call visibility

rag_agent = Agent(
    model=OpenAIChat(id="gpt-4o"),
    agent_id="rag-agent",
    knowledge=knowledge_base, # Add the knowledge base to the agent
    tools=[DuckDuckGo()],
    show_tool_calls=True,
    markdown=True,
)

4. Set up the Playground for interactive demo:

   • Creates a Playground instance with our RAG agent

   • Generates an interactive app for testing

app = Playground(agents=[rag_agent]).get_app()

5. Finally, we serve the Playground app:

   • Serves the Playground app

   • Enables hot reloading for development

if __name__ == "__main__":
    serve_playground_app("rag_agent:app", reload=True)

Running the App

With our code in place, it's time to launch the app.

• In your terminal, navigate to the project folder, and run the following command

python3 rag_agent.py

• Open your web browser and navigate to the URL provided in the console output to interact with the RAG agent through the playground interface.

Working Application Demo

Conclusion

And just like that, you’ve built a RAG agent with web search capabilities in only 15 lines of code! With GPT-4o as the brain behind the agent and Phidata doing the heavy lifting, your agent can respond intelligently to queries and explore academic or general web data.

For the next steps, consider integrating other APIs for specialized queries. You could also explore expanding the playground interface to handle multi-agent collaboration or additional tools from Phidata’s extensive library.

Keep experimenting and refining to build even smarter AI solutions!

We share hands-on tutorials like this 2-3 times a week, to help you stay ahead in the world of AI. If you're serious about levelling up your AI skills and staying ahead of the curve, subscribe now and be the first to access our latest tutorials.

Bonus worth $50 💵💰

Share this newsletter on your social channels and tag Unwind AI (X, LinkedIn, Threads, Facebook) to get AI resource pack worth $50 for FREE. Valid for limited time only!