Build an AI Lead Generation Agent

Sales teams spend countless hours manually searching for and qualifying potential leads. This repetitive task not only consumes time but also results in inconsistent lead quality. Let’s automate this process to help sales teams focus on what matters most - building relationships and closing deals.

In this tutorial, we'll build an AI Lead Generation Agent that automatically discovers and qualifies potential leads from Quora. Using Firecrawl for intelligent web scraping, Phidata for agent orchestration, and Composio for Google Sheets integration, you'll create a system that can continuously generate and organize qualified leads with minimal human intervention.

Our lead generation agent will help sales teams identify potential customers who are actively discussing or seeking solutions in their target market.

Don’t forget to share this tutorial on your social channels and tag Unwind AI (X, LinkedIn, Threads, Facebook) to support us!

What We’re Building

This implementation creates an automated lead generation system that searches Quora for potential leads and organizes them in Google Sheets.

Features:

• 🎯 Targeted Search - Uses Firecrawl's search endpoint to find relevant Quora URLs based on your search criteria

• 💡 Intelligent Extraction - Leverages Firecrawl's LLM extract functionality to pull user information from Quora profiles

• ⚙️ Automated Processing - Formats extracted user information into a clean, structured format

• 💻 Google Sheets Integration - Automatically creates and populates Google Sheets with lead information

• ✍️ Customizable Criteria - Allows you to define specific search parameters to find your ideal leads for your niche

Tech Stack

Firecrawl serves as our intelligent web scraping layer, providing advanced capabilities to extract structured data from Quora. It handles the complexities of web scraping, including JavaScript rendering and anti-bot mechanisms, while converting raw HTML into clean, structured data ready for our AI processing pipeline.

Phidata powers our agent architecture, enabling us to create and orchestrate multiple AI agents that work together. It provides the framework for building our lead generation agent with built-in memory, knowledge integration, and reasoning capabilities, making it perfect for complex workflows like lead qualification.

Composio handles our external tool integration, particularly connecting our agent with Google Sheets. It manages authentication, token handling, and secure connections to external services, allowing our agent to seamlessly write qualified leads to spreadsheets without dealing with complex API implementations.

For our language model, we're using OpenAI’s GPT-4o, which provides the intelligence needed to analyze and qualify leads based on extracted data.

Prerequisites

Before we begin, make sure you have the following:

1. Python installed on your machine (version 3.10 or higher is recommended)

2. Your OpenAI, Firecrawl, and Composio API key

3. A code editor of your choice (we recommend VS Code or PyCharm for their excellent Python support)

4. Basic familiarity with Python programming

Step-by-Step Instructions

Setting Up the Environment

First, let's get our development environment ready:

1. Clone the GitHub repository:

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git

🌟 Don't forget to star the opensource repo to show your support.

2. Go to the ai_lead_generation_agent folder:

cd ai_agent_tutorials/ai_lead_generation_agent

3. Install the required dependencies:

pip install -r requirements.txt

4. Important things to do in Composio:

   • in the terminal, run this command: composio add googlesheets

   • In your Composio dashboard, create a new Google sheet intergation and make sure it is active in the active integrations/connections tab

5. Obtain your API keys:

   • Get your Firecrawl API key from Firecrawl's website

   • Get your Composio API key from Composio's website

   • Get your OpenAI API key from OpenAI's website

Creating the Streamlit App

Let’s create our app. Create a new file ai_lead_generation_agent.py and add the following code:

1. Let's set up our imports and define schemas:

import streamlit as st
import requests
from phi.agent import Agent
from phi.tools.firecrawl import FirecrawlTools
from phi.model.openai import OpenAIChat
from firecrawl import FirecrawlApp
from pydantic import BaseModel, Field
from typing import List
from composio_phidata import Action, ComposioToolSet

class QuoraUserInteractionSchema(BaseModel):
    username: str = Field(description="Username of the poster")
    bio: str = Field(description="User bio")
    post_type: str = Field(description="Type: question or answer")
    timestamp: str = Field(description="Post timestamp")

2. Create URL search function:

def search_for_urls(company_description, firecrawl_api_key, num_links):
    url = "https://api.firecrawl.dev/v1/search"
    headers = {
        "Authorization": f"Bearer {firecrawl_api_key}",
        "Content-Type": "application/json"
    }
    payload = {
        "query": f"quora websites where people are looking for {company_description}",
        "limit": num_links,
        "lang": "en"
    }
    response = requests.post(url, json=payload, headers=headers)

3. Implement user info extraction:

def extract_user_info_from_urls(urls, firecrawl_api_key):
    user_info_list = []
    firecrawl_app = FirecrawlApp(api_key=firecrawl_api_key)
    
    for website_url in urls:
        data = firecrawl_app.scrape_url(website_url, {
            'formats': ['extract'],
            'extract': {
                'schema': QuoraPageSchema.model_json_schema(),
            }
        })

4. Create data formatting function:

def format_user_info_to_flattened_json(user_info_list):
    flattened_data = []
    for info in user_info_list:
        website_url = info["website_url"]
        for interaction in info["user_info"]:
            flattened_interaction = {
                "Website URL": website_url,
                "Username": interaction["username"],
                "Bio": interaction["bio"]
                # ... other fields
            }
            flattened_data.append(flattened_interaction)

5. Set up Google Sheets agent:

def create_google_sheets_agent(composio_api_key, openai_api_key):
    composio_toolset = ComposioToolSet(api_key=composio_api_key)
    google_sheets_tool = composio_toolset.get_tools(
        actions=[Action.GOOGLESHEETS_SHEET_FROM_JSON]
    )[0]
    
    google_sheets_agent = Agent(
        model=OpenAIChat(id="gpt-4o-mini"),
        tools=[google_sheets_tool],
        system_prompt="Create and update Google Sheets with user information."
    )

6. Implement sheet writing function:

def write_to_google_sheets(flattened_data, composio_api_key, openai_api_key):
    google_sheets_agent = create_google_sheets_agent(
        composio_api_key, 
        openai_api_key
    )
    
    create_sheet_response = google_sheets_agent.run(
        f"Create a new Google Sheet:\n"
        f"Title: Quora User Info\n"
        f"Data: {flattened_data}"
    )

7. Create the Gameplay Agent:

def main():
    st.title("🎯 AI Lead Generation Agent")
    
    with st.sidebar:
        st.header("API Keys")
        firecrawl_api_key = st.text_input("Firecrawl API Key", type="password")
        openai_api_key = st.text_input("OpenAI API Key", type="password")
        composio_api_key = st.text_input("Composio API Key", type="password")

8. Add user inputs:

num_links = st.number_input(
    "Number of links to search", 
    min_value=1, 
    max_value=10, 
    value=3
)

company_description = st.text_input(
    "Enter your company description:",
    placeholder="e.g. AI voice cloning"
)

9. Implement lead generation workflow:

if st.button("Generate Leads"):
    with st.spinner("Searching URLs..."):
        urls = search_for_urls(
            company_description, 
            firecrawl_api_key, 
            num_links
        )
        
    with st.spinner("Extracting info..."):
        user_info = extract_user_info_from_urls(
            urls, 
            firecrawl_api_key
        )

10. Add display logic:

if urls:
    st.subheader("Quora Links Used:")
    for url in urls:
        st.write(url)
        
    if google_sheets_link:
        st.success("Lead generation completed!")
        st.markdown(f"[View Sheet]({google_sheets_link})")

11. Error handling:

if not all([firecrawl_api_key, openai_api_key, composio_api_key]):
    st.error("Please fill in all API keys.")
    
try:
    # Process workflow
except Exception as e:
    st.error(f"Error: {str(e)}")

12. Status tracking:

with st.spinner("Processing..."):
    # Show progress
    st.write("Step 1/4: Searching URLs")
    st.write("Step 2/4: Extracting info")
    st.write("Step 3/4: Formatting data")
    st.write("Step 4/4: Creating sheet")

Running the App

With our code in place, it's time to launch the app.

• In your terminal, navigate to the project folder, and run the following command

streamlit run ai_lead_generation_agent.py

• Streamlit will provide a local URL (typically http://localhost:8501).

Working Application Demo

Conclusion

You've now built a sophisticated AI Game Design Agent Team that can generate comprehensive game concepts through multi-agent collaboration. Want to make it even better? Here are some fun ideas:

• Add image generation capabilities for concept art

• Implement save/load functionality for game concepts

• Add version control for iterative concept development

• Expand the system to generate design documents

• Integrate with project management tools

Keep experimenting and have fun watching these AI agents work together as a team to develop your game plan!

We share hands-on tutorials like this 2-3 times a week, to help you stay ahead in the world of AI. If you're serious about leveling up your AI skills and staying ahead of the curve, subscribe now and be the first to access our latest tutorials.

Don’t forget to share this tutorial on your social channels and tag Unwind AI (X, LinkedIn, Threads, Facebook) to support us!