Build an AI 3D PyGame Visualizer with DeepSeek R1

Turning natural language descriptions into working PyGame visualizations typically requires significant coding expertise and time. Let's build a system that automates the entire process using AI.

In this tutorial, we'll create an AI multi-agent PyGame generator that converts text descriptions into fully functional visualizations. It coordinates four specialized AI agents working together to bring your PyGame ideas to life.

Each agent has a specific role: the Navigator agent handles browser navigation to Trinket.io, the Coder agent manages code input, the Executor agent runs the code, and the Viewer agent monitors the visualization. This division of labor creates a smooth, automated pipeline from concept to working visualization.

The system combines several powerful tools:

• DeepSeek R1 provides the reasoning and code generation capabilities needed to turn natural language descriptions into working PyGame code.

• OpenAI GPT-4o handles the clean extraction of executable code from R1's output.

• Agno, an open-source framework for building agentic systems, enables us to create and coordinate our team of specialized agents with features like memory management and tool integration.

• Browser Use rounds out our toolkit by providing seamless browser automation, allowing our agents to interact with Trinket.io for code visualization.

Don’t forget to share this tutorial on your social channels and tag Unwind AI (X, LinkedIn, Threads, Facebook) to support us!

What We’re Building

This application implements a sophisticated multi-agent system that generates PyGame code from natural language descriptions and automatically visualizes it in the browser.

Features:

• Natural language to PyGame code conversion using DeepSeek R1's reasoning capabilities

• Code extraction and refinement through GPT-4

• Team of four specialized agents working in concert:

  1. Navigator Agent - Handles browser navigation to Trinket.io

  2. Coder Agent - Manages code input and editing

  3. Executor Agent - Runs the code in the browser

  4. Viewer Agent - Monitors visualization output

• Automated code visualization on Trinket.io through Browser Use

• Clean, intuitive Streamlit interface for user interaction

• Real-time visualization feedback

Prerequisites

Before we begin, make sure you have the following:

1. Python installed on your machine (version 3.10 or higher is recommended)

2. Your DeepSeek and OpenAI API key

3. A code editor of your choice (we recommend VS Code or PyCharm for their excellent Python support)

4. Basic familiarity with Python programming

Step-by-Step Instructions

Setting Up the Environment

First, let's get our development environment ready:

1. Clone the GitHub repository:

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git

🌟 Don't forget to star the opensource repo to show your support.

2. Go to the ai_3dpygame_r1 folder:

cd ai_agent_tutorials/ai_3dpygame_r1

3. Install the required dependencies:

pip install -r requirements.txt

4. Obtain your DeepSeek and OpenAI API keys.

Creating the Streamlit App

Let’s create our app. Create a new file ai_3dpygame_r1.py and add the following code:

1. Let's set up our imports:

import streamlit as st
from openai import OpenAI
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat as AgnoOpenAIChat
from langchain_openai import ChatOpenAI 
import asyncio
from browser_use import Browser

2. Set up the Streamlit interface and initialize session state:

st.set_page_config(page_title="PyGame Code Generator", layout="wide")

if "api_keys" not in st.session_state:
    st.session_state.api_keys = {
        "deepseek": "",
        "openai": ""
    }

3. Create the sidebar for API configuration:

with st.sidebar:
    st.title("API Keys Configuration")
    st.session_state.api_keys["deepseek"] = st.text_input(
        "DeepSeek API Key",
        type="password",
        value=st.session_state.api_keys["deepseek"]
    )
    st.session_state.api_keys["openai"] = st.text_input(
        "OpenAI API Key",
        type="password",
        value=st.session_state.api_keys["openai"]
    )

4. Set up the main UI with an example query:

st.title("🎮 AI 3D Visualizer with DeepSeek R1")
example_query = "Create a particle system simulation where 100 particles emit from the mouse position..."
query = st.text_area(
    "Enter your PyGame query:",
    height=70,
    placeholder=f"e.g.: {example_query}"
)

col1, col2 = st.columns(2)
generate_code_btn = col1.button("Generate Code")
generate_vis_btn = col2.button("Generate Visualization")

5. Implement code generation with DeepSeek:

if generate_code_btn and query:
    deepseek_client = OpenAI(
        api_key=st.session_state.api_keys["deepseek"],
        base_url="https://api.deepseek.com"
    )

    system_prompt = """You are a Pygame and Python Expert that specializes in making games..."""

    deepseek_response = deepseek_client.chat.completions.create(
        model="deepseek-reasoner",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": query}
        ],
        max_tokens=1  
    )

6. Extract code using OpenAI:

openai_agent = AgnoAgent(
        model=AgnoOpenAIChat(
            id="gpt-4o",
            api_key=st.session_state.api_keys["openai"]
        ),
        show_tool_calls=True,
        markdown=True
    )

    extraction_prompt = f"""Extract ONLY the Python code from the following content..."""
    code_response = openai_agent.run(extraction_prompt)
    extracted_code = code_response.content

7. Implement the browser automation agents:

async def run_pygame_on_trinket(code: str) -> None:
    browser = Browser()
    async with await browser.new_context() as context:
        model = ChatOpenAI(
            model="gpt-4o", 
            api_key=st.session_state.api_keys["openai"]
        )
        
        agent1 = Agent(
            task='Go to https://trinket.io/features/pygame...',
            llm=model,
            browser_context=context,
        )
        
        executor = Agent(
            task='Executor. Execute the code...',
            llm=model,
            browser_context=context
        )

8. Add code viewing and running agents:

coder = Agent(
            task='Coder. Your job is to wait for the user...',
            llm=model,
            browser_context=context
        )
        
        viewer = Agent(
            task='Viewer. Your job is to just view...',
            llm=model,
            browser_context=context,
        )

9. Implement visualization generation:

elif generate_vis_btn:
    if "generated_code" not in st.session_state:
        st.warning("Please generate code first before visualization")
    else:
        async def run_pygame_on_trinket(code: str) -> None:
            # ... (browser automation code)
            with st.spinner("Running code on Trinket..."):
                try:
                    await agent1.run()
                    await coder.run()
                    await executor.run()
                    await viewer.run()
                    st.success("Code is running on Trinket!")
                except Exception as e:
                    st.error(f"Error running code on Trinket: {str(e)}")

Running the App

With our code in place, it's time to launch the app.

• In your terminal, navigate to the project folder, and run the following command

streamlit run ai_3dpygame_r1.py

• Streamlit will provide a local URL (typically http://localhost:8501).

Working Application Demo

Conclusion

You've built a multi-agent AI system that transforms natural language descriptions into working PyGame visualizations. What makes this system powerful is the collaboration between these specialized AI agents.

Here are some potential enhancements to consider:

• Implementing a feedback loop where the Viewer agent can suggest improvements to the generated code.

• Creating a code history feature to save and revisit previous visualizations.

• Adding support for more complex PyGame features like 3D rendering and physics simulations.

• Adding voice input capabilities for describing desired visualizations.

Keep experimenting with different agent configurations and specialized roles to build even more sophisticated AI systems.

We share hands-on tutorials like this 2-3 times a week, to help you stay ahead in the world of AI. If you're serious about leveling up your AI skills and staying ahead of the curve, subscribe now and be the first to access our latest tutorials.

Don’t forget to share this tutorial on your social channels and tag Unwind AI (X, LinkedIn, Threads, Facebook) to support us!