Build ChatGPT Clone with Local Llama 3.1 in Python

Are you fascinated by AI chatbots but put off by the cost and privacy concerns of commercial solutions?

What if I told you that you could build your own ChatGPT-like application that runs entirely on your local machine, for free? 🤯 

In this tutorial, I’ll walk you through the process of creating a ChatGPT clone by just using Llama-3.1 model, LM Studio, Streamlit and Python.

What We're Building?

We'll create a web-based chat interface that uses the Llama-3.1 language model to generate responses using Streamlit. The best part? It runs locally on your computer without needing an internet connection, and it's completely free to use!

Prerequisites

Before we begin, make sure you have:

1. Python installed on your machine (version 3.7 or higher recommended)

2. Basic familiarity with Python programming

3. A code editor of your choice (we recommend VSCode or PyCharm for their excellent Python support)

Code Walkthrough: Step-by-Step Instructions

Step-1: Setting Up the Environment

First, let's get our development environment ready:

1. Clone the GitHub repository:

git clone https://github.com/Shubhamsaboo/awesome-llm-apps.git

🌟 Don't forget to star the opensource repo to show your support.

2. Install the required dependencies:

pip install -r requirements.txt

3. Download and install the LM Studio desktop app.

4. Using LM Studio, download the Llama-3.1 8B instruct model.

5. Start the server in LM Studio to expose the Llama-3.1 8B instruct model as an OpenAI API. Video tutorial to do that here 👇️ 

Step 2: Creating the Streamlit App

Now, let's create our Streamlit app. Create a new file named chatgpt_clone_llama3.py and add the following code:

• Import Required Libraries At the top of your file, add:

import streamlit as st
from openai import OpenAI

• Set Up the Streamlit App Add the following code to create the basic structure of your app:

st.title("ChatGPT Clone using Llama-3 ")
st.caption("Chat with locally hosted Llama-3 using the LM Studio 💯")

• Initialize OpenAI Client Connect to your local Llama-3.1 server. This points to the local server set up using LM Studio.

client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

• Initialize Chat History Implement chat history management.

if "messages" not in st.session_state:
    st.session_state.messages = []

• Display Chat History Add code to display existing chat messages. This displays the chat history using Streamlit's chat_message() and markdown() functions.

for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

• Create User Input Field Add an input field for user messages. This accepts user input using st.chat_input(), adds the user message to the chat history, and displays it in a chat message container.

if prompt := st.chat_input("What is up?"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

• Generate LLM Response Implement the LLM response generation function. This generates a response using the Llama-3 model with client.chat.completions.create().

response = client.chat.completions.create(
    model="lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF",
    messages=st.session_state.messages, 
    temperature=0.7
)

• Display AI Response Add the assistant's response to the chat history and display it. This adds the assistant response to the chat history and displays it in a chat message container.

st.session_state.messages.append({"role": "assistant", "content": response.choices[0].message.content})
with st.chat_message("assistant"):
    st.markdown(response.choices[0].message.content)

Step 3: Running the App

With our code in place, it's time to see our AI assistant in action:

• Start the Streamlit App In your terminal, navigate to the project directory and run.

streamlit run chatgpt_clone_llama3.py

• Access Your AI Assistant Streamlit will provide a local URL (usually http://localhost:8501). Open this in your web browser to interact with your AI assistant.

Conclusion

Congratulations! You've just built your own ChatGPT clone using the Llama-3 model. This application demonstrates the power of opensource language models and how they can be leveraged to create impressive AI applications right on your local machine.

Remember, while this is a great starting point, there's always room for improvement. You could experiment with different models, add more features to the interface, or even implement some form of memory or context management to make the conversations more coherent over longer exchanges.

If you've made it this far, don't forget to subscribe to Unwind AI for more awesome tutorials on building. And if you think others would find it useful too, please share it with your friends!