Today’s top AI Highlights:

1. Replit AI Agent to build and deploy web apps from scratch

2. Open-source tool to simulate real-world LLM inference

3. Groq releases its first multimodal model on GroqCloud

4. OpenAI considering $2,000/month subscription for Strawberry and Orion

5. Math OCR model that outputs LaTeX and markdown

& so much more!

Read time: 3 mins

Latest Developments

Stop Guessing, Start Measuring LLM Inference 📈

Neural Magic has open-sourced GuideLLM to help you evaluate and optimize LLM deployment. It simulates real-world inference workloads so you can assess the performance, resource requirements, and costs associated with deploying LLMs on diverse hardware configurations. This helps to make informed decisions about LLM deployment strategies, ensuring efficiency and scalability.

Key Highlights:

1. Comprehensive Performance Evaluation - Get insights under various load scenarios, like request latency, time to first token (TTFT), and inter-token latency (ITL). This helps identify bottlenecks and ensure your LLM deployments meet desired service level objectives.

2. Resource Optimization and Cost Estimation - It helps identify the most suitable hardware configurations for your LLMs and estimate the financial implications of various deployment strategies.

3. Scalability Testing - Simulate scaling your LLM deployments to handle large numbers of concurrent users. This ensures that performance remains consistent even under high load conditions.

4. Getting Started - You'll need an OpenAI-compatible server like vLLM to get started. Set your target server, model, data type, and desired performance benchmarks. Detailed instructions and examples are available in the GuideLLM GitHub repository.

AI Agents Build and Deploy Your Apps 🤖

Replit has launched Replit Agents in early access. These AI agents function as automated software developers within Replit's existing online IDE to build apps from scratch. You just have to use simple language instructions to tell the agent what you want to build. Replit Agents will then create the necessary code, set up the development environment, install dependencies, and even deploy the finished application to the cloud.

Key Highlights:

1. Step-by-step building and editing - Agents don't just write code and throw it at you. They present a development plan, break down the building process into individual steps, and give you the option to approve, pause, or edit at each stage.

2. See exactly what the agent is doing - Every action the agent takes, from code generation to dependency installation, is displayed in the Replit interface. This complete transparency makes it easy to follow along and learn how the app is being built.

3. One-click deployment - Forget about configuring servers or databases. When you're ready to launch your app, Replit Agents handle the entire deployment process directly to Replit's cloud hosting.

4. Get Started - Replit Agents is available to use for Replit subscribers. Just go to Replit logged-in homepage. Write what you want to make and click "start building"

Quick Bites

Groq has released its first multimodal model LLaVA v1.5 7B on GroqCloud. It can handle image, audio, and text inputs. Initial testing by Artificial Analysis shows that its response time is 4X faster than GPT-4o. You can try it here.

Google DeepMind has introduced AlphaProteo, a new AI system to create novel proteins that bind to target molecules, helping advance drug development, disease understanding, and biosensors. It achieves higher success rates and binding strengths compared to existing methods.

OpenAI has crossed 1 million paying business users via its various enterprise plans. Reportedly, OpenAI is also considering hiking its subscription price to $2,000 a month for its upcoming model Strawberry and flagship LLM Orion.

Google is rolling out its AI feature “Ask Photos” in the U.S., allowing users to search their photo library with natural language queries. The AI can handle complex searches like finding the best photos from specific trips or recalling events, using photo content and metadata 

Tools of the Trade

1. Texify: An OCR model that converts images or PDFs containing math into markdown and LaTeX that can be rendered by MathJax ($$ and $ are delimiters). It can run on CPU, GPU, or MPS.

2. Polar v1.0: A funding and monetization platform for developers to get paid for their work, including open-source projects. It offers donations, crowdfunding, memberships, and SaaS with built-in integrations to handle sales tax and product delivery.

3. PR-Agent: A Chrome extension that gives context-aware assistance in your GitHub environment to analyze pull requests, automate reviews, highlight changes, and suggest code improvements.

4. Awesome LLM Apps: Build awesome LLM apps using RAG to interact with data sources like GitHub, Gmail, PDFs, and YouTube videos through simple text. These apps will let you retrieve information, engage in chat, and extract insights directly from content on these platforms.

Hot Takes

1. My take: the founder cult in SV is totally out of control. It formalizes a de facto aristocracy as the preferred form of governance in tech. It leads to outsize discrepancies in risk taking and outcomes for most tech workers. It enables some of the most toxic psychopaths to be venerated as “visionaries”. Most of the worst people I came across in tech were “founders”. ~
   Bojan Tunguz
   

2. I think people massively overestimate the value of potential “training data” - there’s this widespread idea that AI companies desperately want to suck in every line of text on the planet, but I don’t think that’s actually true ~
   Simon Willison

Meme of the Day

That’s evil

That’s all for today! See you tomorrow with more such AI-filled content.

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!

Unwind AI - Twitter | LinkedIn | Instagram | Facebook

Awesome LLM Apps | Sponsor Us

PS: We curate this AI newsletter every day for FREE, your support is what keeps us going. If you find value in what you read, share it with at least one (or 20) of your friends!