GPT-4 Vision Operates Computer Autonomously 🖥️

Today’s top AI Highlights:

1. Framework for Multimodal AI to Operate a Computer Itself

2. Starling-7B: Increasing LLM Helpfulness & Harmlessness with RLAIF

3. AI Model that Earns €10,000 per month

4. Text-to-Diagram without API Key

& so much more!

Read time: 3 mins

Latest Developments 🌍

Multimodal AI Agent for Self-operating Computers 🖥️

This AI Agent not only learns but can autonomously operate your computer just like you’d do. Hyperwrite has opensourced Self-Operating Computer framework that equips multimodal models to view the screen and decide on mouse and keyboard actions to achieve objectives.

Key Highlights:

1. The framework can work seamlessly with any multimodal model. It is currently integrated with GPT-4 Vision as the default model.

2. The team is developing a model for more accurate click location predictions to address the current challenge of high error rates in estimating XY mouse click locations by GPT-4 V.

Small But Mighty Model with RL-AI-F 💪

A 7B model that outperforms every model to date and closely matches GPT-4! Berkeley's AI Research Lab has opensourced Starling-7B model developed through Reinforcement Learning from AI Feedback (RLAIF). The team has also released a new ranking dataset, Nectar, that leverages GPT-4 labeled and introduces an innovative reward training and policy tuning pipeline.

Key Highlights:

1. Nectar is a high-quality GPT-4 labeled ranking dataset containing 183K chat prompts, each with 7 responses from various models, resulting in 3.8M pairwise comparisons. This dataset is critical for RLHF research.

2. The team released a reward model trained on the Nectar dataset. They fine-tuned the Openchat 3.5 language model using this reward model, resulting in improved scores on the MT-Bench and AlpacaEval, which assess the chatbot's helpfulness.

3. Upon fine-tuning with the reward model, Starling-7B significantly improved the performance of the Openchat 3.5 language model, increasing the MT-Bench score from 7.81 to 8.09 and AlpacaEval score, a metric assessing chatbot helpfulness, from 88.51% to 91.99%.

AI Model that Earns €10,000 per month

Imagine an AI model earning more from advertisements that even an entire agency might not be able to make! It’s a reality now. Meet Aitana, a Spanish AI model created by the Clueless Agency which has a massive following of over 121,000 people on Instagram. Aitana's engagements, including advertising contracts earning her over €1,000 per ad and a prominent role as the face of Big, a sports supplement brand.

Beyond her digital persona, Aitana's character is meticulously crafted by the agency, detailing her life and activities like being a fitness enthusiast with a complex character, reflecting current societal tastes and trends, to create an engaging and relatable personality.

Alongside these efficiency advantages, the AI model offers more predictable and manageable workflows, significantly reducing the unpredictability associated with human models being themselves.

Tools of the Trade ⚒️

• Text-to-diagram in Excalidraw: Use AI to generate diagrams from plain text, without any API token.

• ob1: First auto-generative backend tool that creates your entire backend needs including functions, schema, and it'll even spin up a database if you don't have one.

• Walles.ai: A browser extension that leverages GPT-4 and GPT-4 Vision for text extraction from images, math problem solving, text translations or paraphrasing, YouTube video summaries, and more.

• Manot: Insight management platform for computer vision models, offering automated feedback, data curation, and cost reduction. It provides actionable insights using a 5 billion image data lake and advanced AI to identify model blind spots.

Hot Takes 🔥

1. Someone should build a tool for inviting people to parties that actually understands the idea of inviting a couple instead of an individual w a +1. ~ Emmett Shear

2. Six predictions for AI in 2024: - A hyped AI company will go bankrupt or get acquired for a ridiculously low price - Open-source LLMs will reach the level of the best closed-source LLMs - Big breakthroughs in AI for video, time-series, biology and chemistry - We will talk much more about the cost (monetary and environmental) of AI - A popular media will be mostly AI-generated - 10 millions AI builders on Hugging Face leading to no increase of unemployment ~ Clement Delangue

Meme of the Day 🤡

That’s all for today!

See you tomorrow with more such AI-filled content. Don’t forget to subscribe and give your feedback below 👇

Real-time AI Updates 🚨

⚡️ Follow me on Twitter @Saboo_Shubham for lightning-fast AI updates and never miss what’s trending!!

PS: I curate this AI newsletter every day for FREE, your support is what keeps me going. If you find value in what you read, share it with your friends by clicking the share button below!