There’s a new leader, technically, in the race for AI assistant dominance, and it’s Anthropic’s new Claude 3.5 Sonnet. The newly released model outperforms both Gemini 1.5 Pro and ChatGPT-4o across a spectrum of benchmark tests, the company announced on Thursday. This new iteration of Sonnet is the first in Anthropic’s upcoming line of 3.5 models, and it significantly outperforms the more expansive Opus 3.0 model, and does so at a fraction of the larger model’s energy cost. Compute efficiency is becoming an increasingly important aspect of AI system design, especially as the cost of both powering and cooling AI data centers soars while the infrastructure pushes into the gigawatt range. “Claude 3.5 Sonnet operates at twice the speed of Claude 3 Opus,” the Anthropic team wrote in a blog post. “This performance boost, combined with cost-effective pricing, makes Claude 3.5 Sonnet ideal for complex tasks such as context-sensitive customer support and orchestrating multistep workflows.” The new model has reportedly set benchmark results across three standardized tests: graduate-level reasoning with GPQA, undergraduate-level knowledge with MMLU, and coding proficiency with HumanEval. It beat out Google’s Gemini 1.5 Pro, Meta’s Llama-400b, and OpenAI’s ChatGPT-4o, though not by any huge margin and typically only by a couple percentage points. Sonnet 3.5 is being billed as Anthropic’s “strongest vision model yet. ” It’s capable of performing a number of vision-based tasks — like interpreting charts and graphs or transcribing text from imperfect image sources like screenshots or scanned receipts — more accurately than Opus 3.0. In fact, Sonnet 3.5 beat out Opus 3.0 by anywhere from 6 to 17 points across industry standard vision benchmarks. The new model is also reportedly much more competent at handling humor and can converse in a much more lifelike manner. Sonnet will also be the first Anthropic AI to offer the Artifacts feature to users. Rather than generate images or code snippets directly into the flow of the conversation, Artifacts will create that content in a dedicated space to the side of the chat. This allows users to create “a dynamic workspace where they can see, edit, and build upon Claude’s creations in real time, seamlessly integrating AI-generated content into their projects and workflows,” the Anthropic team claims. It also announced that Claude will soon support team collaboration wherein a company can store its data, documents and projects in a single, central silo, with Claude acting as an on-demand assistant. You can try out Claude 3.5 Sonnet today for free on the Claude.ai website and the Claude iOS app (a Claude Pro or Team subscription will garner you significantly higher rate limits). Third-party integration is also available through the Anthropic API, Amazon Bedrock, and Google Cloud’s Vertex AI. Claude Haiku 3.5 and Opus 3.5 are scheduled for release later in the year. We’re now into the third year of the AI boom, and industry leaders are showing no signs of slowing down, pushing out newer and (presumably) more capable models on a regular basis. ChatGPT, of course, remains the undisputed leader. But with more than a half-dozen models available from OpenAI alone, figuring out which one to use for your specific project can be a daunting task.
o1 French AI startup Mistral announced Monday that it is incorporating a half-dozen new features and capabilities into its free generative AI work assistant, dubbed le Chat (French for “the cat”), that will put the open-source chatbot on par with leading frontier models from OpenAI and Anthropic. Le Chat can now search the web and provide cited sources, similar to what Perplexity and SearchGPT both offer. Mistral’s chatbot now also offers a Canvas feature akin to Claude’s Artifacts where users can modify and edit content and code. What’s more, le Chat can now generate images thanks to an integration with Black Forest Labs’ Flux Pro, the same image generator that powers Grok-2’s capabilities. In the ever-growing large language model (LLMs) landscape, two front-runners stand out from the rest of the race: Anthropic’s Claude 3.5 Sonnet and OpenAI’s GPT-4o (the “o” stands for “Omni”). Both AIs boast impressive capabilities, but which reigns supreme? This guide dives deep into Claude 3.5 Sonnet and GPT-4o, dissecting their strengths and weaknesses across various tasks. We’ll not only explore their relative accuracy and response speeds, but also unpack their pricing structures and service tiers. By the end of this article, you’ll be armed with the knowledge to make an informed decision about which LLM best suits your specific needs and budget.
Pricing and tiers
ChatGPT users can access the GPT-4o model for free, but the number of queries you’re allowed to ask it within a three-hour span is limited. Exceeding that limit will move you down to using GPT-3.5, OpenAI’s older, less capable model, until the cooldown timer has reset.