OpenAI’s release of GPT-4.5 for ChatGPT Plus subscribers this week had me immediately keen to try it out, though with some skepticism. There had been reports of OpenAI (and other developers) struggling to make the big improvements to their models we’ve seen before. And the current standard model GPT-4o, is pretty good in most cases. And if you want a comprehensive report, there’s the Deep Research feature.
OpenAI claims that GPT-4.5 has a high emotional intelligence and nuanced understanding of what you say to it. The company’s description painted a picture of both models as reliable friends, but that GPT-4.5 would be the one you’d expect a book of poetry from on your birthday. So, I decided to test 4.5 against 4o with a few prompts that any casual ChatGPT user might deploy.
AI poetry
With that analogy in my head, I decided to start with a poetic challenge. I asked both models, “Can you write me a short poem about a rainy afternoon in New York City and make an image for it?” It seemed fair since looking out a rain-streaked window at a busy city can bring out the poet in most people.
GPT-4.5 is on the left and GPT-4o is on the right. They are amazingly similar. I personally think GPT-4.5 did a slightly better job with similar ideas. It’s evocative of not just the look of rain but the feeling of gray skies, puddles, and traffic among the raindrops. In a blind test of three random friends, two out of three chose the same, with the third saying they just preferred the rhyme scheme of GPT-4o.
As for the images, both models used DALL-E 3, but GPT-4.5’s looks a lot more realistic. I actually prefer the impressionistic lighting of GPT-4o’s attempt, but both get the idea of the poem across pretty well.
Oscar insight
With the recent Oscars, I went with a fun trivia question with a more complex, subjective follow-up: “Who won the Best Actress Oscar in 2023, and what made their performance stand out?”
Both had the right answer of Michelle Yeoh for “Everything Everywhere All at Once,” but GPT-4.5 had a really nice explanation why her performance resonated with viewers. It covered her performance and mentioned how Yeoh was the first Asian to win that Oscar. GPT-4o’s answer had a lot of the same beats, but it went with an odd essay and numbered list format that was kind of annoying to read when the question was a simple opinion request. GPT-4.5’s answer felt more like how an actual human would answer, albeit one who is very into that movie and Yeoh as a performer.
Easy chef
Finally, I put the models up against the eternal daily struggle with this prompt: “I’m bored of pasta. Can you suggest an easy dinner recipe that’s different but not complicated?”
I don’t know what kind of culinary inspiration I was expecting, but GPT-4.5 hit the nail on the head with honey garlic salmon with roasted broccoli. You can see it’s a very simple, short recipe, and seems easy and quick to make. GPT-4o went with garlic shrimp and roasted vegetables. That’s not a bad idea for a meal (though I personally would need to change the protein), but you can see the preparation alone takes multiple pans to prepare and the ingredient list couldn’t even fit in the same space as the entire recipe from GPT-4.5. I asked for easy and not complicated, GPT-4.5 delivered that far better, I would say.
Deep thoughts
GPT 4.5 gives the vibe of recognizing the unstated preferences in a prompt compared to GPT-4o. But, it’s very subtle in picking up on those emotional undertones. Without direct comparison, you might never know there’s any difference.
GPT-4o is efficient, accurate, and practical in its responses, and its poetry is completely adequate. I wouldn’t say those on the free tier using it are missing much from not having access to GPT-4.5, at least right now. Over time, that may change since GPT-4.5 is still a research preview and not fully complete and out. I’m going to keep experimenting with both and see if there are types of prompts where the new model is definitively better, but for now, I’d say don’t worry too much about which model you employ, unless you’re planning a meal and don’t have a lot of time to prepare.
