OpenAI has released an update to its popular language model, ChatGPT, to enhance its accuracy and improve its ability to handle math equations.
Per the January 30 release notes: “We’ve upgraded the ChatGPT model with improved factuality and mathematical capabilities.”
It’s expected that the latest update to ChatGPT will allow it to handle complicated calculations and deliver more precise answers.
This would make ChatGPT a more valuable resource for students, researchers, and professionals who need quick and dependable information.
In practice, ChatGPT is still far from perfect regarding handling equations. However, there are some noticeable improvements in its ability to return factual responses.
Here are some observations on the January 30 update based on my testing and feedback shared on Twitter.
ChatGPT Accuracy – Hit & Miss
One notable improvement to ChatGPT’s accuracy is that it’s no longer possible to trick it into giving an incorrect answer.
There was a meme showing how ChatGPT could be talked into giving the wrong answer if you said your wife disagreed with its response.
Although it may seem absurd, it was actually the case. See an example in the tweet below:
Yeah 😎 pic.twitter.com/XRq4ldxjpt
— Nema Zime (@ProgFromSouth) January 30, 2023
Now, ChatGPT will continue to return the correct response, even if you try to convince it otherwise.
Here’s a test I ran following the January 30 update:
Screenshot from: chat.openai.com/chat. January 2023.That’s a positive sign. However, the negative feedback on the January 30 update outweighs the good.
A test I always go back to is asking ChatGPT who is the taller basketball player between Shaquille O’Neal and Yao Ming.
ChatGPT continues to get this wrong, despite returning the correct heights of the two men.
Interestingly, if you point out its flaws, it will correct itself.
Screenshot from: chat.openai.com/chat. January 2023.People on Twitter point out that ChatGPT struggles with math equations when typed out in complete sentences instead of numbers and symbols.
ChatGPT's Jan 30 update promises "improved factuality and mathematical capabilities".
I tried it on previous failure modes, but it failed.
The right answers here are 44% (not 46%) and 1555.8.. (not 1551.9..). pic.twitter.com/pAsMeC9UZU
— Deedy (@debarghya_das) January 31, 2023
On the other hand, it appears to perform exceptionally well when fed questions from standardized tests.
According to one individual, ChatGPT is capable of passing the math section of an SAT:
Just tried the upgraded ChatGPT model with improved math capabilities –
It just crushed the math with calculator section of a 2020 SAT and only made two mistakes.
Here are two examples of the problems it was solving in less than 5 seconds🤯 pic.twitter.com/srLcSfE8An
— Charis Zhang (@gmchariszhang) January 30, 2023
Perhaps ChatGPT handles standardized test questions better because it’s language the AI model has encountered before, as opposed to user-inputted questions it’s seeing for the first time.
Overall, feedback on this update is mixed. Without fact-checking first, I would still be cautious about relying on ChatGPT’s responses.
In Summary
The release of this update, the third major update since the introduction of ChatGPT, underscores OpenAI’s continuous efforts to stay ahead in the AI industry.
Despite enhanced capabilities, ChatGPT still has a long way to go.
Based on OpenAI’s previous update schedule, further improvements to ChatGPT can probably be expected soon.
Featured Image: rafapress/Shutterstock
