Despite OpenAI’s confidence in the o1-pro, initial user feedback has been mixed.
The model, which has been available on OpenAI’s AI-powered chatbot platform ChatGPT for ChatGPT Pro subscribers since December last year, struggled with tasks like solving Sudoku puzzles and understanding simple optical illusion jokes.
Additionally, internal benchmarks from late last year indicated that while the o1-pro performed slightly better than the standard o1 on coding and math problems, it did not show significant improvement in these areas.