OpenAI's Deep Research just became more accessible—but how well does it actually perform? If you don’t have the access, credits, or time to test it yourself, don’t worry, I’ve done the heavy lifting for you.
I put it through its paces, exploring how it handles academic research, peer-reviewed papers, and in-depth analysis. Can it find high-quality sources? Does it truly understand methodology? How well does it synthesise complex findings? My PDF takes you on a quick but insightful journey through these tests, showcasing both its strengths and limitations.
This is a must-read for academics, researchers, students, and anyone curious about the future of AI-powered research. Let’s dive in—does Deep Research live up to the hype?
Key takeaways:
Generic Research: The original request produced useful insights and was generally on target, but did not fully meet academic expectations. The primary issue was with the sources referenced, while relevant, they lacked the depth and connection to high-quality journal articles expected in academia. This was an intentional test, as I wanted to observe its default behaviour without specific guidance. When I explicitly asked it to focus only on peer-reviewed papers, its responses improved. However, deep research seemed to prioritise papers available in repositories and focused heavily on abstracts rather than full content. While passable for general information, this approach is limited for academic research, making it more suitable as an overview or starting point rather than a comprehensive analysis.
Directly targeting it with a DOI: When I directed the AI to my open-access paper, things got more interesting. Even though it was an open-access paper it could not access it directly. However, it persisted, ultimately finding the study through alternative sources, demonstrating adaptability and effort in refining its search. By directly targeting it with a specific study, the AI analysed the paper in extreme detail, making it a more practical option for in-depth research—although some mistakes remained.
- Acknowledging Oversight: I appreciated how the AI admitted the study should have been included in its initial assessment. I did test again with a non-relevant study, and it identify that the second paper should not have been included.
- Accuracy and Mistakes: While the AI’s interpretation was about 95% accurate, there were a few small errors. For example, it incorrectly claimed the study was conducted before Gemini’s release, when this was not the case. There were also minor misinterpretations of methodology and a few small factual inaccuracies in its explanation.
- Depth of Analysis and Usefulness: The AI provided extensive analysis, including summaries, discussions of limitations, and comparisons to other empirical studies using this approach.
The level of information was extensive, but a quick prompt adjustment can fine-tune its length.
Comments