OpenAI’s new model – still not that smart

OpenAI began making its latest natural language model, GPT-3, available to developers and outside researchers this week. It is a massive transformer-based neural network with 175 billion parameters (for perspective, its predecessor, GPT-2, has between 1 and 2 billion parameters) and was trained on the largest dataset of any model, including the entirety of Wikipedia, among other input sources. Because of its size, it can perform specific tasks like translation, text summarization, text generation, and it can even write working code, all without requiring the model to be specially tuned for the task. Researchers have produced novel applications already including a SQL generator to allow users to query a database in natural language, and OpenAI plans to commercialize the product as a cloud AI service for enterprise customers later this year.