Oct 29, 2021 | Shaoni Ghosh
The boundaries of Artificial Intelligence have flung open to the modernity of the 21st century, transcending unimaginable gaps. In doing so, it has earned a recognizable pedestal, mounting onto more incredible discoveries.
(Must Check: Top 20 artificial intelligence applications for 2021)
It has moved forward towards the generation of predictive models, such that it initiates the technology enabling search engines and other messaging applications to envisage the very next word one is going to type.
As aforementioned, Artificial Intelligence models of language were quite proficient at some tasks in the last few years. Predictive language models are particularly effective at estimating the next word in a string of text.
(Related Reading: What is Predictive Modelling?)
According to MIT research, they also learn about the fundamental meaning of language. These models can complete activities that appear to need some level of true comprehension, such as answering questions and story completion.
According to the research, the better a computer model is at estimating the next word, the more it mimics the human brain. It claims that computer models that do well on other sorts of linguistic problems do not exhibit this brain-like resemblance.
As mentioned by MIT News, the research was published in the Proceedings of the National Academy of Sciences this week. The National Science Foundation, through the National Institutes of Health, MIT scholarships, and the McGovern Institute, contributed to the research.
MIT researchers have developed computer models that can compose text that is comparable to that produced by humans. They matched language processing areas in the human brain to language-processing models using a similar tactic.
The researchers looked at 43 different language models, including a few that were specifically designed for next-word prediction.
Deep neural networks are a type of model that includes next-word prediction algorithms. Computational "nodes" make connections of different strength, and layers transport information between one another in predetermined ways in these networks.
The researchers analyzed the activity of the nodes that make up the network when each model was presented with a string of words. They matched these patterns to human brain activity, which was assessed in volunteers who completed three language tasks.
Human behavioral measurements such as how quickly people could read the text were closely associated with activity in such models.
Based on prior sequences, predictive models like GPT-3 can forecast what would happen next. They may generate predictions based on a large amount of information rather than just the past few words. No brain circuits that correlate to this sort of processing have yet been discovered, according to researchers.
(Also Check: Predictive Analytics: Techniques and Applications)
The researchers will now create versions of these models to determine how modest changes in their design influence their performance and capacity to replicate human data. They also intend to integrate these computer models with ones already produced by Tenenbaum's team.