![]() That was unfortunate, because language is the main way we humans communicate.īefore Transformers were introduced in 2017, the way we used deep learning to understand text was with a type of model called a Recurrent Neural Network or RNN that looked something like this: But for a long time, nothing comparably good existed for language tasks (translation, text summarization, text generation, named entity recognition, etc). Let’s say you wanted to translate a sentence from English to French. An RNN would take as input an English sentence, process the words one at a time, and then, sequentially, spit out their French counterparts. ![]() ![]() Means something very different from the sentence: The key word here is “sequential.” In language, the order of words matters and you can’t just shuffle them around. So any model that’s going to understand language must capture word order, and recurrent neural networks did this by processing one word at a time, in a sequence.īut RNNs had issues. ![]() By the time got to the end of a paragraph, they’d forget what happened at the beginning.įirst, they struggled to handle large sequences of text, like long paragraphs or essays. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |