Timeline of transformers
From Timelines
This is a timeline of FIXME.
Contents
Sample questions
The following are some interesting questions that can be answered by reading this timeline:
Big picture
| Time period | Development summary | More details |
|---|
Full timeline
| Year | Month and date | Event type | Details |
|---|---|---|---|
| 2017 | June | Google researchers first describe the transformer algorithm that would turbocharge the power of chatbots. | |
| 2018 | June 11 | OpenAI releases a paper entitled Improving Language Understanding by Generative Pre-Training, in which they introduces the Generative Pre-trained Transformer (GPT).[1] | |
| 2019 | February 14 | OpenAI releases Generative Pre-trained Transformer 2 (GPT-2). | |
| 2020 | June 11 | OpenAI releases Generative Pre-trained Transformer 3 (GPT-3) in beta. | |
| 2023 | February 18 | A paper evaluates the performance of Generative Pre-trained Transformer (GPT) models for machine translation, covering various aspects such as the quality of different GPT models, the effect of prompting strategies, robustness towards domain shifts and document-level translation. The experiment includes eighteen different translation directions involving high and low resource languages, as well as non English-centric translations. The results show that GPT models achieve competitive translation quality for high resource languages, while having limited capabilities for low resource languages. Hybrid approaches, which combine GPT models with other translation systems, can further enhance the translation quality. The paper provides valuable insights for researchers and practitioners in the field to understand the potential and limitations of GPT models for translation.[2] |
Meta information on the timeline
How the timeline was built
The initial version of the timeline was written by FIXME.
Funding information for this timeline is available.
Feedback and comments
Feedback for the timeline can be provided at the following places:
- FIXME
What the timeline is still missing
Timeline update strategy
See also
External links
References
- ↑ Radford, Alec; Narasimhan, Karthik; Salimans, Tim; Sutskever, Ilya (11 June 2018). "Improving Language Understanding by Generative Pre-Training" (PDF). OpenAI. p. 12. Archived from the original (PDF) on 26 January 2021. Retrieved 23 January 2021.
- ↑ Hendy, Amr; Abdelrehim, Mohamed; Sharaf, Amr; Raunak, Vikas; Gabr, Mohamed; Matsushita, Hitokazu; Kim, Young Jin; Afify, Mohamed; Awadalla, Hany Hassan (17 February 2023). "How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation". arXiv:2302.09210 [cs]. doi:10.48550/arXiv.2302.09210.
