Advances in Engineering Innovation
Vol. 8, 28 June 2024
Open
Access | Article
Research on Data Intelligent Generation and Analysis Based on ChatGPT
* Author to whom correspondence should be addressed.
Abstract
This paper conducts research on data intelligent generation and analysis based on the ChatGPT model. To explore ChatGPT's performance and limitations in machine translation tasks, the concepts of the Transformer model and previous studies were reviewed to gain a deep understanding of the principles and roles of components such as attention mechanisms, encoders, decoders, and word embeddings. By controlling ChatGPT through code for machine translation and performing manual verification, the model's limitations in handling synonyms, technical terms, and specific domain languages were identified.
Keywords
deep learning, GPT, transformer, code generation
References
1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., & Gomez, A. N., et al. (2017). Attention is all you need. arXiv. https://arxiv.org/abs/1706.03762
2. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI. https://www.openai.com/research/language-understanding-generative-pretraining
3. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., & Amodei, D. (2020). Language models are few-shot learners. arXiv. https://arxiv.org/abs/2005.14165
4. Sun, B., & Li, K. (2021). Neural dialogue generation methods in open domain: A survey. Natural Language Processing Research, 1(3-4), 56-70. https://doi.org/10.2991/nlpr.d.210715.001
5. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., ... & Rush, A. M. (2020, October). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations (pp. 38-45). https://doi.org/10.18653/v1/2020.emnlp-demos.6
6. Kalyan, K. S., Rajasekharan, A., & Sangeetha, S. (2021). Ammus: A survey of transformer-based pretrained models in natural language processing. arXiv preprint arXiv:2108.05542. https://arxiv.org/abs/2108.05542
7. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. https://arxiv.org/abs/1810.04805
8. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. https://doi.org/10.1145/3065386
9. Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., ... & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8, 1-74. https://doi.org/10.1186/s40537-021-00444-8
10. Shi, L., Wang, Y., Cheng, Y., & Wei, R. B. (2020). A review of attention mechanism in natural language processing. Data Analysis and Knowledge Discovery, 4(5), 1-14. https://doi.org/10.11925/infotech.2096-3467.2019.0462
11. Xu, G., & Wang, H. F. (2011). Development of topic models in natural language processing. Journal of Computer Science and Technology, 34(8), 1423-1436. https://doi.org/10.1007/s11390-011-1193-3
12. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. https://arxiv.org/abs/1406.1078
13. Yin, W., Kann, K., Yu, M., & Schütze, H. (2017). Comparative study of CNN and RNN for natural language processing. arXiv preprint arXiv:1702.01923. https://arxiv.org/abs/1702.01923
14. Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016, November). Deep API learning. In Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering (pp. 631-642). https://doi.org/10.1145/2950290.2950334
15. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. https://arxiv.org/abs/1810.04805
Data Availability
The datasets used and/or analyzed during the current study will be available from the authors upon reasonable request.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Authors who publish this journal agree to the following terms:
1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See Open Access Instruction).
- Volume Title
- ISBN (Print)
- ISBN (Online)
- Published Date
- 28 June 2024
- Series
- Advances in Engineering Innovation
- ISSN (Print)
- 2977-3903
- ISSN (Online)
- 2977-3911
- DOI
- 10.54254/2977-3903/8/2024082
- Copyright
- 28 June 2024
- Open Access
- This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited