Objective: The goal of this assignment is to build a Transformer model in PyTorch or Tensorflow for natural language processing (NLP), e.g., predicting the sentiment (or attitude) of a sentence in the form of sequence classification. In this task, the Transformer needs to process a sentence and output a sentiment class (like positive, negative, or neutral). Specifically, you will: 1. Choose a dataset (like the IMDb dataset for movie reviews). and preprocess the data: tokenize the sentences, truncate/pad them to a uniform length, and convert them to tensor format. 2. Utilize a built-in packages or libraries such as Pytorch (recommended) and Tensorflow to construct your model. 3. Utilize a built-in method to perform forward and backward computations. 4. Evaluate the improve the performance of your algorithm. 5. Run your code on CHEAHA (Optional, not graded). The tutorial of using Cheaha can be found here HINT: Using Hugging Face libraries to find a suitable dataset. You may refer to Lecture 9 slides for some piece of sample codes. Tasks and Requirements: 1. Algorithm Implementation: • Employ the transformer for sentiment classification on the dataset. 2. Performance Improvement Strategies: • Analyze how to improve the performance of your implementations, including and not limited to: tuning hyperparameters such as number of nodes in hidden layer, strength of regularization, model ensemble, batch normalization, etc. Explore by yourself! 3. Comprehensive Report: ° Prepare a detailed report encompassing the following sections: Background and Method Introduction: Provide an overview of the transformer and its application in sentiment classification. Dataset and Tasks Description: Describe the a sentiment analysis dataset and outline the specific classification tasks undertaken. ■ Algorithms Used: Elaborate on the implementation details of your codes. Attach screenshot of the codes whenever necessary. ■ Results: Present and discuss the classification results obtained. ■ Methods of Improvements: Discuss the strategies employed to enhance the performance of your algorithm. 4. Submission Format: ⚫ Submit your work in the form of Jupyter Notebook (.ipynb) and HTML files, along with the final report. Submit each file separately. Grading Criteria: ⚫ Implementation of the transformer algorithm (40%): Successfully implement a transformer to perform sentiment classification. . Results and Improvement (40%) : Thoughtful considerations and implementations for validating and improving your algorithm performance. ⚫ Report Quality (20%): Overall quality, clarity, organization, and thoroughness of the submitted report. . Use Cheaha to run you code on server. (Optional)

Fig: 1