1:19 5G Assignment details DS 402, Section 001: TREND IN DATA SCI (22411--UP---P-D... Objective This lab is meant for you to work with a neural network that solves MDPs the same way the table- based Q-learning agent did (but we switched the function representation and approximation). Recipe Ingredients Add the following files to your project from Lab3: agent/DeepQNetworkAgent.py testLab4.py Arrange the DeepQNetworkAgent.py in the agent folder. The test file should be in the top level directory of your code project. This project has a dependency on the sklearn's MLPRegressor. To add this package to your virtual machine in PyCharm, go to Preferences -> Project - > Python Interpreter. There you can press the + button and search for "scikit-learn" ("sklearn" will also work, but it is deprecated). Submit assignment ◄ Previous Next ▸ 12 51 Dashboard Calendar To-do Notifications Inbox 1:19A 5G Assignment details DS 402, Section 001: TREND IN DATA SCI (22411--UP---P-D... Your Tasks Before you begin the graded tasks, feel free to take some time reviewing all the new code and output of lab 4's tests 1+2, which follow "the movie" format we have seen in previous labs. As you do this, feel free to insert print statements anywhere you think might clarify things for you. 1. Task 1: Understand DQN training: A. Run lab4test3 to observe how our basic framework trains up a neural network. B. (TURN THIS IN) Prepare a learning curve for this training run. What do you see in it? In particular, what do you think is causing it to not improve consistently over time? 2. Task 2: DQNs and greed A. Run lab4test4 to observe three training sessions varying the probability of selecting a greedy action. B. (TURN THIS IN) Prepare learning curves for each of the three trainings. What do you see in them? 3. Task 3: DQNs and learning rate A. Run lab4test5 to observe three training sessions varying the initial learning rate Submit assignment ◄ Previous Next ▸ 12 51 Dashboard Calendar To-do Notifications Inbox 1:19A 5G Assignment details DS 402, Section 001: TREND IN DATA SCI (22411--UP---P-D... curves for each of the three trainings. What do you see in them? 3. Task 3: DQNs and learning rate A. Run lab4test5 to observe three training sessions varying the initial learning rate of the ADAM optimizer. B. (TURN THIS IN) Prepare learning curves for these training sessions. What do you see in them? 4. Task 4: DQNs and network depth A. Run lab4test7 (note that we skipped 6, feel free to check that test out, but it is not required) to observe three training sessions varying the depth of the network while keeping the neuron count fixed. B. (TURN THIS IN) Prepare learning curves for these training sessions. What do you see in them? 5. Task 5: DQNs and network width A. Run lab4test8 to observe three training sessions varying the width of the network. B. (TURN THIS IN) Prepare learning curves for these training sessions. What do you see in them? 6. Task 6: DQNs and regularization A. Run lab4test9 to observe three training Submit assignment ◄ Previous Next ▸ 12 51 Dashboard Calendar To-do Notifications Inbox 1:19A 5G Assignment details DS 402, Section 001: TREND IN DATA SCI (22411--UP---P-D... do you see in them? 5. Task 5: DQNs and network width A. Run lab4test8 to observe three training sessions varying the width of the network. B. (TURN THIS IN) Prepare learning curves for these training sessions. What do you see in them? 6. Task 6: DQNs and regularization A. Run lab4test9 to observe three training sessions varying the amount of regularization we apply to the network. B. (TURN THIS IN) Prepare learning curves for these training sessions. What do you see in them? 7. Task 7: Compare DQN with table-based Q- learning. A. (TURN THIS IN) Refer back to your answers from lab 3. Compare and contrast your answers to this lab with the analogous questions there. B. (TURN THIS IN) Examine lab11test10. Using the framework there, create the best DQN you can to solve that specified MDP (feel free to change the source in the DQNAgent itself too). Provide any modified source, a learning curve for your agent, and a reflection Submit assignment ◄ Previous Next ▸ 12 51 Dashboard Calendar To-do Notifications Inbox 1:19 5G Assignment details DS 402, Section 001: TREND IN DATA SCI (22411--UP---P-D... B. (TURN THIS IN) Prepare learning curves for these training sessions. What do you see in them? 7. Task 7: Compare DQN with table-based Q- learning. Submit A. (TURN THIS IN) Refer back to your answers from lab 3. Compare and contrast your answers to this lab with the analogous questions there. B. (TURN THIS IN) Examine lab11test10. Using the framework there, create the best DQN you can to solve that specified MDP (feel free to change the source in the DQNAgent itself too). Provide any modified source, a learning curve for your agent, and a reflection about approaches you tried as you settled on a particular network configuration and why you settled on what you did. A file that is readable (pdf, docx, etc) containing your charts, explanations, and neural network creation function(s). ◄ Previous Submit assignment Next ▸ 12 51 Dashboard Calendar To-do Notifications Inbox