Collaboration policy:
You are encouraged to do the assignment in pairs but if you prefer, can also work alone.
You must indicate this at the top of the assignment.
Each person still needs to submit the assignment.
They can be highly similar but don't have to
Collaboration with more than 1 person is not allowed.
Instructions:
For this assignment you'll be using TensorFlow playground: Link to Website
In this assignment, it's crucial to understand that many of the questions do not have a definitive
right or wrong answer, and there may be various valid ways to approach and solve them. The
emphasis is on providing well-thought-out and insightful responses rather than merely writing
lengthy answers. Quality is prioritized over quantity, so you don't need to worry about the length
of your responses. Focus on effectively conveying your main points, whether that involves
writing longer explanations or using concise wording. Both approaches are acceptable and
encouraged. The goal is to showcase your understanding and critical thinking skills through your
explanations.
What do I need to upload? Kindly provide your responses directly within this document and
proceed to upload the completed file.
Please go to the next page CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks
The main objective of this assignment is to offer students hands-on experience with neural
networks using TensorFlow Playground, an interactive web-based tool. Through practical
experimentation, you will explore various aspects of neural network design, including different
architectures, activation functions, and hyperparameters. The goal is to help you grasp how these
choices impact the neural network's behavior and performance.
The assignment comprises three main questions, each focusing on distinct datasets and
configurations. For each experiment, you are required to provide details of the parameters used (if
it's not specified as a fix amount in the question), training and testing loss values, the number of
epochs needed to reach the desired outcome (loss below 0.15), or the number of epochs when the
loss stabilizes. Additionally, you are encouraged to include an image of their best output, which
may be cropped to show the losses. For all questions, you should evaluate the experiments using
learning rates of 0.01, 0.1, 0.3, 1, and 3, and you are expected to choose the most effective learning
rate and comment on any intriguing behavior observed during the experiments (ex: I didn't observe
significant differences when changing the learning rate from 0.01 to 0.1. However, upon setting it
to 0.3, the model exhibited faster boundary identification, but when I changed it to 3 ... ).
Q1. Create a neural network with following setup (initial configuration): (8/20 Points)
Data:
●
●
Dataset: Gausian
Ratio of training to test data: 60%
Noise: 20
Batch size: 10
O
O
Features: X1, X2
Number of hidden layers: 0
1.1. Experiment with the linear activation function and choose a learning rate that suits your
preference. Evaluate whether this network configuration can effectively find the decision
boundary for the given dataset. Present your findings and results. (1 Point)
1.2. Switch the dataset to “Exclusive Or” while keeping all other parameters unchanged.
Execute the model and assess its capability to identify the decision boundary. Provide
reasons for why it can or cannot find the decision boundary. (1 Point)
1.3. In your experimentation, incorporate the sigmoid and tanh activation functions separately.
Determine whether these network structures can successfully discover the decision
boundary. (1 Point)
1.4. Introduce an additional hidden layer with two neurons to the network, utilizing both the
tanh and sigmoid activation functions. Report the outcome and describe any observed
effects resulting from this modification. (1 Point)
1.5. Increase the number of neurons in the hidden layer and elaborate on the noticeable changes
you observe in the model's performance. (1 Point) CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks
1.6. Remove the neurons added in the previous step, and instead, add another hidden layer
containing two neurons (resulting in two hidden layers, each with two neurons). Provide a
detailed report on the results obtained and offer explanations for any significant
observations. (1 Point)
1.7. Remove all the hidden layers and select the linear activation function. use engineered
features as follows:
1.7.1. Use X and X2 and report your result. (1 Point)
1.7.2. Use X₁ X₂ and report your result. (1 Point)
Q2. Create a neural network with following setup (initial configuration): (4/20 Points)
Data:
O
Dataset: Circle
O
Ratio of training to test data: 70%
O
Noise: 10
O Batch size: 10
2.1. Utilize the sigmoid activation function and conduct experiments by gradually adding varying
numbers of layers and neurons to the neural network until the decision boundary is successfully
found. Document your findings, including the network structure image, and provide detailed
explanations of your observations. (2 Points)
2.2. Explore alternative approaches, such as feature engineering and changing the activation
functions (tanh, ReLU), to achieve the decision boundary using fewer nodes and layers. Share your
results, accompanied by the network structure image, and elaborate on the insights gained from
your experiments. (2 Points)
Q3: Create a neural network with following setup (initial configuration): (8/20 Points)
Data:
O
O
Dataset: Spiral
Ratio of training to test data: 70%
Noise: 50
Batch size: 10
3.1. Explore the TensorFlow Playground using any available tools to discover the decision
boundary while ensuring that the test loss is below 0.08 and the train loss is below 0.05. Document
and explain your observations, and include an image of the network structure and the obtained
result. (3 Points)
3.2. Set the noise level to 25 and use your network structure from previous example to find decision
boundary for other three dataset. Was your network able to find the decision boundary with loss
less than 0.15 for these datasets? (2 Points) CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks
3.3. Summarize your overall takeaways from all the experiments conducted up to this point.
Reflect on the outcomes and observations you made, discussing any trends or patterns you noticed
in the neural network's behavior with different configurations. (3 Points)
Fig: 1
Fig: 2
Fig: 3