CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks

Collaboration policy:

You are encouraged to do the assignment in pairs but if you prefer, can also work alone.

You must indicate this at the top of the assignment.

Each person still needs to submit the assignment.

They can be highly similar but don't have to

Collaboration with more than 1 person is not allowed.

Instructions:

For this assignment you'll be using TensorFlow playground: Link to Website

In this assignment, it's crucial to understand that many of the questions do not have a definitive

right or wrong answer, and there may be various valid ways to approach and solve them. The

emphasis is on providing well-thought-out and insightful responses rather than merely writing

lengthy answers. Quality is prioritized over quantity, so you don't need to worry about the length

of your responses. Focus on effectively conveying your main points, whether that involves

writing longer explanations or using concise wording. Both approaches are acceptable and

encouraged. The goal is to showcase your understanding and critical thinking skills through your

explanations.

What do I need to upload? Kindly provide your responses directly within this document and

proceed to upload the completed file.

Please go to the next page CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks

The main objective of this assignment is to offer students hands-on experience with neural

networks using TensorFlow Playground, an interactive web-based tool. Through practical

experimentation, you will explore various aspects of neural network design, including different

architectures, activation functions, and hyperparameters. The goal is to help you grasp how these

choices impact the neural network's behavior and performance.

The assignment comprises three main questions, each focusing on distinct datasets and

configurations. For each experiment, you are required to provide details of the parameters used (if

it's not specified as a fix amount in the question), training and testing loss values, the number of

epochs needed to reach the desired outcome (loss below 0.15), or the number of epochs when the

loss stabilizes. Additionally, you are encouraged to include an image of their best output, which

may be cropped to show the losses. For all questions, you should evaluate the experiments using

learning rates of 0.01, 0.1, 0.3, 1, and 3, and you are expected to choose the most effective learning

rate and comment on any intriguing behavior observed during the experiments (ex: I didn't observe

significant differences when changing the learning rate from 0.01 to 0.1. However, upon setting it

to 0.3, the model exhibited faster boundary identification, but when I changed it to 3 ... ).

Q1. Create a neural network with following setup (initial configuration): (8/20 Points)

Data:

●

●

Dataset: Gausian

Ratio of training to test data: 60%

Noise: 20

Batch size: 10

O

O

Features: X1, X2

Number of hidden layers: 0

1.1. Experiment with the linear activation function and choose a learning rate that suits your

preference. Evaluate whether this network configuration can effectively find the decision

boundary for the given dataset. Present your findings and results. (1 Point)

1.2. Switch the dataset to “Exclusive Or” while keeping all other parameters unchanged.

Execute the model and assess its capability to identify the decision boundary. Provide

reasons for why it can or cannot find the decision boundary. (1 Point)

1.3. In your experimentation, incorporate the sigmoid and tanh activation functions separately.

Determine whether these network structures can successfully discover the decision

boundary. (1 Point)

1.4. Introduce an additional hidden layer with two neurons to the network, utilizing both the

tanh and sigmoid activation functions. Report the outcome and describe any observed

effects resulting from this modification. (1 Point)

1.5. Increase the number of neurons in the hidden layer and elaborate on the noticeable changes

you observe in the model's performance. (1 Point) CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks

1.6. Remove the neurons added in the previous step, and instead, add another hidden layer

containing two neurons (resulting in two hidden layers, each with two neurons). Provide a

detailed report on the results obtained and offer explanations for any significant

observations. (1 Point)

1.7. Remove all the hidden layers and select the linear activation function. use engineered

features as follows:

1.7.1. Use X and X2 and report your result. (1 Point)

1.7.2. Use X₁ X₂ and report your result. (1 Point)

Q2. Create a neural network with following setup (initial configuration): (4/20 Points)

Data:

O

Dataset: Circle

O

Ratio of training to test data: 70%

O

Noise: 10

O Batch size: 10

2.1. Utilize the sigmoid activation function and conduct experiments by gradually adding varying

numbers of layers and neurons to the neural network until the decision boundary is successfully

found. Document your findings, including the network structure image, and provide detailed

explanations of your observations. (2 Points)

2.2. Explore alternative approaches, such as feature engineering and changing the activation

functions (tanh, ReLU), to achieve the decision boundary using fewer nodes and layers. Share your

results, accompanied by the network structure image, and elaborate on the insights gained from

your experiments. (2 Points)

Q3: Create a neural network with following setup (initial configuration): (8/20 Points)

Data:

O

O

Dataset: Spiral

Ratio of training to test data: 70%

Noise: 50

Batch size: 10

3.1. Explore the TensorFlow Playground using any available tools to discover the decision

boundary while ensuring that the test loss is below 0.08 and the train loss is below 0.05. Document

and explain your observations, and include an image of the network structure and the obtained

result. (3 Points)

3.2. Set the noise level to 25 and use your network structure from previous example to find decision

boundary for other three dataset. Was your network able to find the decision boundary with loss

less than 0.15 for these datasets? (2 Points) CGSC 3601-B: Artificial Intelligence and Cognitive Science Late Summer 2023 - Assignment3: Neural Networks

3.3. Summarize your overall takeaways from all the experiments conducted up to this point.

Reflect on the outcomes and observations you made, discussing any trends or patterns you noticed

in the neural network's behavior with different configurations. (3 Points)