3.5) Which of the following statements is/are true?

2.3) Which of the following statements is/are true? For sigmoid activation function, as input to the function becomes larger and positive, the activation value approaches 1. Similarly, as it becomes very negative, the activation value approaches 0. For training MLPs, we do not use activation function as a step function, as in the flat regions there is no slope and hence no gradient to learn. For tanh activation function, as the value increases along the positive horizontal axis, the activation value tends to 1 and along the negative horizontal axis, it tends towards -1, with the centre value at 0.5. The ReLU activation function is a piecewise linear function with a flat region for negative input values and a linear region for positive input values.

3.4) Which of the following statements is/are true? Learning using optimizers such as AdaGrad or RMSProp is adaptive as the learning rate is changed based on a pre-defined schedule after every epoch. Since different dimension has different impact, adapting the learning rate per parameter could lead to good convergence solution. Since AdaGrad technique, adapts the learning rate based on the gradients, it could converge faster but it also suffers from an issue with the scaling of the learning rate, which impacts the learning process and could lead to sub-optimal solution. RMSProp technique is similar to the AdaGrad technique but it scales the learning rate using exponentially decaying average of squared gradients. 0 Adam optimizer always converge at better solution than the stochastic gradient descent optimizer.

2.1) Which of the following statement(s) is/are true? Perceptron network with just input and output layer converges for linearly separable cases only. Logistic regression model and perceptron network are the same model. Unlike Perceptron network, logistic regression model can converge for non-linear decision boundary cases. 0 By changing the activation function to logistic and by using gradient descent algorithm, Perceptron network can be made to address non-linear decision boundary cases.

(2) A report in PDF format (8 marks) should have the following sections a. A description of how to handle the missing values in your code and report the results. (2.5 Marks) b. A description of the regression technique you used. (3 Marks) c. A description of the results to report. (2.5 Marks)

2.7 Which of the following statements is/are true? At the output layer of a binary classification problem, we could either use sigmoid activation function with single output neuron or softmax function with two neurons, to produce similar results. For a classification application, that predicts whether a task is personal or official and which also predicts whether it is high or low priority, we could use two output neuron and apply sigmoid activation function on each output neuron. For a classification application, that predicts whether Team A would win, lose or draw the match, we could use three output neurons with softmax activation function applied at the output layer.

2.8 Which of the following points hold true for gradient descent? It is an iterative algorithm and every step it finds the gradient of the cost function with respect to the parameters to minimize the cost. For a pure convex function or a regular bowl-shaped cost function, it can converge to the optimal solution irrespective of the learning rate Since cost functions can be of any shape, it can only achieve local minima but not global minimum, provided the cost function is not convex. Normalizing the variables and bringing the magnitude of these variables to same scale, ensures faster convergence.

COMP 1002 Assignment 4 More Induction! Please read the following important notes: Your assignment must be completed on your own. Sharing solutions, looking for and/or copying solutions from unauthorized sources, and posting the assignment questions online are all forms of academic misconduct. Committing an act of academic misconduct will result in a grade of 0 and reported to the department head. • Only one problem set will be graded and it will be selected at random. Students who attempt to solve the problem but do not get a perfect score are allowed to submit a Reflection to obtain bonus marks on their Assignment. If you do not attempt to solve the selected problem set, then you cannot submit a reflection. • Please submit a file for each problem set and clearly state the problem set in the file's name. Acceptable file formats are PDF, JPG, or PNG. Do not include your name/id in your submission. • If your solutions are handwritten, please ensure your solution is legible. Double check both your image quality and handwriting. If we cannot read your solution, then you will get a mark of 0. • While writing your solutions, you will be graded on both your approach and correctness. A given problem may be solvable in a number of different ways - we will not grade you on efficiency. That being said, if a problem says to use a specific technique (i.e. Natural Deduction) then you must solve it using this technique./nWith this understanding please complete the following questions: 1. Theory A Kirby string is defined recursively as follows: i) (^o^) is a Kirby string. ii) (ToT) is a Kirby string. iii) if A is a Kirby string, then <A> is a Kirby string. That is concatenated with A concatenated with >. iv) if A, B, C are all Kirby strings, then A...B...C is a Kirby string. That is A concatenated with ... concatenated with B concatenated with ... concatenated with C. v) Nothing else is a Kirby string. a) Give a Kirby string that requires the use of rule iv, but such that K₁, K₂, K3 are all different Kirby strings. Explain how to derive your Kirby string using the rules given. b) Disprove the following statement: Every Kirby string has the o character in the centre of the Kirby string. c) Prove by structural induction that every Kirby string has an odd number of characters.

3. Assessment The following questions were given to Chat GPT. Determine what's correct, fix all the mistakes, and assign a letter grade: A,B,C,D,F. Rainbow sequences are a special flavor of infinite sequences. They are defined inductively as follows. i) The infinite sequence of increasing positive odd integers (1,3,5,7,9,11,...) is a rainbow sequence. ii) If (t₁, t2, t3,ti,) is a rainbow sequence, then (t₁, t2, t3,..., ti + 2,...) (the sequence just like the first one, except any single value t, is replaced with the value ti + 2) is a rainbow sequence. iii) If (t₁, t₂, t3,.,t₁) is a rainbow sequence, then (t₁, t2, t3,..., t; — 2,...) (the sequence just like the first one, except any single value ti is replaced with the value ti-2) is a rainbow sequence. a) Prove by structural induction that every rainbow sequence is an infinite se- quence only consisting of odd values. 1. Base Case: The initial rainbow sequence, (1, 3, 5, 7, 9, ...), is an infinite sequence consisting only of odd values. This is true because it consists of positive odd integers. 2. Inductive Step: Assume that a rainbow sequence, (t1, t2, t3,..., ti, ...), consists of only odd values. We need to show that the sequence obtained by applying either rule ii) or rule iii) will also consist of only odd values. • Applying rule ii): The value ti + 2 is obtained by adding 2 to an odd integer ti. Since adding 2 to an odd integer always results in an odd integer, the new sequence will still consist of only odd values. • Applying rule iii): The value ti-2 is obtained by subtracting 2 from an odd integer ti. Again, subtracting 2 from an odd integer always results in an odd integer, so the new sequence will still consist of only odd values. Therefore, based on the base case and the inductive step, we can conclude that every rainbow sequence is an infinite sequence consisting only of odd values./nb) Show that it is not the case that all infinite sequences consisting of only odd values are rainbow sequences. In other words, disprove the converse of part b). c) To disprove the converse of part b) and show that not all infinite sequences consisting only of odd values are rainbow sequences, we can provide a counterexample. Consider the infinite sequence (1, 3, 5, 7, 9, 11, ...). This sequence consists only of odd values, but it does not follow the rules of rainbow sequences. According to the rainbow sequence rules, the values in the sequence must be obtained by either adding 2 or subtracting 2 from the previous values. In the provided sequence, the difference between consecutive terms is always 2. However, in a rainbow sequence, the difference between consecutive terms alternates between 2 and 0. Therefore, the sequence (1, 3, 5, 7, 9, 11, ...) consisting only of odd values is not a rainbow sequence, disproving the converse of part b).

Report the output values of the model for the following 4 test data points (use the values at iteration 30, assuming that iteration 0 denotes the beginning) Data point 1 = (-1,-1,-1) Data point 2 = (0.1, 0.2.0.3) Data point 3 = (0.5, 0.5, 0.5) Data point 4 = (1.5, 2, 0.5)

The main objective of this assignment is to offer students hands-on experience with neural networks using TensorFlow Playground, an interactive web-based tool. Through practical experimentation, you will explore various aspects of neural network design, including different architectures, activation functions, and hyperparameters. The goal is to help you grasp how these choices impact the neural network's behavior and performance. The assignment comprises three main questions, each focusing on distinct datasets and configurations. For each experiment, you are required to provide details of the parameters used (if it's not specified as a fix amount in the question), training and testing loss values, the number of epochs needed to reach the desired outcome (loss below 0.15), or the number of epochs when the loss stabilizes. Additionally, you are encouraged to include an image of their best output, which may be cropped to show the losses. For all questions, you should evaluate the experiments using learning rates of 0.01, 0.1, 0.3, 1, and 3, and you are expected to choose the most effective learning rate and comment on any intriguing behavior observed during the experiments (ex: I didn't observe significant differences when changing the learning rate from 0.01 to 0.1. However, upon setting it to 0.3, the model exhibited faster boundary identification, but when I changed it to 3 ...)./nQ1. Create a neural network with following setup (initial configuration): (8/20 Points) • Data: Dataset: Gausian Ratio of training to test data: 60% 。 Noise: 20 。 Batch size: 10 Features: X1, X2 Number of hidden layers: 0 1.1. Experiment with the linear activation function and choose a learning rate that suits your preference. Evaluate whether this network configuration can effectively find the decision boundary for the given dataset. Present your findings and results. (1 Point) 1.2. Switch the dataset to "Exclusive Or" while keeping all other parameters unchanged. Execute the model and assess its capability to identify the decision boundary. Provide reasons for why it can or cannot find the decision boundary. (1 Point) 1.3. In your experimentation, incorporate the sigmoid and tanh activation functions separately. Determine whether these network structures can successfully discover the decision boundary. (1 Point) 1.4. Introduce an additional hidden layer with two neurons to the network, utilizing both the tanh and sigmoid activation functions. Report the outcome and describe any observed effects resulting from this modification. (1 Point) 1.5. Increase the number of neurons in the hidden layer and elaborate on the noticeable changes you observe in the model's performance. (1 Point)/n1.6. Remove the neurons added in the previous step, and instead, add another hidden layer containing two neurons (resulting in two hidden layers, each with two neurons). Provide a detailed report on the results obtained and offer explanations for any significant observations. (1 Point) 1.7. Remove all the hidden layers and select the linear activation function. use engineered features as follows: 1.7.1. Use X and X2 and report your result. (1 Point) 1.7.2. Use X1 X2 and report your result. (1 Point)