3.5) Which of the following statements is/are true?

Fixing an optimal learning rate is not a trivial task as setting it too high, might lead it to

diverge and if it is set too low, it might converge but could be extremely slow and time-

consuming.

Setting up an initial learning rate and then reducing the learning rate when training

plateaus increases the chances of reaching good solution than keeping the learning rate

constant through-out the training.

Learning rate scheduling such as exponential, piece-wise constant and step-decay, in

which the learning rates are reduced over the epochs based on the respective schedules,

would always result in better performance when compare to a schedule which increases

and then decrease the learning rate over the epochs, as increasing the learning rate

initially could lead to divergence.