Too large of a step results in extreme results, in which the screen is either all orange or all blue. Too small of a step results in an almost blank screen, in which the steps are too small to make real progress in diminishing the loss function.

A small amount of neurons and hidden layers can help with large steps while large amounts of neurons and hidden layers speed up the process for small steps. The large amounts of neurons and hidden layers also make the learning process from medium sized steps quicker.