Deep learning: 18 MCQs with answers

ReLU

ReLU (Rectified Linear Unit) outputs zero for negative inputs and the input itself for positive inputs, introducing non-linearity.

To measure how well the model's predictions match the true labels

The loss function quantifies the difference between predicted and actual values, guiding the optimization process.

Gradient descent

Gradient descent iteratively adjusts weights in the direction that reduces the loss, using gradients computed via backpropagation.

One complete pass over the entire training dataset

An epoch is a full cycle through the training data, allowing the model to see each sample once.

It controls the step size of weight updates.

The learning rate scales the gradient, determining how much weights are adjusted in each iteration.

Binary cross-entropy

Binary cross-entropy measures the difference between two probability distributions for binary outcomes.

To introduce non-linearity into the model

Without non-linear activations, the network would be a linear transformation, unable to learn complex patterns.

Deep learning models have more than one hidden layer.

Deep learning models are characterized by having multiple hidden layers, enabling them to learn hierarchical features.

It converts raw scores into probabilities that sum to 1.

Softmax takes a vector of raw scores and outputs a probability distribution over classes, with all probabilities summing to 1.

Dropout

Dropout randomly deactivates a fraction of neurons during each forward pass, forcing the network to learn redundant representations and reducing overfitting.

They automatically learn spatial hierarchies of features.

CNNs use convolutional layers to capture local patterns and build hierarchical representations, making them highly effective for images.

Recurrent Neural Network (RNN)

RNNs have recurrent connections that allow information to persist, making them suitable for sequential data.

Using ReLU activation functions

ReLU and its variants help maintain gradient flow because their derivative is constant for positive inputs, avoiding saturation.

To stabilize and accelerate training by normalizing layer inputs

Batch normalization normalizes the inputs of each layer to have zero mean and unit variance, which helps training converge faster.

RNNs have connections that form cycles, allowing memory of previous inputs.

RNNs have recurrent connections that create loops, enabling them to maintain a hidden state that captures information from past time steps.

True

Deep learning models typically need large datasets to learn complex patterns and avoid overfitting.

False

Sigmoid saturates for large positive/negative inputs, causing gradients to vanish; ReLU is often preferred.

Gradients become extremely small, preventing weights from updating effectively.

In deep networks, gradients can diminish exponentially as they are backpropagated, leading to negligible weight updates in early layers.