Computer Science: Machine Learning Basics

What is the primary difference between supervised and unsupervised learning?

Supervised learning uses labeled datasets to train algorithms to classify data or predict outcomes, whereas unsupervised learning analyzes and clusters unlabeled datasets to find hidden patterns without human intervention.

In logistic regression, what function is used to map predicted values to probabilities between 0 and 1?

The Sigmoid function (or Logistic function), defined as f(x) = 1 / (1 + e^-x), is used to squash any real-valued number into a range between 0 and 1.

Define overfitting in the context of machine learning model performance.

Overfitting occurs when a model learns the noise and specific details in the training data too well, resulting in high accuracy on training data but poor generalization to new, unseen data.

What does the 'Accuracy' metric represent in a classification task?

Accuracy is the ratio of correctly predicted observations to the total observations, calculated as (TP + TN) / (TP + TN + FP + FN).

Is K-Means clustering a supervised or unsupervised learning algorithm?

K-Means is an unsupervised learning algorithm because it groups data points into clusters based on feature similarity without using pre-defined target labels.

Why is a validation set used separately from the test set during model development?

The validation set is used to tune hyperparameters and perform model selection, while the test set is reserved for a final, unbiased evaluation of the chosen model's performance.

Explain the Bias-Variance trade-off regarding model complexity.

As model complexity increases, bias typically decreases (better fit to training data) but variance increases (higher sensitivity to small fluctuations), leading to a 'sweet spot' that minimizes total error.

How does L1 regularization (Lasso) differ from L2 regularization (Ridge) in its effect on model weights?

L1 regularization adds the absolute value of coefficients to the loss function, often resulting in sparse models where some weights are exactly zero, while L2 adds the squared magnitude and shrinks weights toward zero without eliminating them.

In a binary classification problem, what is the trade-off between Precision and Recall?

Increasing the threshold for classification usually increases precision (fewer false positives) but decreases recall (more false negatives), meaning you are more certain of your positive predictions but miss more actual positive cases.

What is the core mechanism behind a Random Forest model?

Random Forest is an ensemble learning method that builds multiple decision trees using bagging (bootstrap aggregating) and feature randomness, then averages their predictions to reduce variance and prevent overfitting.