Data Science Core: quick revision, 18 bites

Data Science vs. Data Engineering vs. Analytics

Know the roles that turn data into decisions.

**Data Science** extracts insights and builds models from data. **Data Engineering** builds the infrastructure to collect, store, and process data. **Analytics** applies statistical and computational methods to describe and interpret data for decision-making.

Takeaway. Data engineering supports analytics, which is a core part of data science.

Reporting vs. Analytics

Reports show what happened; analytics explains why.

Takeaway. Reporting is descriptive; analytics goes deeper to diagnose and predict.

The Analytics Process

A structured path from raw data to insight.

Takeaway. Analytics is a systematic process, not a single tool or technique.

Types of Analytical Techniques

Four lenses to view your data.

Takeaway. Each technique answers a different question: what, why, what next, and what to do.

Traits of Big Data

Big Data is more than just volume.

Takeaway. Big Data is characterized by volume, velocity, variety, veracity, and value.

Python Data Structures for Data Science

Lists, dicts, and sets are your building blocks.

Takeaway. Choose the right structure: lists for order, dicts for mapping, sets for uniqueness.

NumPy Vectorized Operations

Speed up computations with arrays.

**NumPy** provides **ndarray** for efficient numerical computation. **Vectorized operations** apply functions element-wise without explicit loops, making code faster and more concise.

Takeaway. Vectorization leverages C-level loops for speed and readability.

Pandas DataFrames and Series

Tabular data made easy.

**Pandas** introduces two core structures: **Series** (1D labeled array) and **DataFrame** (2D table with rows and columns). DataFrames allow filtering, grouping, merging, and cleaning with intuitive methods.

Takeaway. DataFrames are the workhorse for data manipulation and cleaning in Python.

Visualization with Matplotlib and Seaborn

A picture reveals patterns.

Takeaway. Use Matplotlib for customization and Seaborn for statistical plots.

Data Types and Measurement Scales

Know your data's scale before analyzing.

Takeaway. Scale determines which statistical operations are valid.

Population vs. Sample

Sampling makes inference possible.

A **population** includes all members of a defined group. A **sample** is a subset selected to represent the population. Statistical inference uses sample statistics to estimate population parameters.

Takeaway. We analyze samples to draw conclusions about populations.

Percentiles, Deciles, Quartiles

Divide data into meaningful parts.

**Percentiles** divide data into 100 equal parts; **deciles** into 10; **quartiles** into 4. Quartiles (Q1, Q2, Q3) are key for box plots and spread analysis.

Takeaway. These measures summarize distribution position and spread.

Measures of Variation and Shape

Spread and shape tell the full story.

Takeaway. Central tendency alone is misleading; always examine variation and shape.

Probability Distributions Recap

Distributions model randomness.

Takeaway. Each distribution has a context; choose based on data type and process.

Central Limit Theorem (CLT)

The sampling distribution becomes normal.

The **Central Limit Theorem** states that the sampling distribution of the sample mean approaches a normal distribution as sample size increases, regardless of the population's distribution (given finite variance).

Takeaway. CLT justifies using normal-based inference for large samples.

Sampling Methods

How you sample matters.

Takeaway. Probabilistic sampling supports unbiased estimation and inference.

Estimation Fundamentals

Estimate population parameters from samples.

Takeaway. Estimation provides both a guess and a measure of uncertainty.

Maximum Likelihood Estimation (MLE)

Find parameters that make data most likely.

**MLE** chooses parameter values that maximize the likelihood function, i.e., the probability of observing the given sample. It yields estimators with desirable properties (consistent, efficient).

Takeaway. MLE is a principled method for parameter estimation.