Modeling 1: Overview and linear regression in R¶
Intro and Objectives¶
One of the workhorses of statistical predictive modeling is the family of linear models. We’ll do things like multiple linear regression for numeric predictions and logistic regression as a classifier for binary response variables. We’ll use these relatively simple models as a way to also learn about important modeling topics such as partitioning data into training and test sets, model training, validation and diagnostics. We’ll also use regression to introduce the notion of parameter estimation, error metrics for assessing model fit and for comparing candidate models against each other. These topics underlie all of statistical learning algorithms.
Readings¶
PDSwR - Ch 5, 7.1
RforE - Ch 19 and 21
ISLR - Ch 1-3 and 4.1-4.3
An Introduction to Statistical Learning with applications in R
This is an outstanding book, which is available as a free pdf. ISLR covers the main statistical learning topics at a nice introductory level. There is a more mathy version that preceded it called Elements of Statistical Learning. It also has a freely available pdf.
I will be listing the associated reading that you can do from ISLR as we explore various statistical learning topics - starting with linear regression.
In-depth introduction to machine learning in 15 hours of expert videos - The ISLR authors have created a whole series of video lectures based on their book.
Downloads and other resources¶
If you are rusty on statistics, there’s a really good OpenIntro Stats book available as a free online book or you can pay what you want for a paperback copy. It includes R based material.
Activities¶
We are going to work through a series of tutorials exploring the topic of building, using and evaluating predictive linear regression models.
SCREENCAST - Overview of statistical modeling session (5:32)
- 0_ProbDists_notes.Rmd
random variate generation
probability distribution computations
- 1_LinRegModels_notes.Rmd
basics of correlation
simple linear regression
- 2_MultLinRegModels_notes.Rmd
dealing with missing data
partition into training and test datasets
SCREENCAST - MLR - Part 1: imputation of missing values (5:39)
SCREENCAST - MLR - Part 2: partition into train and test (2:47)
SCREENCAST - MLR - Part 4: fitting simple additive models (6:36)
SCREENCAST - MLR - Part 5: fitting models with interaction terms (4;14)
- 3_ComparingCompetingModels_1_notes.Rmd
building and fitting linear models
interactions terms
using linear models for prediction
- 4_ComparingCompetingModels_2_notes.Rmd
k-crossfold validation
comparison of performance on test data
SCREENCAST - Comparing models - cross validation and predictive accuracy (23:59)
- 5_ResidualAnalysis_notes.Rmd
normality of residuals
constantcy of variance
- 6_MLB_regression_modeling_notes.Rmd
build the best model you can
Explore (OPTIONAL)¶
Regression modeling¶
STAT 501 - Regression Modeling (Penn State Univ online) - Penn State has a bunch of free online resources for learning stats.
Regression modeling textbooks
Regression and Other Stories (Gelman, Hill, Vehtari) Hot off the press. This book is about applied regression, uses R, and includes things like Bayesian approaches and using simulation.
Applied Linear Statistical Models (Michael Kutner, Christopher Nachtsheim, John Neter , William Li) - A classic text. Well worth owning and studying.
Linear Models with R (Faraway) A good R-centric book on linear modeling.
Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis (Harrel) - I took a workshop given by Frank Harrel and have never looked at regression modeling quite the same. This is a very modern, more advanced look at applied regression modeling. Frank Harrel is a big “reproducible analysis” advocate, uses R, and is the author of several R packages.
- StatQuest YouTube Channel - Josh Starmer
General data science¶
Field Guide to Data Science - Booz, Allen, Hamilton (one of the big consulting firms)
