# EDA with R¶

## Intro and Objectives¶

We will begin to do exploratory data analysis in R. After completing the activities in this module, you should be able to explore a dataset using:

descriptive statistics,

simple R scripts including writing your own functions,

basic (and not so basic) plots with ggplot2.

We are going to explore a dataset related to New York City condo evaluations for fiscal year 2011-2012. It was obtained from the NYC Open Data initiative - https://data.cityofnewyork.us/.

## Readings¶

RforE - Chapters 6, 7, 8

r4ds - Chapters 1-2, 10-12

PDSwR - Chapters 3, 4

## Downloads and other resources¶

Other Resources:

## Activities¶

We will work through two tutorials on EDA (with a short detour on creating user defined functions in R)

**EDA1a_summarystats_shell.Rmd**

**summarystats_4470.R**

**EDA1b_basicplots_shell.Rmd**SCREENCAST: Intro to ggplot2 (2:55)

SCREENCAST: The Grammar of Graphics underlying ggplot2 (7:24)

SCREENCAST: Histograms (5:29)

SCREENCAST: Boxplots and violin plots, Faceting and reusing plot objects (5:40)

SCREENCAST: Scatter plots (2:33)

SCREENCAST: Themes (5:40)

SCREENCAST: Density plots and other histogram alternatives (8:45)

SCREENCAST: Bar charts (6:13)

SCREENCAST: Correlation plots and Geeky fun with xkcd (6:28)

## Explore (OPTIONAL)¶

A framework for exploratory data analysis - As you browse this, there’s a “More Pages” button at the bottom. You can also download the pdf from the GitHub site.

### Data visualization¶

There’s no doubt that ggplot is awesome, but check out what can be done if you have a good grasp of base plotting in R. When I read this, it felt a bit like matplotlib, the venerable Python based plotting package.

R Graph Catalog Plot demos and code

Box plot comic - You can always count on xkcd.

Some data visualization resources from my MIS 5460: Business Analytics class

R Shiny app for visualizing NYC bike share data- Lots of interesting logistical challenges with bike share programs. EDA plays a role in gaining insight to system dynamics.

### Percentiles¶

It’s easy to get enamored with averages. They don’t tell the whole story. Look at percentiles, too.

Examples of why percentiles are important for performance monitoring by Dynatrace, Elastic, AppSignal and Optimizely.

Percentiles in PostgreSQL - this SQL flavor has implemented some nice percentile and time weighted stats functions, including approximate percentiles.

### R Markdown¶

Now that you know some basic R Markdown, you might want to dig into its capabilities a little further. This overview provided by the folks at R Studio is a good place to start. In addition, Ch 27-28 of RforE covers R Markdown, knitr, and LaTex.

Daring Fireball: Markown Syntax site - This is John Gruber’s site - he developed Markdown.

Tips and tricks for working with images and figures in R Markdown documents - Very comprehensive post on the ins and outs of working with both external images as well as R generated images within R Markdown documents.