Due Friday 2016-November-18.

Big picture

Templates you can follow

Please just tell me what to do

If you don’t feel like dreaming up your own thing, here’s a Gapminder blueprint that is a minimal but respectable way to complete the assignment. You are welcome to remix R code already written by someone, student or JB, in this class, but credit/link appropriately, e.g. in comments.

JB has provided a template, using a different dataset, 01_justR, that should help make this concrete.

Download the data

Download the raw data for our example, gapminder.tsv.

  • Option 1: via an R script using download.file

    download.file("https://raw.githubusercontent.com/jennybc/gapminder/master/inst/gapminder.tsv", destfile="gapminder.tsv")
  • Option 2: in a shell script using curl or wget.

    curl -o gapminder.tsv https://raw.githubusercontent.com/jennybc/gapminder/master/inst/gapminder.tsv
    wget https://raw.githubusercontent.com/jennybc/gapminder/master/inst/gapminder.tsv

Perform exploratory analyses

  • Bring the data in as data frame.
  • Save a couple descriptive plots to file with highly informative names.
  • Reorder the continents based on life expectancy. You decide the details.
  • Sort the actual data in a deliberate fashion. You decide the details, but this should at least implement your new continent ordering.
  • Write the Gapminder data to file(s), for immediate and future reuse.

Perform statistical analyses

  • Import the data created in the first script.
  • Make sure your new continent order is still in force. You decide the details.
  • Fit a linear regression of life expectancy on year within each country. Write the estimated intercepts, slopes, and residual error variance (or sd) to file. The R package broom may be useful here.
  • Find the 3 or 4 “worst” and “best” countries for each continent. You decide the details.

Generate figures

Create a figure for each continent, and write one file per continent, with an informative name. The figure should give scatterplots of life expectancy vs. year, faceting on country, fitted line overlaid.

Automate the pipeline

Write a master R script that simply source()s the three scripts, one after the other. Tip: you may want a second “clean up / reset” script that deletes all the output your scripts leave behind, so you can easily test and refine your strategy, i.e. without repeatedly deleting stuff “by hand”. You can run the master script or the cleaning script from a shell with Rscript.

Render your RMarkdown report generating Markdown and HTML using rmarkdown::render.

  • To render an RMarkdown report and emulate RStudio’s “Knit HTML” button, use rmarkdown::render('myAwesomeReport.rmd')
  • To render an R script and emulate RStudio’s “Compile Notebook” button, use rmarkdown::render('myAwesomeScript.R')

Write a Makefile to automate your pipeline using make. See the Links section below for help. Also demonstrated in the example 02_rAndMake and in the example 03_knitWithoutRStudio

  • To run an R script use Rscript myAwesomeScript.R
  • To render an RMarkdown report, use Rscript -e "rmarkdown::render('myAwesomeReport.rmd')"
  • To render an R script, use Rscript -e "rmarkdown::render('myAwesomeScript.R')"
  • See the Makefile in 03_knitWithoutRStudio to see these commands in action

Provide a link to a README.md page that explains how your pipeline works and links to the remaining files. Your peers and the TAs should be able to go to this landing page and re-run your analysis quickly and easily.

Consider including an image showing a graphical view (the dependency diagram) of your pipeline using makefile2graph. On Mac or Linux you can install makefile2graph using Homebrew or Linuxbrew with the command brew install makefile2graph.

I want to aim higher!

Follow the basic Gapminder blueprint above, but find a different data aggregation task, different panelling/faceting emphasis, focus on different variables, etc.

Use non-Gapminder data – like maybe the candy survey?

This means you’ll need to spend more time on data cleaning and sanity checking. You will probably have an entire script (or more!) devoted to data prep. Examples:

Experiment with running R code saved in a script from within R Markdown. Here’s some official documentation on code externalization.

Embed pre-existing figures in an R Markdown document, i.e. an R script creates the figures, then the report incorporates them. General advice on writing figures to file is here. See an example of this in an R Markdown file in one of the examples.

Import pre-existing data in an R Markdown document, then format nicely as a table.

Use Pandoc and/or LaTeX to explore new territory in document compilation. You could use Pandoc as an alternative to rmarkdown (or knitr) for Markdown to HTML conversion; you’d still use rmarkdown for conversion of R Markdown to Markdown. You would use LaTeX to get PDF output from Markdown.


Written by Shaun Jackman and Jenny Bryan.