Due Friday 2017-November-14.

Big picture

(We started this in class during the lecture.)

You can use Make, Remake or (if you feel the urge for something more) a combinantion of Make and Remake.

Templates you can follow

More detailed instructions (optional)

If you don’t feel like dreaming up your own thing, here’s a Gapminder blueprint that is a minimal but respectable way to complete the assignment. You are welcome to remix R code already written by someone in this class, but do credit/link appropriately, e.g. in comments.

Jennifer Bryan has provided a template, using a different dataset, 01_justR, that should help make this concrete.

Download the data

Download the raw data for our example, gapminder.tsv.

  • Option 1: via an R script using download.file

    download.file("https://raw.githubusercontent.com/jennybc/gapminder/master/inst/gapminder.tsv", destfile="gapminder.tsv")
  • Option 2: in a shell script using curl or wget.

    curl -o gapminder.tsv https://raw.githubusercontent.com/jennybc/gapminder/master/inst/gapminder.tsv
    wget https://raw.githubusercontent.com/jennybc/gapminder/master/inst/gapminder.tsv

Perform exploratory analyses

  • Bring the data in as data frame.
  • Save a couple descriptive plots to file with highly informative names.
  • Reorder the continents based on life expectancy. You decide the details.
  • Sort the actual data in a deliberate fashion. You decide the details, but this should at least implement your new continent ordering.
  • Write the Gapminder data to file(s), for immediate and future reuse.

Perform statistical analyses

  • Import the data created in the first script.
  • Make sure your new continent order is still in force. You decide the details.
  • Fit a linear regression of life expectancy on year within each country. Write the estimated intercepts, slopes, and residual error variance (or sd) to file. The R package broom may be useful here.
  • Find the 3 or 4 “worst” and “best” countries for each continent. You decide the details.

Generate figures

Create a figure for each continent, and write one file per continent, with an informative name. The figure should give scatterplots of life expectancy vs. year, faceting on country, fitted line overlaid.

Automate the pipeline

Write a master R script that simply source()s the three scripts, one after the other. Tip: you may want a second “clean up / reset” script that deletes all the output your scripts leave behind, so you can easily test and refine your strategy, i.e. without repeatedly deleting stuff “by hand”. You can run the master script or the cleaning script from a shell with Rscript.

Render your RMarkdown report generating Markdown and HTML using rmarkdown::render.

  • To render an RMarkdown report and emulate RStudio’s “Knit HTML” button, use rmarkdown::render('myAwesomeReport.rmd')
  • To render an R script and emulate RStudio’s “Compile Notebook” button, use rmarkdown::render('myAwesomeScript.R')

Write a Makefile to automate your pipeline using make. See the Links section below for help. Also demonstrated in the example 02_rAndMake and in the example 03_knitWithoutRStudio

  • To run an R script use Rscript myAwesomeScript.R
  • To render an RMarkdown report, use Rscript -e "rmarkdown::render('myAwesomeReport.rmd')"
  • To render an R script, use Rscript -e "rmarkdown::render('myAwesomeScript.R')"
  • See the Makefile in 03_knitWithoutRStudio to see these commands in action

Provide a link to a README.md page that explains how your pipeline works and links to the remaining files. Your peers and the TAs should be able to go to this landing page and re-run your analysis quickly and easily.

Consider including an image showing a graphical view (the dependency diagram) of your pipeline using makefile2graph. On Mac or Linux you can install makefile2graph using Homebrew or Linuxbrew with the command brew install makefile2graph.

I want to aim higher!

Follow the basic Gapminder blueprint above, but find a different data aggregation task, different panelling/faceting emphasis, focus on different variables, etc.

Use non-Gapminder data – like the singer data or your own?

This means you’ll need to spend more time on data cleaning and sanity checking. You will probably have an entire script (or more!) devoted to data prep. Examples:

Experiment with running R code saved in a script from within R Markdown. Here’s some official documentation on code externalization.

Embed pre-existing figures in an R Markdown document, i.e. an R script creates the figures, then the report incorporates them. General advice on writing figures to file is here. See an example of this in an R Markdown file in one of the examples.

Import pre-existing data in an R Markdown document, then format nicely as a table.

Use Pandoc and/or LaTeX to explore new territory in document compilation. You could use Pandoc as an alternative to rmarkdown (or knitr) for Markdown to HTML conversion; you’d still use rmarkdown for conversion of R Markdown to Markdown. You would use LaTeX to get PDF output from Markdown.

Authors

Written mostly by Shaun Jackman and Jenny Bryan with a little edit from Giulio Dalla Riva.

Peer Review

The peer review is ready and is due November 17, 2017 (before midnight)! Here’s what you’ll need to do:

  1. Find your github username in the table below. If it’s not there, let Giulio know! Slack me @giulio.
  2. Add the people who will be giving you a review as collaborators to the repo containing your homework submission.
  3. Give a review of this homework for the two people you’ve been assigned to. There should be an issue in their repo titled something like hw0x ready for grading – put your review in there as a comment.
    • If there is no such issue, make one! (in their repo)
    • Please don’t mix up your reviewers and reviewees!

** If you were assigned to a MISSING repo, please contact me @gvdr **

** If you did not yet communicate me your github handle, please do **

Check out the guidelines for giving a peer review.

Your_github Instructions
abishekarun Please add xinmiaow and wswade2 as collaborators to your repo containing hw07. Please review the hw07 submission of cheungamanda and sepkamal.
Alexander-Morin Please add arsbar24 and peterwhitman as collaborators to your repo containing hw07. Please review the hw07 submission of ZimingY and margotgunning.
AnhKhoaVo Please add emwest and suminwei2772 as collaborators to your repo containing hw07. Please review the hw07 submission of zxkathy and xinyaofan.
arsbar24 Please add emilymistick and marbazua as collaborators to your repo containing hw07. Please review the hw07 submission of peterwhitman and Alexander-Morin.
arthursunbao Please add vanflad and swynes as collaborators to your repo containing hw07. Please review the hw07 submission of emilymistick and marbazua.
bcahn7 Please add vibudh2209 and qiaoyuet as collaborators to your repo containing hw07. Please review the hw07 submission of CassKon and mlawre01.
CassKon Please add qiaoyuet and bcahn7 as collaborators to your repo containing hw07. Please review the hw07 submission of mlawre01 and mylinhthibodeau.
cheungamanda Please add wswade2 and abishekarun as collaborators to your repo containing hw07. Please review the hw07 submission of sepkamal and Kozp.
dorawyy Please add Kozp and yidie as collaborators to your repo containing hw07. Please review the hw07 submission of emwest and suminwei2772.
emilymistick Please add swynes and arthursunbao as collaborators to your repo containing hw07. Please review the hw07 submission of marbazua and arsbar24.
emwest Please add yidie and dorawyy as collaborators to your repo containing hw07. Please review the hw07 submission of suminwei2772 and AnhKhoaVo.
farihakhan Please add xinyaofan and gbraich as collaborators to your repo containing hw07. Please review the hw07 submission of ilgan and yuanjisun.
gbraich Please add zxkathy and xinyaofan as collaborators to your repo containing hw07. Please review the hw07 submission of farihakhan and ilgan.
hannahdxz Please add yuanjisun and menglinzhou as collaborators to your repo containing hw07. Please review the hw07 submission of wenzhengzzz and Mathnstein.
heathersummers Please add santiagodr and mattsada as collaborators to your repo containing hw07. Please review the hw07 submission of SonjaS1 and vibudh2209.
HScheiber Please add yeonukkim and hsmohammed as collaborators to your repo containing hw07. Please review the hw07 submission of santiagodr and mattsada.
hsmohammed Please add Mathnstein and yeonukkim as collaborators to your repo containing hw07. Please review the hw07 submission of HScheiber and santiagodr.
ilgan Please add gbraich and farihakhan as collaborators to your repo containing hw07. Please review the hw07 submission of yuanjisun and menglinzhou.
jmurthy12 Please add yanchaoluo and Shirlett as collaborators to your repo containing hw07. Please review the hw07 submission of juansbr7 and KateJohnson.
juansbr7 Please add Shirlett and jmurthy12 as collaborators to your repo containing hw07. Please review the hw07 submission of KateJohnson and vanflad.
KateJohnson Please add jmurthy12 and juansbr7 as collaborators to your repo containing hw07. Please review the hw07 submission of vanflad and swynes.
Kozp Please add cheungamanda and sepkamal as collaborators to your repo containing hw07. Please review the hw07 submission of yidie and dorawyy.
marbazua Please add arthursunbao and emilymistick as collaborators to your repo containing hw07. Please review the hw07 submission of arsbar24 and peterwhitman.
margotgunning Please add Alexander-Morin and ZimingY as collaborators to your repo containing hw07. Please review the hw07 submission of xinmiaow and wswade2.
Mathnstein Please add hannahdxz and wenzhengzzz as collaborators to your repo containing hw07. Please review the hw07 submission of yeonukkim and hsmohammed.
mattsada Please add HScheiber and santiagodr as collaborators to your repo containing hw07. Please review the hw07 submission of heathersummers and SonjaS1.
menglinzhou Please add ilgan and yuanjisun as collaborators to your repo containing hw07. Please review the hw07 submission of hannahdxz and wenzhengzzz.
mlawre01 Please add bcahn7 and CassKon as collaborators to your repo containing hw07. Please review the hw07 submission of mylinhthibodeau and rainerlempert.
mylinhthibodeau Please add CassKon and mlawre01 as collaborators to your repo containing hw07. Please review the hw07 submission of rainerlempert and Tangjiahui26.
peterwhitman Please add marbazua and arsbar24 as collaborators to your repo containing hw07. Please review the hw07 submission of Alexander-Morin and ZimingY.
qiaoyuet Please add SonjaS1 and vibudh2209 as collaborators to your repo containing hw07. Please review the hw07 submission of bcahn7 and CassKon.
rainerlempert Please add mlawre01 and mylinhthibodeau as collaborators to your repo containing hw07. Please review the hw07 submission of Tangjiahui26 and rishadhabib.
rishadhabib Please add rainerlempert and Tangjiahui26 as collaborators to your repo containing hw07. Please review the hw07 submission of vmichalowski and yanchaoluo.
santiagodr Please add hsmohammed and HScheiber as collaborators to your repo containing hw07. Please review the hw07 submission of mattsada and heathersummers.
sepkamal Please add abishekarun and cheungamanda as collaborators to your repo containing hw07. Please review the hw07 submission of Kozp and yidie.
Shirlett Please add vmichalowski and yanchaoluo as collaborators to your repo containing hw07. Please review the hw07 submission of jmurthy12 and juansbr7.
SonjaS1 Please add mattsada and heathersummers as collaborators to your repo containing hw07. Please review the hw07 submission of vibudh2209 and qiaoyuet.
suminwei2772 Please add dorawyy and emwest as collaborators to your repo containing hw07. Please review the hw07 submission of AnhKhoaVo and zxkathy.
swynes Please add KateJohnson and vanflad as collaborators to your repo containing hw07. Please review the hw07 submission of arthursunbao and emilymistick.
Tangjiahui26 Please add mylinhthibodeau and rainerlempert as collaborators to your repo containing hw07. Please review the hw07 submission of rishadhabib and vmichalowski.
vanflad Please add juansbr7 and KateJohnson as collaborators to your repo containing hw07. Please review the hw07 submission of swynes and arthursunbao.
vibudh2209 Please add heathersummers and SonjaS1 as collaborators to your repo containing hw07. Please review the hw07 submission of qiaoyuet and bcahn7.
vmichalowski Please add Tangjiahui26 and rishadhabib as collaborators to your repo containing hw07. Please review the hw07 submission of yanchaoluo and Shirlett.
wenzhengzzz Please add menglinzhou and hannahdxz as collaborators to your repo containing hw07. Please review the hw07 submission of Mathnstein and yeonukkim.
wswade2 Please add margotgunning and xinmiaow as collaborators to your repo containing hw07. Please review the hw07 submission of abishekarun and cheungamanda.
xinmiaow Please add ZimingY and margotgunning as collaborators to your repo containing hw07. Please review the hw07 submission of wswade2 and abishekarun.
xinyaofan Please add AnhKhoaVo and zxkathy as collaborators to your repo containing hw07. Please review the hw07 submission of gbraich and farihakhan.
yanchaoluo Please add rishadhabib and vmichalowski as collaborators to your repo containing hw07. Please review the hw07 submission of Shirlett and jmurthy12.
yeonukkim Please add wenzhengzzz and Mathnstein as collaborators to your repo containing hw07. Please review the hw07 submission of hsmohammed and HScheiber.
yidie Please add sepkamal and Kozp as collaborators to your repo containing hw07. Please review the hw07 submission of dorawyy and emwest.
yuanjisun Please add farihakhan and ilgan as collaborators to your repo containing hw07. Please review the hw07 submission of menglinzhou and hannahdxz.
ZimingY Please add peterwhitman and Alexander-Morin as collaborators to your repo containing hw07. Please review the hw07 submission of margotgunning and xinmiaow.
zxkathy Please add suminwei2772 and AnhKhoaVo as collaborators to your repo containing hw07. Please review the hw07 submission of xinyaofan and gbraich.