1 Learning Objectives

1.1 Introduction

This is a tutorial to get you started with package development in R, meant to accompany two lectures of STAT 547. It’s similar (and somewhat based off of) Jenny’s tutorial. Jenny’s is more comprehensive, so is useful as a learning resource. This tutorial focusses more exclusively on the RStudio workflow to building packages.

Why make a package in R? Here are just a few big reasons:

  • Built-in checks that your functions are working and are sensible.
  • Easy way to store and load your data.
  • Allows for documentation of functions that you’ve written.

Think aid for a type of analysis, not an analysis itself.

I’ll try to break this up into “waves” of instruction, with work periods in between. Work on your Homework 9 if you’re ahead!

1.2 Wave 1: The bare minimum, plus bare minimum best practices

1.2.1 1.1 Bare minimum

Follow along as we make an R package called powers that contains a function square that squares its input. Let’s initiate it:

  • RStudio —> New project —> R Package
    • Initiate git (optional, but recommended).
  • Check out the files that have been created

Now, delete the man folder and R scripts (if present), and start a new R script in the R directory, called square.R. Write a function called square that squares its input.

Build the package:

  • Build and Reload, or in newer versions of RStudio, Install and Restart.
    • This compiles the package, and loads it.
    • Try leaving the project, do library(powers), and use the function! Pretty cool, eh?

1.2.2 1.2 Documentation

Comment above functions with #’, and use tags starting with @. Here’s documentation for the square function – copy and paste it above the square function.

#' Square a vector
#'
#' That's it -- this function just squares a vector.
#'
#' @param x The vector to be squared.
#'
#' @return A vector that is the square of \code{x}.
#'
#' @details
#' This function isn't complicated.
#'
#' And it almost certainly doesn't need two paragraphs in the "Details"
#' section!
#'
#' Here are some reasons why putting a list in this section is excessive:
#' \itemize{
#'      \item This \code{square} function is quite simple.
#'      \item There's nothing else to say about \code{square}.
#' }
#'
#' @examples
#' square(1:10)
#' square(-5)
#' @export

Note: now that we’re using roxygen2 to document our package, we need to specify which functions we wish to make viewable by the user by indicating the @export tag, as above.

Type roxygen2::roxygenize() in the console, or more simply, document() from the devtools package. Then Build and Reload/Install and Restart the package.

Notable things that have happened:

  • Your function is now documented. Check it out with ?square!
    • This happens due to the creation of an Rd file in the man folder.
  • Your NAMESPACE is populated. Contains all things you want the user to see.

1.2.3 1.3 Checking

It’s a good idea to check your package early and often to see that everything is working.

Click Check under the Build menu. It checks lots of things for you! We’ll see more examples of this.

1.2.4 1.4 Function Dependencies

Make another, more general function to compute any power:

pow <- function(x, a=2) x^a

It can go in the same R script as square, or a different one – your choice.

We’ll make square depend on pow.

Aftering Build and Reload/Install and Restarting, you’ll notice that you can’t use pow because it’s not exported. But, square still works! We call pow an internal function.

Note: you should still document your internal function! But mention that the function is internal. Users will be able to access the documentation like normal, but still won’t be able to (easily) use the function.

If you want to be able to use internal functions as a developer, but don’t want users to have (easy) access to the functions, then run load_all instead of Build and Reload/Install and Restart.

1.3 Exercise

Using at least two .R scripts:

  • Make and document pow. Try out other roxygen2 tags.
  • Make and document another function, say cube, that raises a vector to the power of 3.

Make sure at least square and cube are exported to the NAMESPACE. Optionally, pow. Make sure you Check the package often!

Finished early? Do more – work on Assignment 9, and/or try out more documentation features that comes with roxygen2.

1.4 Wave 2: More documentation, and testing

1.4.1 2.1 DESCRIPTION file

Every R package has this. It contains the package’s metadata. Let’s edit it:

  • Add a title and brief description.
    • R is picky about these! Check out the rules.
  • Add your name.
    • Use the Authors@R field instead of the default Author and Maintainer fields.
  • Pick a license.

1.4.2 2.2 Testing with testthat

We’ve already seen package Checks – this checks that the pieces of your R package are in place, and that even your examples don’t throw errors. We should not only check that our functions are working, but that they give us results that we’d expect.

The testthat package is useful for this. Initialize it in your R package by running devtools::use_testthat().

As a template, save the following script in a file called test_square.

context("Squaring non-numerics")

test_that("At least numeric values work.", {
    num_vec <- c(0, -4.6, 3.4)
    expect_identical(square(numeric(0)), numeric(0))
    expect_identical(square(num_vec), num_vec^2)
})

test_that("Logicals automatically convert to numeric.", {
    logic_vec <- c(TRUE, TRUE, FALSE)
    expect_identical(square(logic_vec), logic_vec^2)
})

Then, you can execute those tests by running devtools::test(), or clicking Build -> Test package.

These sanity checks are very important as your R package becomes more complex!

1.5 Exercise

  • Edit your DESCRIPTION file. Make sure things run smoothly when you Check the R package.
  • Move the numeric(0) test to a new file, under a new context.
  • Extend the tests to include your cube function.

Done early? Work on your Homework 9 R package.

1.6 Wave 3: Higher-level User Documentation

1.6.1 3.1 Package Documentation

Just like we do for functions, we can make a manual (.Rd) page for our entire R package, too. For example, check out the documentation for ggplot2:

?ggplot2         # Can execute only if `ggplot2` is loaded.
package?ggplot2  # Always works.

To do so, just execute devtools::use_package_doc(). You’ll see a new R script come up with roxygen2-style documentation to NULL. Document as you’d do functions, and run roxygen2::roxygenise() or devtools::document() to generate the .Rd file.

Here’s sample documentation:

#' Convenient Computation of Powers
#'
#' Are you tired of using the power operator, \code{^} or \code{**} in R?
#' Use this package to call functions that apply common powers
#' to your vectors.
#'
#' @name powers
#' @author Vincenzo Coia
#' @note This package isn't actually meant to be serious. It's just for
#' teaching purposes.
#' @docType package

1.6.2 3.2 Vignettes

It’s a good idea to write a vignette (or several) for your R package to show how the package is useful as a whole. Documentation for individual functions don’t suffice for this purpose!

To write a vignette called "my_vignette", just run

devtools::use_vignette("my_vignette")

Some things happen automatically, but it’s up to you to modify the .Rmd document to provide adequate instruction. Change the template to suit your package. The only real “catch” to doing this is making sure the title is replaced in both instances.

Then just Knit, and then run devtools::build_vignettes() to build the vignettes.

Note: You’ll have to remove inst/doc from your .gitignore file that gets created, if you want to push it to github.

1.6.3 3.3 README

Just as most projects should have a README file in the main directory, so should an R package.

Purposes:

  • Inform someone stumbling across your project what they’ve stumbled across.
    • At a high level (like “This is an R package”), but also
    • somewhat at a lower level too, like your description file. This becomes a little redundant.
  • I like to use the README to inform developers the main workflow and spirit behind developing the package.
    • There are some things that you’d want other potential developers to know about the package as a whole, yet are irrelevant to users!

How to do it:

You could just make and edit a README.md file like normal. But you’ll probably want to briefly demonstrate some code, so you’ll need an .Rmd. Let devtools set that up for you:

devtools::use_readme_rmd()

knit and you’re done!

1.7 Exercises

Create the above three types of documentation, without looking at my version. Then compare.

Ideally, you’ll have more to document because you’ve been working on expanding this (or another) R package for Homework 9 already.

1.8 Wave 4

1.8.1 4.1 Dependencies

We can use functions from other R packages within our homemade R package, too. We need to do two things:

  • Use the syntax package_name::function_name() whenever you want to use function_name from package_name.
  • Indicate that your R package depends on package_name in the DESCRIPTION file by executing the command devtools::use_package("package_name").

There are other methods, but this is the easiest.

Example: Add ggplot2 dependency to plot the resulting computations. Do so by adding a plot to pow – change pow’s guts to the following:

res <- x^a
p <- ggplot2::qplot(x, res)
print(p)
return(res)

Note: Here’s an example of the benefits of not having your functions do too much – I only needed to change pow alone to get the changes to work for square and cube.

1.8.2 4.2 Adding data to your R package

You can store and document datasets within R packages. Here’s one useful way.

Note: This currently doesn’t seem to be present in the companion tutorial from Jenny. Check out the R Packages “data chapter” for a resource.

In the console:

  1. Store your data as R objects. Say you have two: foo and bar.
  2. Execute devtools::use_data(foo, bar).

foo and bar will be saved as .Rdata files in the new /data directory. These are available upon loading the package.

To document the data, for each object (i.e., for each of foo and bar), put roxygen2-style documentation above the character "foo" and "bar" in an R script in the /R folder.

Example:

Add tenvec to the package:

tenvec <- 1:10
devtools::use_data(tenvec)

Then make a new R script in the /R folder containing:

#' Integer vector from 1 to 10
#'
#' Self-explanatory!
#'
#' @format This (and the next) is a new tag that applies for data documentation.
#' @source Note that you shouldn't use the `@export` tag when documenting data!
"tenvec"

1.8.3 4.3 Launching your Package to GitHub

If I want to put an R package on GitHub, I typically just:

  1. Click “New” in GitHub to make a new repo. Don’t initialize with README.
  2. Follow the instructions github provides, which involves two lines to execute in the terminal.
    • Those two lines can be found here in Jenny’s Happy git book.

There is also the devtools::use_github() way – although, to me, it seems overly complicated (perhaps there’s an advantage I don’t know about). It’s just a matter of following the instructions, which are not worth demonstrating here.

1.9 Exercise

Some things you might find value in trying:

  • Make, store, and document more data – perhaps negtenvec, the negative of tenvec.
  • Give the user an option as to whether they want to display the plot or not (hint: do so via function arguments).
  • Try launching your package on github.
    • Please do make it public. R packages are meant to be viewed by the world! If it makes you feel more comfortable, put a disclaimer at the top of the README if you don’t want people to take it seriously.
LS0tCnRpdGxlOiAiU1RBVCA1NDdNIENsYXNzIE1lZXRpbmcgMDk6IFIgcGFja2FnZXMsIFBhcnQgSSIKb3V0cHV0OgogICAgaHRtbF9ub3RlYm9vazoKICAgICAgICB0b2M6IHRydWUKICAgICAgICB0aGVtZTogY2VydWxlYW4KICAgICAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKZWRpdG9yX29wdGlvbnM6IAogIGNodW5rX291dHB1dF90eXBlOiBpbmxpbmUKLS0tCgojIExlYXJuaW5nIE9iamVjdGl2ZXMKCgojIyBJbnRyb2R1Y3Rpb24KClRoaXMgaXMgYSB0dXRvcmlhbCB0byBnZXQgeW91IHN0YXJ0ZWQgd2l0aCBwYWNrYWdlIGRldmVsb3BtZW50IGluIFIsIG1lYW50IHRvIGFjY29tcGFueSB0d28gbGVjdHVyZXMgb2YgU1RBVCA1NDcuIEl0J3Mgc2ltaWxhciAoYW5kIHNvbWV3aGF0IGJhc2VkIG9mZiBvZikgW0plbm55J3MgdHV0b3JpYWxdKGh0dHA6Ly9zdGF0NTQ1LmNvbS9wYWNrYWdlczA2X2Zvb2ZhY3RvcnMtcGFja2FnZS5odG1sKS4gSmVubnkncyBpcyBtb3JlIGNvbXByZWhlbnNpdmUsIHNvIGlzIHVzZWZ1bCBhcyBhIGxlYXJuaW5nIHJlc291cmNlLiBUaGlzIHR1dG9yaWFsIGZvY3Vzc2VzIG1vcmUgZXhjbHVzaXZlbHkgb24gdGhlIFJTdHVkaW8gd29ya2Zsb3cgdG8gYnVpbGRpbmcgcGFja2FnZXMuCgpXaHkgbWFrZSBhIHBhY2thZ2UgaW4gUj8gSGVyZSBhcmUganVzdCBhIGZldyBiaWcgcmVhc29uczoKCi0gQnVpbHQtaW4gY2hlY2tzIHRoYXQgeW91ciBmdW5jdGlvbnMgYXJlIHdvcmtpbmcgYW5kIGFyZSBzZW5zaWJsZS4KLSBFYXN5IHdheSB0byBzdG9yZSBhbmQgbG9hZCB5b3VyIGRhdGEuCi0gQWxsb3dzIGZvciBkb2N1bWVudGF0aW9uIG9mIGZ1bmN0aW9ucyB0aGF0IHlvdSd2ZSB3cml0dGVuLiAKClRoaW5rIF9haWQgZm9yIGEgdHlwZSBvZiBhbmFseXNpc18sIG5vdCBhbiBhbmFseXNpcyBpdHNlbGYuIAoKSSdsbCB0cnkgdG8gYnJlYWsgdGhpcyB1cCBpbnRvICJ3YXZlcyIgb2YgaW5zdHJ1Y3Rpb24sIHdpdGggd29yayBwZXJpb2RzIGluIGJldHdlZW4uIFdvcmsgb24geW91ciBIb21ld29yayA5IGlmIHlvdSdyZSBhaGVhZCEKCiMjIFdhdmUgMTogVGhlIGJhcmUgbWluaW11bSwgcGx1cyBiYXJlIG1pbmltdW0gYmVzdCBwcmFjdGljZXMKCiMjIyAxLjEgQmFyZSBtaW5pbXVtIAoKRm9sbG93IGFsb25nIGFzIHdlIG1ha2UgYW4gUiBwYWNrYWdlIGNhbGxlZCBgcG93ZXJzYCB0aGF0IGNvbnRhaW5zIGEgZnVuY3Rpb24gYHNxdWFyZWAgdGhhdCBzcXVhcmVzIGl0cyBpbnB1dC4gTGV0J3MgaW5pdGlhdGUgaXQ6CgotIFJTdHVkaW8g4oCUPiBOZXcgcHJvamVjdCDigJQ+IFIgUGFja2FnZQogICAgLSBJbml0aWF0ZSBnaXQgKG9wdGlvbmFsLCBidXQgcmVjb21tZW5kZWQpLgotIENoZWNrIG91dCB0aGUgZmlsZXMgdGhhdCBoYXZlIGJlZW4gY3JlYXRlZAoKTm93LCBkZWxldGUgdGhlIGBtYW5gIGZvbGRlciBhbmQgUiBzY3JpcHRzIChpZiBwcmVzZW50KSwgYW5kIHN0YXJ0IGEgbmV3IFIgc2NyaXB0IGluIHRoZSBgUmAgZGlyZWN0b3J5LCBjYWxsZWQgYHNxdWFyZS5SYC4gV3JpdGUgYSBmdW5jdGlvbiBjYWxsZWQgYHNxdWFyZWAgdGhhdCBzcXVhcmVzIGl0cyBpbnB1dC4KCkJ1aWxkIHRoZSBwYWNrYWdlOgoKKiBgQnVpbGQgYW5kIFJlbG9hZGAsIG9yIGluIG5ld2VyIHZlcnNpb25zIG9mIFJTdHVkaW8sIGBJbnN0YWxsIGFuZCBSZXN0YXJ0YC4KICAgICogVGhpcyBjb21waWxlcyB0aGUgcGFja2FnZSwgYW5kIGxvYWRzIGl0LgogICAgKiBUcnkgbGVhdmluZyB0aGUgcHJvamVjdCwgZG8gYGxpYnJhcnkocG93ZXJzKWAsIGFuZCB1c2UgdGhlIGZ1bmN0aW9uISBQcmV0dHkgY29vbCwgZWg/CgojIyMgMS4yIERvY3VtZW50YXRpb24gCgpDb21tZW50IGFib3ZlIGZ1bmN0aW9ucyB3aXRoIGAj4oCZYCwgYW5kIHVzZSB0YWdzIHN0YXJ0aW5nIHdpdGggYEBgLiBIZXJlJ3MgZG9jdW1lbnRhdGlvbiBmb3IgdGhlIGBzcXVhcmVgIGZ1bmN0aW9uIC0tIGNvcHkgYW5kIHBhc3RlIGl0IGFib3ZlIHRoZSBgc3F1YXJlYCBmdW5jdGlvbi4KCmBgYHtyfQojJyBTcXVhcmUgYSB2ZWN0b3IKIycKIycgVGhhdCdzIGl0IC0tIHRoaXMgZnVuY3Rpb24ganVzdCBzcXVhcmVzIGEgdmVjdG9yLgojJwojJyBAcGFyYW0geCBUaGUgdmVjdG9yIHRvIGJlIHNxdWFyZWQuCiMnCiMnIEByZXR1cm4gQSB2ZWN0b3IgdGhhdCBpcyB0aGUgc3F1YXJlIG9mIFxjb2Rle3h9LgojJwojJyBAZGV0YWlscwojJyBUaGlzIGZ1bmN0aW9uIGlzbid0IGNvbXBsaWNhdGVkLgojJwojJyBBbmQgaXQgYWxtb3N0IGNlcnRhaW5seSBkb2Vzbid0IG5lZWQgdHdvIHBhcmFncmFwaHMgaW4gdGhlICJEZXRhaWxzIgojJyBzZWN0aW9uIQojJwojJyBIZXJlIGFyZSBzb21lIHJlYXNvbnMgd2h5IHB1dHRpbmcgYSBsaXN0IGluIHRoaXMgc2VjdGlvbiBpcyBleGNlc3NpdmU6CiMnIFxpdGVtaXplewojJyAgICAgIFxpdGVtIFRoaXMgXGNvZGV7c3F1YXJlfSBmdW5jdGlvbiBpcyBxdWl0ZSBzaW1wbGUuCiMnICAgICAgXGl0ZW0gVGhlcmUncyBub3RoaW5nIGVsc2UgdG8gc2F5IGFib3V0IFxjb2Rle3NxdWFyZX0uCiMnIH0KIycKIycgQGV4YW1wbGVzCiMnIHNxdWFyZSgxOjEwKQojJyBzcXVhcmUoLTUpCiMnIEBleHBvcnQKYGBgCgoKTm90ZTogbm93IHRoYXQgd2XigJlyZSB1c2luZyBgcm94eWdlbjJgIHRvIGRvY3VtZW50IG91ciBwYWNrYWdlLCB3ZSBuZWVkIHRvIHNwZWNpZnkgd2hpY2ggZnVuY3Rpb25zIHdlIHdpc2ggdG8gbWFrZSB2aWV3YWJsZSBieSB0aGUgdXNlciBieSBpbmRpY2F0aW5nIHRoZSBgQGV4cG9ydGAgdGFnLCBhcyBhYm92ZS4KClR5cGUgYHJveHlnZW4yOjpyb3h5Z2VuaXplKClgIGluIHRoZSBjb25zb2xlLCBvciBtb3JlIHNpbXBseSwgYGRvY3VtZW50KClgIGZyb20gCnRoZSBgZGV2dG9vbHNgIHBhY2thZ2UuIFRoZW4gYEJ1aWxkIGFuZCBSZWxvYWRgL2BJbnN0YWxsIGFuZCBSZXN0YXJ0YCB0aGUgcGFja2FnZS4KCk5vdGFibGUgdGhpbmdzIHRoYXQgaGF2ZSBoYXBwZW5lZDoKCiogWW91ciBmdW5jdGlvbiBpcyBub3cgZG9jdW1lbnRlZC4gQ2hlY2sgaXQgb3V0IHdpdGggYD9zcXVhcmVgIQogICAgKiBUaGlzIGhhcHBlbnMgZHVlIHRvIHRoZSBjcmVhdGlvbiBvZiBhbiBgUmRgIGZpbGUgaW4gdGhlIGBtYW5gIGZvbGRlci4KKiBZb3VyIE5BTUVTUEFDRSBpcyBwb3B1bGF0ZWQuIENvbnRhaW5zIGFsbCB0aGluZ3MgeW91IHdhbnQgdGhlIHVzZXIgdG8gc2VlLgoKIyMjIDEuMyBDaGVja2luZyAKCkl0J3MgYSBnb29kIGlkZWEgdG8gYGNoZWNrYCB5b3VyIHBhY2thZ2UgZWFybHkgYW5kIG9mdGVuIHRvIHNlZSB0aGF0IGV2ZXJ5dGhpbmcgaXMgd29ya2luZy4gCgpDbGljayBgQ2hlY2tgIHVuZGVyIHRoZSBgQnVpbGRgIG1lbnUuIEl0IGNoZWNrcyBsb3RzIG9mIHRoaW5ncyBmb3IgeW91ISBXZSdsbCBzZWUgbW9yZSBleGFtcGxlcyBvZiB0aGlzLiAKCiMjIyAxLjQgRnVuY3Rpb24gRGVwZW5kZW5jaWVzCgpNYWtlIGFub3RoZXIsIG1vcmUgZ2VuZXJhbCBmdW5jdGlvbiB0byBjb21wdXRlIGFueSBwb3dlcjoKCmBgYHtyfQpwb3cgPC0gZnVuY3Rpb24oeCwgYT0yKSB4XmEKYGBgCgpJdCBjYW4gZ28gaW4gdGhlIHNhbWUgUiBzY3JpcHQgYXMgYHNxdWFyZWAsIG9yIGEgZGlmZmVyZW50IG9uZSAtLSB5b3VyIGNob2ljZS4KCldlJ2xsIG1ha2UgYHNxdWFyZWAgZGVwZW5kIG9uIGBwb3dgLiAKCkFmdGVyaW5nIGBCdWlsZCBhbmQgUmVsb2FkYC9gSW5zdGFsbCBhbmQgUmVzdGFydGBpbmcsIHlvdSdsbCBub3RpY2UgdGhhdCB5b3UgY2FuJ3QgdXNlIGBwb3dgIGJlY2F1c2UgaXQncyBub3QgYGV4cG9ydGBlZC4gQnV0LCBgc3F1YXJlYCBzdGlsbCB3b3JrcyEgV2UgY2FsbCBgcG93YCBhbiBfaW50ZXJuYWxfIGZ1bmN0aW9uLiAKCl9fTm90ZV9fOiB5b3Ugc2hvdWxkIHN0aWxsIGRvY3VtZW50IHlvdXIgaW50ZXJuYWwgZnVuY3Rpb24hIEJ1dCBtZW50aW9uIHRoYXQgdGhlIGZ1bmN0aW9uIGlzIGludGVybmFsLiBVc2VycyB3aWxsIGJlIGFibGUgdG8gYWNjZXNzIHRoZSBkb2N1bWVudGF0aW9uIGxpa2Ugbm9ybWFsLCBidXQgc3RpbGwgd29uJ3QgYmUgYWJsZSB0byAoZWFzaWx5KSB1c2UgdGhlIGZ1bmN0aW9uLgoKSWYgeW91IHdhbnQgdG8gYmUgYWJsZSB0byB1c2UgaW50ZXJuYWwgZnVuY3Rpb25zIGFzIGEgZGV2ZWxvcGVyLCBidXQgZG9uJ3Qgd2FudCB1c2VycyB0byBoYXZlIChlYXN5KSBhY2Nlc3MgdG8gdGhlIGZ1bmN0aW9ucywgdGhlbiBydW4gYGxvYWRfYWxsYCBpbnN0ZWFkIG9mIGBCdWlsZCBhbmQgUmVsb2FkYC9gSW5zdGFsbCBhbmQgUmVzdGFydGAuIAoKCiMjIEV4ZXJjaXNlCgpVc2luZyBhdCBsZWFzdCB0d28gYC5SYCBzY3JpcHRzOgoKLSBNYWtlIGFuZCBkb2N1bWVudCBgcG93YC4gVHJ5IG91dCBvdGhlciBgcm94eWdlbjJgIHRhZ3MuCi0gTWFrZSBhbmQgZG9jdW1lbnQgYW5vdGhlciBmdW5jdGlvbiwgc2F5IGBjdWJlYCwgdGhhdCByYWlzZXMgYSB2ZWN0b3IgdG8gdGhlIHBvd2VyIG9mIDMuCgpNYWtlIHN1cmUgYXQgbGVhc3QgYHNxdWFyZWAgYW5kIGBjdWJlYCBhcmUgZXhwb3J0ZWQgdG8gdGhlIE5BTUVTUEFDRS4gT3B0aW9uYWxseSwgYHBvd2AuIE1ha2Ugc3VyZSB5b3UgYENoZWNrYCB0aGUgcGFja2FnZSBvZnRlbiEKCkZpbmlzaGVkIGVhcmx5PyBEbyBtb3JlIC0tIHdvcmsgb24gQXNzaWdubWVudCA5LCBhbmQvb3IgdHJ5IG91dCBtb3JlIGRvY3VtZW50YXRpb24gZmVhdHVyZXMgdGhhdCBjb21lcyB3aXRoIGByb3h5Z2VuMmAuCgojIyBXYXZlIDI6IE1vcmUgZG9jdW1lbnRhdGlvbiwgYW5kIHRlc3RpbmcKCgojIyMgMi4xIERFU0NSSVBUSU9OIGZpbGUKCkV2ZXJ5IFIgcGFja2FnZSBoYXMgdGhpcy4gSXQgY29udGFpbnMgdGhlIHBhY2thZ2UncyBtZXRhZGF0YS4gTGV0J3MgZWRpdCBpdDoKCi0gQWRkIGEgdGl0bGUgYW5kIGJyaWVmIGRlc2NyaXB0aW9uLiAKICAgIC0gUiBpcyBwaWNreSBhYm91dCB0aGVzZSEgQ2hlY2sgb3V0IFt0aGUgcnVsZXNdKGh0dHA6Ly9yLXBrZ3MuaGFkLmNvLm56L2Rlc2NyaXB0aW9uLmh0bWwjcGtnLWRlc2NyaXB0aW9uKS4KLSBBZGQgeW91ciBuYW1lLgogICAgLSBVc2UgdGhlIFtgQXV0aG9yc0BSYCBmaWVsZF0oaHR0cDovL3ItcGtncy5oYWQuY28ubnovZGVzY3JpcHRpb24uaHRtbCNhdXRob3IpIGluc3RlYWQgb2YgdGhlIGRlZmF1bHQgYEF1dGhvcmAgYW5kIGBNYWludGFpbmVyYCBmaWVsZHMuIAotIFBpY2sgYSBbbGljZW5zZV0oaHR0cDovL3ItcGtncy5oYWQuY28ubnovZGVzY3JpcHRpb24uaHRtbCNsaWNlbnNlKS4KCiMjIyAyLjIgVGVzdGluZyB3aXRoIGB0ZXN0dGhhdGAKCldlJ3ZlIGFscmVhZHkgc2VlbiBwYWNrYWdlIGBDaGVja2BzIC0tIHRoaXMgY2hlY2tzIHRoYXQgdGhlIHBpZWNlcyBvZiB5b3VyIFIgcGFja2FnZSBhcmUgaW4gcGxhY2UsIGFuZCB0aGF0IGV2ZW4geW91ciBleGFtcGxlcyBkb24ndCB0aHJvdyBlcnJvcnMuIFdlIHNob3VsZCBub3Qgb25seSBjaGVjayB0aGF0IG91ciBmdW5jdGlvbnMgYXJlIF93b3JraW5nXywgYnV0IHRoYXQgdGhleSBnaXZlIHVzIHJlc3VsdHMgdGhhdCB3ZSdkIF9leHBlY3RfLgoKVGhlIGB0ZXN0dGhhdGAgcGFja2FnZSBpcyB1c2VmdWwgZm9yIHRoaXMuIEluaXRpYWxpemUgaXQgaW4geW91ciBSIHBhY2thZ2UgYnkgcnVubmluZyBgZGV2dG9vbHM6OnVzZV90ZXN0dGhhdCgpYC4KCkFzIGEgdGVtcGxhdGUsIHNhdmUgdGhlIGZvbGxvd2luZyBzY3JpcHQgaW4gYSBmaWxlIGNhbGxlZCBgdGVzdF9zcXVhcmVgLgoKYGBgCmNvbnRleHQoIlNxdWFyaW5nIG5vbi1udW1lcmljcyIpCgp0ZXN0X3RoYXQoIkF0IGxlYXN0IG51bWVyaWMgdmFsdWVzIHdvcmsuIiwgewogICAgbnVtX3ZlYyA8LSBjKDAsIC00LjYsIDMuNCkKICAgIGV4cGVjdF9pZGVudGljYWwoc3F1YXJlKG51bWVyaWMoMCkpLCBudW1lcmljKDApKQogICAgZXhwZWN0X2lkZW50aWNhbChzcXVhcmUobnVtX3ZlYyksIG51bV92ZWNeMikKfSkKCnRlc3RfdGhhdCgiTG9naWNhbHMgYXV0b21hdGljYWxseSBjb252ZXJ0IHRvIG51bWVyaWMuIiwgewogICAgbG9naWNfdmVjIDwtIGMoVFJVRSwgVFJVRSwgRkFMU0UpCiAgICBleHBlY3RfaWRlbnRpY2FsKHNxdWFyZShsb2dpY192ZWMpLCBsb2dpY192ZWNeMikKfSkKYGBgCgpUaGVuLCB5b3UgY2FuIGV4ZWN1dGUgdGhvc2UgdGVzdHMgYnkgcnVubmluZyBgZGV2dG9vbHM6OnRlc3QoKWAsIG9yIGNsaWNraW5nIGBCdWlsZGAgLT4gYFRlc3QgcGFja2FnZWAuIAoKVGhlc2Ugc2FuaXR5IGNoZWNrcyBhcmUgdmVyeSBpbXBvcnRhbnQgYXMgeW91ciBSIHBhY2thZ2UgYmVjb21lcyBtb3JlIGNvbXBsZXghIAoKIyMgRXhlcmNpc2UKCi0gRWRpdCB5b3VyIERFU0NSSVBUSU9OIGZpbGUuIE1ha2Ugc3VyZSB0aGluZ3MgcnVuIHNtb290aGx5IHdoZW4geW91IGBDaGVja2AgdGhlIFIgcGFja2FnZS4gCi0gTW92ZSB0aGUgYG51bWVyaWMoMClgIHRlc3QgdG8gYSBuZXcgZmlsZSwgdW5kZXIgYSBuZXcgY29udGV4dC4KLSBFeHRlbmQgdGhlIHRlc3RzIHRvIGluY2x1ZGUgeW91ciBgY3ViZWAgZnVuY3Rpb24uCgpEb25lIGVhcmx5PyBXb3JrIG9uIHlvdXIgSG9tZXdvcmsgOSBSIHBhY2thZ2UuIAoKIyMgV2F2ZSAzOiBIaWdoZXItbGV2ZWwgVXNlciBEb2N1bWVudGF0aW9uCgojIyMgMy4xIFBhY2thZ2UgRG9jdW1lbnRhdGlvbgoKSnVzdCBsaWtlIHdlIGRvIGZvciBmdW5jdGlvbnMsIHdlIGNhbiBtYWtlIGEgbWFudWFsIChgLlJkYCkgcGFnZSBmb3Igb3VyIGVudGlyZSBSIHBhY2thZ2UsIHRvby4gRm9yIGV4YW1wbGUsIGNoZWNrIG91dCB0aGUgZG9jdW1lbnRhdGlvbiBmb3IgYGdncGxvdDJgOgoKYGBgCj9nZ3Bsb3QyICAgICAgICAgIyBDYW4gZXhlY3V0ZSBvbmx5IGlmIGBnZ3Bsb3QyYCBpcyBsb2FkZWQuCnBhY2thZ2U/Z2dwbG90MiAgIyBBbHdheXMgd29ya3MuCmBgYAoKVG8gZG8gc28sIGp1c3QgZXhlY3V0ZSBgZGV2dG9vbHM6OnVzZV9wYWNrYWdlX2RvYygpYC4gWW91J2xsIHNlZSBhIG5ldyBSIHNjcmlwdCBjb21lIHVwIHdpdGggYHJveHlnZW4yYC1zdHlsZSBkb2N1bWVudGF0aW9uIHRvIGBOVUxMYC4gRG9jdW1lbnQgYXMgeW91J2QgZG8gZnVuY3Rpb25zLCBhbmQgcnVuIGByb3h5Z2VuMjo6cm94eWdlbmlzZSgpYCBvciBgZGV2dG9vbHM6OmRvY3VtZW50KClgIHRvIGdlbmVyYXRlIHRoZSBgLlJkYCBmaWxlLiAKCkhlcmUncyBzYW1wbGUgZG9jdW1lbnRhdGlvbjoKCmBgYAojJyBDb252ZW5pZW50IENvbXB1dGF0aW9uIG9mIFBvd2VycwojJwojJyBBcmUgeW91IHRpcmVkIG9mIHVzaW5nIHRoZSBwb3dlciBvcGVyYXRvciwgXGNvZGV7Xn0gb3IgXGNvZGV7Kip9IGluIFI/CiMnIFVzZSB0aGlzIHBhY2thZ2UgdG8gY2FsbCBmdW5jdGlvbnMgdGhhdCBhcHBseSBjb21tb24gcG93ZXJzCiMnIHRvIHlvdXIgdmVjdG9ycy4KIycKIycgQG5hbWUgcG93ZXJzCiMnIEBhdXRob3IgVmluY2Vuem8gQ29pYQojJyBAbm90ZSBUaGlzIHBhY2thZ2UgaXNuJ3QgYWN0dWFsbHkgbWVhbnQgdG8gYmUgc2VyaW91cy4gSXQncyBqdXN0IGZvcgojJyB0ZWFjaGluZyBwdXJwb3Nlcy4KIycgQGRvY1R5cGUgcGFja2FnZQpgYGAKCiMjIyAzLjIgVmlnbmV0dGVzCgpJdCdzIGEgZ29vZCBpZGVhIHRvIHdyaXRlIGEgdmlnbmV0dGUgKG9yIHNldmVyYWwpIGZvciB5b3VyIFIgcGFja2FnZSB0byBzaG93IGhvdyB0aGUgcGFja2FnZSBpcyB1c2VmdWwgYXMgYSB3aG9sZS4gRG9jdW1lbnRhdGlvbiBmb3IgaW5kaXZpZHVhbCBmdW5jdGlvbnMgZG9uJ3Qgc3VmZmljZSBmb3IgdGhpcyBwdXJwb3NlIQoKVG8gd3JpdGUgYSB2aWduZXR0ZSBjYWxsZWQgYCJteV92aWduZXR0ZSJgLCBqdXN0IHJ1bgoKYGBgCmRldnRvb2xzOjp1c2VfdmlnbmV0dGUoIm15X3ZpZ25ldHRlIikKYGBgCgpTb21lIHRoaW5ncyBoYXBwZW4gYXV0b21hdGljYWxseSwgYnV0IGl0J3MgdXAgdG8geW91IHRvIG1vZGlmeSB0aGUgYC5SbWRgIGRvY3VtZW50IHRvIHByb3ZpZGUgYWRlcXVhdGUgaW5zdHJ1Y3Rpb24uIENoYW5nZSB0aGUgdGVtcGxhdGUgdG8gc3VpdCB5b3VyIHBhY2thZ2UuIFRoZSBvbmx5IHJlYWwgImNhdGNoIiB0byBkb2luZyB0aGlzIGlzIF9tYWtpbmcgc3VyZSB0aGUgdGl0bGUgaXMgcmVwbGFjZWQgaW4gYm90aCBpbnN0YW5jZXNfLgoKVGhlbiBqdXN0IGBLbml0YCwgYW5kIHRoZW4gcnVuIGBkZXZ0b29sczo6YnVpbGRfdmlnbmV0dGVzKClgIHRvIGJ1aWxkIHRoZSB2aWduZXR0ZXMuIAoKX19Ob3RlX186IFlvdSdsbCBoYXZlIHRvIHJlbW92ZSBgaW5zdC9kb2NgIGZyb20geW91ciBgLmdpdGlnbm9yZWAgZmlsZSB0aGF0IGdldHMgY3JlYXRlZCwgaWYgeW91IHdhbnQgdG8gcHVzaCBpdCB0byBnaXRodWIuIAoKIyMjIDMuMyBSRUFETUUKCkp1c3QgYXMgbW9zdCBwcm9qZWN0cyBzaG91bGQgaGF2ZSBhIGBSRUFETUVgIGZpbGUgaW4gdGhlIG1haW4gZGlyZWN0b3J5LCBzbyBzaG91bGQgYW4gUiBwYWNrYWdlLiAKCl9fUHVycG9zZXNfXzoKCi0gSW5mb3JtIHNvbWVvbmUgc3R1bWJsaW5nIGFjcm9zcyB5b3VyIHByb2plY3Qgd2hhdCB0aGV5J3ZlIHN0dW1ibGVkIGFjcm9zcy4KICAgIC0gQXQgYSBoaWdoIGxldmVsIChsaWtlICJUaGlzIGlzIGFuIFIgcGFja2FnZSIpLCBidXQgYWxzbwogICAgLSBzb21ld2hhdCBhdCBhIGxvd2VyIGxldmVsIHRvbywgbGlrZSB5b3VyIGRlc2NyaXB0aW9uIGZpbGUuIFRoaXMgYmVjb21lcyBhIGxpdHRsZSByZWR1bmRhbnQuCi0gSSBsaWtlIHRvIHVzZSB0aGUgUkVBRE1FIHRvIGluZm9ybSBfZGV2ZWxvcGVyc18gdGhlIG1haW4gd29ya2Zsb3cgYW5kIHNwaXJpdCBiZWhpbmQgX2RldmVsb3BpbmdfIHRoZSBwYWNrYWdlLgogICAgLSBUaGVyZSBhcmUgc29tZSB0aGluZ3MgdGhhdCB5b3UnZCB3YW50IG90aGVyIHBvdGVudGlhbCBkZXZlbG9wZXJzIHRvIGtub3cgYWJvdXQgdGhlIHBhY2thZ2UgYXMgYSB3aG9sZSwgeWV0IGFyZSBpcnJlbGV2YW50IHRvIHVzZXJzIQoKX19Ib3cgdG8gZG8gaXRfXzoKCllvdSBjb3VsZCBqdXN0IG1ha2UgYW5kIGVkaXQgYSBgUkVBRE1FLm1kYCBmaWxlIGxpa2Ugbm9ybWFsLiBCdXQgeW91J2xsIHByb2JhYmx5IHdhbnQgdG8gYnJpZWZseSBkZW1vbnN0cmF0ZSBzb21lIGNvZGUsIHNvIHlvdSdsbCBuZWVkIGFuIGAuUm1kYC4gTGV0IGBkZXZ0b29sc2Agc2V0IHRoYXQgdXAgZm9yIHlvdToKCmBgYApkZXZ0b29sczo6dXNlX3JlYWRtZV9ybWQoKQpgYGAKCmBrbml0YCBhbmQgeW91J3JlIGRvbmUhCgojIyBFeGVyY2lzZXMKCkNyZWF0ZSB0aGUgYWJvdmUgdGhyZWUgdHlwZXMgb2YgZG9jdW1lbnRhdGlvbiwgd2l0aG91dCBsb29raW5nIGF0IFtteSB2ZXJzaW9uXShodHRwczovL2dpdGh1Yi5jb20vdmluY2Vuem9jb2lhL3Bvd2VycykuIFRoZW4gY29tcGFyZS4gCgpJZGVhbGx5LCB5b3UnbGwgaGF2ZSBtb3JlIHRvIGRvY3VtZW50IGJlY2F1c2UgeW91J3ZlIGJlZW4gd29ya2luZyBvbiBleHBhbmRpbmcgdGhpcyAob3IgYW5vdGhlcikgUiBwYWNrYWdlIGZvciBIb21ld29yayA5IGFscmVhZHkuCgojIyBXYXZlIDQKCiMjIyA0LjEgRGVwZW5kZW5jaWVzCgpXZSBjYW4gdXNlIGZ1bmN0aW9ucyBmcm9tIG90aGVyIFIgcGFja2FnZXMgd2l0aGluIG91ciBob21lbWFkZSBSIHBhY2thZ2UsIHRvby4gV2UgbmVlZCB0byBkbyB0d28gdGhpbmdzOgoKLSBVc2UgdGhlIHN5bnRheCBgcGFja2FnZV9uYW1lOjpmdW5jdGlvbl9uYW1lKClgIHdoZW5ldmVyIHlvdSB3YW50IHRvIHVzZSBgZnVuY3Rpb25fbmFtZWAgZnJvbSBgcGFja2FnZV9uYW1lYC4gCi0gSW5kaWNhdGUgdGhhdCB5b3VyIFIgcGFja2FnZSBkZXBlbmRzIG9uIGBwYWNrYWdlX25hbWVgIGluIHRoZSBERVNDUklQVElPTiBmaWxlIGJ5IGV4ZWN1dGluZyB0aGUgY29tbWFuZCBgZGV2dG9vbHM6OnVzZV9wYWNrYWdlKCJwYWNrYWdlX25hbWUiKWAuCgpUaGVyZSBhcmUgb3RoZXIgbWV0aG9kcywgYnV0IHRoaXMgaXMgdGhlIGVhc2llc3QuIAoKX19FeGFtcGxlX186IEFkZCBgZ2dwbG90MmAgZGVwZW5kZW5jeSB0byBwbG90IHRoZSByZXN1bHRpbmcgY29tcHV0YXRpb25zLiBEbyBzbyBieSBhZGRpbmcgYSBwbG90IHRvIGBwb3dgIC0tIGNoYW5nZSBgcG93YCdzIGd1dHMgdG8gdGhlIGZvbGxvd2luZzoKCmBgYApyZXMgPC0geF5hCnAgPC0gZ2dwbG90Mjo6cXBsb3QoeCwgcmVzKQpwcmludChwKQpyZXR1cm4ocmVzKQpgYGAKCl9fTm90ZV9fOiBIZXJlJ3MgYW4gZXhhbXBsZSBvZiB0aGUgYmVuZWZpdHMgb2Ygbm90IGhhdmluZyB5b3VyIGZ1bmN0aW9ucyBkbyB0b28gbXVjaCAtLSBJIG9ubHkgbmVlZGVkIHRvIGNoYW5nZSBgcG93YCBhbG9uZSB0byBnZXQgdGhlIGNoYW5nZXMgdG8gd29yayBmb3IgYHNxdWFyZWAgYW5kIGBjdWJlYC4gCgojIyMgNC4yIEFkZGluZyBkYXRhIHRvIHlvdXIgUiBwYWNrYWdlCgpZb3UgY2FuIHN0b3JlIF9hbmQgZG9jdW1lbnRfIGRhdGFzZXRzIHdpdGhpbiBSIHBhY2thZ2VzLiBIZXJlJ3Mgb25lIHVzZWZ1bCB3YXkuCgpfX05vdGVfXzogVGhpcyBjdXJyZW50bHkgZG9lc24ndCBzZWVtIHRvIGJlIHByZXNlbnQgaW4gdGhlIGNvbXBhbmlvbiB0dXRvcmlhbCBmcm9tIEplbm55LiBDaGVjayBvdXQgW3RoZSBSIFBhY2thZ2VzICJkYXRhIGNoYXB0ZXIiXShodHRwOi8vci1wa2dzLmhhZC5jby5uei9kYXRhLmh0bWwpIGZvciBhIHJlc291cmNlLgoKSW4gdGhlIGNvbnNvbGU6CgoxLiBTdG9yZSB5b3VyIGRhdGEgYXMgUiBvYmplY3RzLiBTYXkgeW91IGhhdmUgdHdvOiBgZm9vYCBhbmQgYGJhcmAuCjIuIEV4ZWN1dGUgYGRldnRvb2xzOjp1c2VfZGF0YShmb28sIGJhcilgLiAKCmBmb29gIGFuZCBgYmFyYCB3aWxsIGJlIHNhdmVkIGFzIGAuUmRhdGFgIGZpbGVzIGluIHRoZSBuZXcgYC9kYXRhYCBkaXJlY3RvcnkuIFRoZXNlIGFyZSBhdmFpbGFibGUgdXBvbiBsb2FkaW5nIHRoZSBwYWNrYWdlLiAKClRvIGRvY3VtZW50IHRoZSBkYXRhLCBmb3IgZWFjaCBvYmplY3QgKGkuZS4sIGZvciBlYWNoIG9mIGBmb29gIGFuZCBgYmFyYCksIHB1dCBgcm94eWdlbjJgLXN0eWxlIGRvY3VtZW50YXRpb24gYWJvdmUgX3RoZSBjaGFyYWN0ZXJfIGAiZm9vImAgYW5kIGAiYmFyImAgaW4gYW4gUiBzY3JpcHQgaW4gdGhlIGAvUmAgZm9sZGVyLgoKX19FeGFtcGxlX186CgpBZGQgYHRlbnZlY2AgdG8gdGhlIHBhY2thZ2U6CgpgYGAKdGVudmVjIDwtIDE6MTAKZGV2dG9vbHM6OnVzZV9kYXRhKHRlbnZlYykKYGBgCgpUaGVuIG1ha2UgYSBuZXcgUiBzY3JpcHQgaW4gdGhlIGAvUmAgZm9sZGVyIGNvbnRhaW5pbmc6CgpgYGAKIycgSW50ZWdlciB2ZWN0b3IgZnJvbSAxIHRvIDEwCiMnCiMnIFNlbGYtZXhwbGFuYXRvcnkhCiMnCiMnIEBmb3JtYXQgVGhpcyAoYW5kIHRoZSBuZXh0KSBpcyBhIG5ldyB0YWcgdGhhdCBhcHBsaWVzIGZvciBkYXRhIGRvY3VtZW50YXRpb24uCiMnIEBzb3VyY2UgTm90ZSB0aGF0IHlvdSBzaG91bGRuJ3QgdXNlIHRoZSBgQGV4cG9ydGAgdGFnIHdoZW4gZG9jdW1lbnRpbmcgZGF0YSEKInRlbnZlYyIKYGBgCgojIyMgNC4zIExhdW5jaGluZyB5b3VyIFBhY2thZ2UgdG8gR2l0SHViCgpJZiBJIHdhbnQgdG8gcHV0IGFuIFIgcGFja2FnZSBvbiBHaXRIdWIsIEkgdHlwaWNhbGx5IGp1c3Q6CgoxLiBDbGljayAiTmV3IiBpbiBHaXRIdWIgdG8gbWFrZSBhIG5ldyByZXBvLiBEb24ndCBpbml0aWFsaXplIHdpdGggUkVBRE1FLgoyLiBGb2xsb3cgdGhlIGluc3RydWN0aW9ucyBnaXRodWIgcHJvdmlkZXMsIHdoaWNoIGludm9sdmVzIHR3byBsaW5lcyB0byBleGVjdXRlIGluIHRoZSB0ZXJtaW5hbC4KICAgIC0gVGhvc2UgdHdvIGxpbmVzIGNhbiBiZSBmb3VuZCBbaGVyZV0oaHR0cDovL2hhcHB5Z2l0d2l0aHIuY29tL2V4aXN0aW5nLWdpdGh1Yi1sYXN0Lmh0bWwjY29ubmVjdC10by1naXRodWIpIGluIEplbm55J3MgSGFwcHkgZ2l0IGJvb2suCiAgICAKVGhlcmUgaXMgYWxzbyB0aGUgW2BkZXZ0b29sczo6dXNlX2dpdGh1YigpYCB3YXldKGh0dHA6Ly9zdGF0NTQ1LmNvbS9wYWNrYWdlczA2X2Zvb2ZhY3RvcnMtcGFja2FnZS5odG1sI3VzZV9naXRodWIoKSkgLS0gYWx0aG91Z2gsIHRvIG1lLCBpdCBzZWVtcyBvdmVybHkgY29tcGxpY2F0ZWQgKHBlcmhhcHMgdGhlcmUncyBhbiBhZHZhbnRhZ2UgSSBkb24ndCBrbm93IGFib3V0KS4gSXQncyBqdXN0IGEgbWF0dGVyIG9mIGZvbGxvd2luZyB0aGUgaW5zdHJ1Y3Rpb25zLCB3aGljaCBhcmUgbm90IHdvcnRoIGRlbW9uc3RyYXRpbmcgaGVyZS4gCgojIyBFeGVyY2lzZQoKU29tZSB0aGluZ3MgeW91IG1pZ2h0IGZpbmQgdmFsdWUgaW4gdHJ5aW5nOgoKLSBNYWtlLCBzdG9yZSwgYW5kIGRvY3VtZW50IG1vcmUgZGF0YSAtLSBwZXJoYXBzIGBuZWd0ZW52ZWNgLCB0aGUgbmVnYXRpdmUgb2YgYHRlbnZlY2AuIAotIEdpdmUgdGhlIHVzZXIgYW4gb3B0aW9uIGFzIHRvIHdoZXRoZXIgdGhleSB3YW50IHRvIGRpc3BsYXkgdGhlIHBsb3Qgb3Igbm90IChoaW50OiBkbyBzbyB2aWEgZnVuY3Rpb24gYXJndW1lbnRzKS4KLSBUcnkgbGF1bmNoaW5nIHlvdXIgcGFja2FnZSBvbiBnaXRodWIuIAogICAgLSBQbGVhc2UgZG8gbWFrZSBpdCBwdWJsaWMuIFIgcGFja2FnZXMgYXJlIG1lYW50IHRvIGJlIHZpZXdlZCBieSB0aGUgd29ybGQhIElmIGl0IG1ha2VzIHlvdSBmZWVsIG1vcmUgY29tZm9ydGFibGUsIHB1dCBhIGRpc2NsYWltZXIgYXQgdGhlIHRvcCBvZiB0aGUgUkVBRE1FIGlmIHlvdSBkb24ndCB3YW50IHBlb3BsZSB0byB0YWtlIGl0IHNlcmlvdXNseS4g