1 Learning Objectives

suppressPackageStartupMessages(library(tidyverse))
suppressPackageStartupMessages(library(gapminder))

We’re ahead of schedule! As a result, I’ll talk about a variety of things first, then we’ll do some exercises.

1.1 1. Statistical Modelling in R

We’ll look at typical data analyses using R.

Note:

  • You aren’t expected to apply this is in your assignments! It’s new to the course because we’re ahead of schedule.
  • It’s OK if you’ve never heard of these statistical analyses. The point is that many model fitting procedures in R are similar.

Many statistical analyses in R follow a similar syntax.

1.1.1 1.1 Linear Regression

1.1.1.1 Model Fitting

You can run a linear regression in R with the lm function. Syntax:

lm(y ~ x1 + x2 + ... + xp, data=your_data_frame)

The first argument is a “formula” object in R. It’s typically used in modelling to separate Y and X values. (In fact, you’ve seen this already in ggplot’s facet_wrap and facet_grid)

Let’s fit the regression curve that we see in this plot, using lm:

ggplot(gapminder, aes(gdpPercap, lifeExp)) +
    geom_point() +
    geom_smooth(method="lm") +
    scale_x_log10()

Here’s the code:

fit1 <- lm(lifeExp ~ log(gdpPercap), data=gapminder)

What does this fit1 object look like?

fit1

That’s odd… what kind of R object is that??

typeof(fit1)

It’s a list, but R isn’t presenting it that way. It just looks like a bunch of text, but it’s not. Let’s use the lapply function to uncover its true nature – a list.

  • NOTE:
    • lapply loops over each component of a vector or list (first argument), applies a function to it (that you specify in the second argument), and outputs the function output in a list.
    • Let’s do this for an lm fit to head(gapminder), so that the output doesn’t take up a lot of space.
fit1_small <- lm(lifeExp ~ log(gdpPercap), data=head(gapminder))
lapply(fit1_small, identity) 

Why isn’t R printing out the list, then? Because it’s a special type of list – it’s of class "lm", something that the makers of the lm function decided. Whenever R encounters this object, it also has a special way of printing it to screen.

This is the idea of the “object oriented” part of R – something covered more in STAT 547 in the “R packages” section.

1.1.1.2 Making predictions from the model

The predict function works on "lm" objects to make predictions. If you don’t specify new data, it will make predictions using the existing X values. Let’s look at the first six:

predict(fit1) %>% head

How about plotted against the original X values? (Which was log gdpPercap)

qplot(log(gapminder$gdpPercap), predict(fit1))

For fun… let’s put this overtop of the scatterplot:

ggplot(gapminder, aes(gdpPercap, lifeExp)) +
    geom_point(alpha=0.1) +
    geom_point(y=predict(fit1), colour="red") +
    scale_x_log10()

You can predict with new data, too, as long as the data frame you enter has the same column names as your X values.

(my_newdata <- data.frame(gdpPercap=c(100, 547, 289)))
predict(fit1, newdata=my_newdata)
predict(fit1, newdata=filter(gapminder, country=="Canada"))

1.1.1.3 Extracting model characteristics

We can extract a bunch of things from the lm output.

  • Regression coefficients? They’re stored in the $coefficients part of the list. Or, use the coeff function.
fit1$coefficients
coef(fit1)
  • Residuals? They’re stored in the $residuals part of the list. Or, use the resid function. (Let’s only display the first six… but plot all of them!)
fit1$residuals %>% head
resid(fit1) %>% head
qplot(log(gapminder$gdpPercap), resid(fit1)) +
    geom_hline(yintercept=0,
               linetype="dashed")

lm is kind of annoying in that not everything you might want is there. You can access more things using the summary function. What’s printed to screen after summary, though, is quite nice!

(summ_fit1 <- summary(fit1))

You can see all sorts of things, like p-values, \(R^2\) (and adjusted \(R^2\) values), and standard errors.

As before, this looks nice and all… but what the heck is this new object? Again, it’s a list. Let’s see its components (again, with the smaller fit, so that we don’t take over all the space on the screen).

summ_fit1_small <- summary(fit1_small)
typeof(summ_fit1_small)
lapply(summ_fit1_small, identity)  # Pry it open!!

There we have it. Now, what would you like to extract?:

  • R-squared? R-squared adjusted? Okay:
summ_fit1$r.squared
summ_fit1$adj.r.squared
  • Estimated standard devaition of the random error term? Okay:
summ_fit1$sigma

Where can we find the documentation for the components of this list, though, if it’s not in the documentation for lm? Look at the documentation of summary.lm.

But wait! Why not just look at the documentation for summary? It’s because summary is a generic function, and depends on the class of object it’s being applied to. If it’s of class "lm", then summary.lm is what’s actually secretly run. Running summary on an object of class "glm"? R will secretly run summary.glm instead.

PS: The broom package makes a lot of this easier and less cryptic. We won’t go over it here. Check out its vignette.

1.1.2 1.2 Generalized Linear Models (like Logistic Regression)

We won’t go over this in as much detail, because it’s quite similar to lm. But if you want to run a Generalized Linear Model (GLM) – such as logistic/binomial regression, or Poission regression – just use the glm function.

Probably the biggest noteworthy difference is the family argument, specifying what type of regression you want to do. Syntax:

## Poisson regression:
glm(y ~ x1 + x2 + ... + xp, family=poisson, data=your_data_frame)
## Logistic (aka Binomial) regression:
glm(y ~ x1 + x2 + ... + xp, family=binomial, data=your_data_frame)

Its output looks similar to lm. It’s also a list disguised as text. It also shows more when you use the summary function. It also works with the predict function. It also becomes tidier when used in conjunction with the broom package.

1.1.3 1.3 Others…

Here are some other packages/functions you might find useful to fit models:

  • (Generalized) Mixed Effects Models
    • Two R packages are available: lme4 and nlme.
      • Check out this discussion on Cross Validated for a comparison of the two packages.
    • I’ve found the function glmer in the lme4 package to be fruitful.
  • Kernel smoothing (i.e. fitting a “smoother”): check out the loess function.
  • Generalized Additive Models: The gam function in either the gam package or mgcv package.
  • Robust linear regression: The rlm function in the MASS package is your friend.
  • Regularized regression (GLM) (lasso, elastic net, or ridge regression): Use the glmnet function in the glmnet package.
    • PS: I highly recommend this if you have more predictors/covariates/features than you know what to do with… this will weed out the unnecessary ones, and produce a model with good prediction accuracy at the same time.

1.2 2. More dplyr

There’s one more important thing with dplyr that you ought to know: applying mutate to a grouped tibble.

Remember applying summarize to a grouped tibble?

gapminder %>%
    group_by(continent) %>% 
    summarize(mean_gdpPercap = mean(gdpPercap),
              n_countries    = length(gdpPercap))

Well, we can also apply mutate to each group, too. For example, let’s calculate the growth in population since the first year on record for each country:

gapminder %>% 
    group_by(country) %>% 
    mutate(pop_growth = pop - pop[1])

Notice that dplyr has retained the original grouping – it hasn’t pealed back one level of grouping. That’s because there’s still more than one row for each group!

How about growth compared to 1972?

gapminder %>% 
    group_by(country) %>% 
    mutate(pop_growth = pop - pop[year==1972])

In general, this type of “grouped mutation” is useful for window functions. What’s that? Well, let’s see other types of functions in R:

  • Vectorized Functions: These take a vector, and operate on each component independently to return a vector of the same length. In other words, they work element-wise.
    • Examples are cos, sin, log, exp, round.
    • We don’t need to group_by in order to mutate with these.
  • Aggregate Functions: These take a vector, and return a vector of length 1 – as if “aggregating” the values in the vector into a single value.
    • Examples are mean, sd, length, typeof.
    • We use these in dplyr’s summarise function.
  • Window Functions: these take a vector, and return a vector of the same length that depends on other values in the vector.
    • Examples are lag, rank, cumsum.
    • See the window-functions vignette for the dplyr package.

1.3 3. More ggplot

(Have you seen the ggplot2 cheatsheet? It contains a lot of useful information on two pages!)

1.3.1 3.1 theme layers

You can change the look of a plot by adding a theme layer to your ggplot layers. This function does not actually change the “nature” of the plot itself – only the look! i.e., the so-called “non-data” type displays.

Examples: - font - justification of titles - rotation of labels - background colour - line thickness - etc…

There are “complete themes” that come with ggplot2, my favourite being theme_bw (I’ve grown tired of the default gray background, so theme_bw is refreshing).

Let’s see an example:

p1 <- ggplot(gapminder, aes(gdpPercap, lifeExp)) +
    facet_wrap(~ continent) +
    geom_point(colour="#386CB0", alpha=0.2) +
    scale_x_log10()
p1 + theme_bw()

The general theme function gives you vast functionality… just check its documentation to see what things you can change. The arguments of theme follow a naming convention: general.to.specific

For example, - axis.title will allow you to change the font of the axis titles. - axis.title.x does the same, but focusses on the x axis.

Note: You can’t change the actual words this way! That’s changing the nature of the plot, and not the look of the plot.

Once we’ve chosen an argument name, we need to specify its value. This is almost always the output of one of the following functions:

  • element_blank (basically means replace with “nothing”)
  • element_rect: allows us to specify features of a rectangle.
  • element_line: allows us to specify features of a line.
  • element_text: allows us to specify font.

Check out their documentation to see exactly how each feature is modified.

  • Example: To p1 above, do all of the following….
    • change the background strip colour to orange,
    • change the axis titles’ font sizes to 14, and
    • change the panel titles’ font sizes to 14 and bolded.
p1 +
    theme(strip.background = element_rect(fill="orange"),
          axis.title = element_text(size=14),
          strip.text = element_text(size=14, face="bold"))

Another example: do the same, but in conjunction with the theme_bw (notice the order)

## Correct:
p1 +
    theme_bw() +
    theme(strip.background = element_rect(fill="orange"),
          axis.title = element_text(size=14),
          strip.text = element_text(size=14, face="bold"))
## Incorrect:
p1 +
    theme(strip.background = element_rect(fill="orange"),
          axis.title = element_text(size=14),
          strip.text = element_text(size=14, face="bold")) +
    theme_bw()  # Overrides the previous `theme` call!

1.3.2 3.2 Modifying scales

Recall that we use some scale to represent the range of values that a variable takes in our data. ggplot chooses defaults for this scale, but we can change those.

Check out this tutorial by Hadley Wickham for scales.

We can modify scales using a suite of functions that have the following naming convention: scale_a_b, where:

  • a is the scale you want to change. Is it colour? size? x position?
  • b typically speaks to the nature of the variable. continuous is your variable is continuous; discrete if discrete. But, could be other things in certain cases, like log10, or date if your variable consists of dates. manual is an option too.

Examples: scale_x_continuous, scale_colour_discrete, scale_y_sqrt.

As usual in ggplot, these functions are added as a layer.

1.3.2.1 Useful arguments

There are many useful arguments here. Some are more self-explanatory than others.

  • name. The first argument. Indicate the name of the scale/legend here.
    • You can also use the labs function for X and Y axes, and even plot title.
p1 + scale_y_continuous("Life Expectancy")
p1 + labs(x="GDP per capita", 
          y="Life Expectancy",
          title="My Plot")

ggplot(gapminder, aes(gdpPercap, lifeExp)) +
    geom_point(aes(colour=continent),
               alpha=0.2) +
    scale_colour_discrete("Continents of\n the World")
  • breaks (Typically of a continuous scale). Here, you get to specify where along the scale you’d like to display a value.
    • Numbers are on the scale of the data (such as population), not the geometric scale (such as a hex colour code, or number of pixels over in a plot).
## Log lines:
p1 + scale_x_log10(breaks=c((1:10)*1000,
                            (1:10)*10000))

p2 <- ggplot(gapminder, aes(gdpPercap, lifeExp)) +
    geom_point(aes(colour=pop/10^9),
               alpha=0.2)
## Default breaks
p2 + scale_colour_continuous("Population\nin billions")
## New breaks
p2 + scale_colour_continuous("Population\nin billions",
                             breaks=seq(0,2,by=0.2))
  • labels. Text to replace the data value labels. Most useful for discrete data.
## Not a good idea:
p2 + scale_colour_continuous("My odd\npopulation\nscale",
                             breaks=c(0.2, 0.7, 1.2),
                             labels=c("small", "bigger", "big"))

## Discrete scale:
ggplot(gapminder, aes(gdpPercap, lifeExp)) +
    geom_point(aes(colour=continent),
               alpha=0.2) +
    scale_colour_discrete(labels=c("Af", "Am", "As", "Eu", "Oc"))
  • limits. Lower and upper bounds of the data that you’d like displayed. Leave one as NA if you want to use the default.
p1 + scale_y_continuous(limits=c(60,NA))
  • position. Position of the scale. Also controllable using theme for the legend.
p1 + scale_y_continuous(position="right")
p2 + theme(legend.position = "bottom")

1.4 4. Exercises.

Practice these concepts in the following exercises.

Exercise 1: Suppose we want to calculate some quantity for each country in the gapminder data set. For each of the following quantities, indicate whether the function is vectorized, aggregate, or window, and use dplyr functions to calculate the specified variable.

  • The change in population from 1962 to 1972.
## It's an Aggregate function.
gapminder %>% 
    filter(year %in% c(1962, 1972)) %>% 
    arrange(year) %>% 
    group_by(country) %>% 
    summarize(pop_chg=diff(pop))
gapminder %>% 
    group_by(country) %>% 
    summarise(pop_chg=pop[year==1972]-pop[year==1962])
  • The population, in billions.
## It's a vectorized function.
gapminder %>%
    mutate(pop_in_bill = pop/10^9)
  • The lagged gdpPercap
    • i.e., the value that appears for 1962 would be the gdpPercap in 1957 (the previous entry).
    • Hint: use the lag function, then filter out the NA’s created with the is.na function.
## It's a window function.
gapminder %>% 
    group_by(country) %>% 
    arrange(year) %>% 
    mutate(lag_gdpPercap=lag(gdpPercap)) %>% 
    filter(!is.na(lag_gdpPercap))

Exercise 2: For the gapminder dataset, make a spaghetti plot showing the population trend (in millions) over time for each country, facetted by continent. Make as many of the following modifications as you can:

  • Colour each line by the log maximum gdpPercap experienced by the country.
  • Rotate the x-axis labels to be vertical.
  • Remove the x-axis title.
  • Give the legend an appropriate title.
  • Put the y-axis on a log-scale.
  • Rename the y-axis title.
  • Add more numbers along the y-axis.
  • Give the plot a title, and center the title.
  • Only label the x axis with years 1950, 1975, and 2000.
  • Move the colour scale to the bottom.
  • Rename the colour legend
gapminder %>% 
    group_by(country) %>% 
    mutate(max_gdpPercap=max(gdpPercap)) %>% 
    ggplot(aes(year, pop/10^6)) + 
    facet_wrap(~ continent) +
    geom_line(aes(group=country,
                  colour=log(max_gdpPercap)),
              alpha=0.25) +
    theme_bw() + # I added this because I like this theme.
    labs(title="Population Trends") +
    scale_y_log10("Population (millions)",
                  breaks=c(0.1, 1, 10, 100, 1000),
                  labels=c(0.1, 1, 10, 100, 1000)) +
    scale_x_continuous("", breaks=c(1950, 1975, 2000)) +
    scale_colour_continuous("log Maximum\nGDP per cap.") +
    theme(axis.text.x = element_text(angle=90),
          plot.title = element_text(hjust=0.5),
          legend.position = "bottom")
LS0tCnRpdGxlOiAiU1RBVCA1NDUgQ2xhc3MgTWVldGluZyAwOCIKb3V0cHV0OgogICAgaHRtbF9ub3RlYm9vazoKICAgICAgICB0b2M6IHRydWUKICAgICAgICB0aGVtZTogY2VydWxlYW4KICAgICAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKZWRpdG9yX29wdGlvbnM6IAogIGNodW5rX291dHB1dF90eXBlOiBpbmxpbmUKLS0tCgojIExlYXJuaW5nIE9iamVjdGl2ZXMKCgpgYGB7cn0Kc3VwcHJlc3NQYWNrYWdlU3RhcnR1cE1lc3NhZ2VzKGxpYnJhcnkodGlkeXZlcnNlKSkKc3VwcHJlc3NQYWNrYWdlU3RhcnR1cE1lc3NhZ2VzKGxpYnJhcnkoZ2FwbWluZGVyKSkKYGBgCgpXZSdyZSBhaGVhZCBvZiBzY2hlZHVsZSEgQXMgYSByZXN1bHQsIEknbGwgdGFsayBhYm91dCBhIHZhcmlldHkgb2YgdGhpbmdzIGZpcnN0LCB0aGVuIHdlJ2xsIGRvIHNvbWUgZXhlcmNpc2VzLiAKCgojIyAxLiBTdGF0aXN0aWNhbCBNb2RlbGxpbmcgaW4gUgoKV2UnbGwgbG9vayBhdCB0eXBpY2FsIGRhdGEgYW5hbHlzZXMgdXNpbmcgUi4KCioqTm90ZSoqOiAKCi0gWW91IGFyZW4ndCBleHBlY3RlZCB0byBhcHBseSB0aGlzIGlzIGluIHlvdXIgYXNzaWdubWVudHMhIEl0J3MgbmV3IHRvIHRoZSBjb3Vyc2UgYmVjYXVzZSB3ZSdyZSBhaGVhZCBvZiBzY2hlZHVsZS4KLSBJdCdzIE9LIGlmIHlvdSd2ZSBuZXZlciBoZWFyZCBvZiB0aGVzZSBzdGF0aXN0aWNhbCBhbmFseXNlcy4gVGhlIHBvaW50IGlzIHRoYXQgbWFueSBtb2RlbCBmaXR0aW5nIHByb2NlZHVyZXMgaW4gUiBhcmUgc2ltaWxhci4KCk1hbnkgc3RhdGlzdGljYWwgYW5hbHlzZXMgaW4gUiBmb2xsb3cgYSBzaW1pbGFyIHN5bnRheC4KCiMjIyAxLjEgTGluZWFyIFJlZ3Jlc3Npb24KCiMjIyMgTW9kZWwgRml0dGluZwoKWW91IGNhbiBydW4gYSBsaW5lYXIgcmVncmVzc2lvbiBpbiBSIHdpdGggdGhlIGBsbWAgZnVuY3Rpb24uIFN5bnRheDoKCmBgYApsbSh5IH4geDEgKyB4MiArIC4uLiArIHhwLCBkYXRhPXlvdXJfZGF0YV9mcmFtZSkKYGBgCgpUaGUgZmlyc3QgYXJndW1lbnQgaXMgYSAiZm9ybXVsYSIgb2JqZWN0IGluIFIuIEl0J3MgdHlwaWNhbGx5IHVzZWQgaW4gbW9kZWxsaW5nIHRvIHNlcGFyYXRlIFkgYW5kIFggdmFsdWVzLiAoSW4gZmFjdCwgeW91J3ZlIHNlZW4gdGhpcyBhbHJlYWR5IGluIGBnZ3Bsb3RgJ3MgIGBmYWNldF93cmFwYCBhbmQgYGZhY2V0X2dyaWRgKQoKTGV0J3MgZml0IHRoZSByZWdyZXNzaW9uIGN1cnZlIHRoYXQgd2Ugc2VlIGluIHRoaXMgcGxvdCwgdXNpbmcgYGxtYDoKCmBgYHtyfQpnZ3Bsb3QoZ2FwbWluZGVyLCBhZXMoZ2RwUGVyY2FwLCBsaWZlRXhwKSkgKwogICAgZ2VvbV9wb2ludCgpICsKICAgIGdlb21fc21vb3RoKG1ldGhvZD0ibG0iKSArCiAgICBzY2FsZV94X2xvZzEwKCkKYGBgCgpIZXJlJ3MgdGhlIGNvZGU6CgpgYGB7cn0KZml0MSA8LSBsbShsaWZlRXhwIH4gbG9nKGdkcFBlcmNhcCksIGRhdGE9Z2FwbWluZGVyKQpgYGAKCldoYXQgZG9lcyB0aGlzIGBmaXQxYCBvYmplY3QgbG9vayBsaWtlPwoKYGBge3J9CmZpdDEKYGBgCgpUaGF0J3Mgb2RkLi4uIHdoYXQga2luZCBvZiBSIG9iamVjdCBpcyB0aGF0Pz8KCmBgYHtyfQp0eXBlb2YoZml0MSkKYGBgCgpJdCdzIGEgbGlzdCwgYnV0IFIgaXNuJ3QgcHJlc2VudGluZyBpdCB0aGF0IHdheS4gSXQganVzdCBsb29rcyBsaWtlIGEgYnVuY2ggb2YgdGV4dCwgYnV0IGl0J3Mgbm90LiBMZXQncyB1c2UgdGhlIGBsYXBwbHlgIGZ1bmN0aW9uIHRvIHVuY292ZXIgaXRzIHRydWUgbmF0dXJlIC0tIGEgbGlzdC4KCi0gTk9URTogCiAgICAtIGBsYXBwbHlgIGxvb3BzIG92ZXIgZWFjaCBjb21wb25lbnQgb2YgYSB2ZWN0b3Igb3IgbGlzdCAoX2ZpcnN0IGFyZ3VtZW50XyksIGFwcGxpZXMgYSBmdW5jdGlvbiB0byBpdCAodGhhdCB5b3Ugc3BlY2lmeSBpbiB0aGUgX3NlY29uZCBhcmd1bWVudF8pLCBhbmQgb3V0cHV0cyB0aGUgZnVuY3Rpb24gb3V0cHV0IGluIGEgbGlzdC4KICAgIC0gTGV0J3MgZG8gdGhpcyBmb3IgYW4gYGxtYCBmaXQgdG8gYGhlYWQoZ2FwbWluZGVyKWAsIHNvIHRoYXQgdGhlIG91dHB1dCBkb2Vzbid0IHRha2UgdXAgYSBsb3Qgb2Ygc3BhY2UuIAoKYGBge3J9CmZpdDFfc21hbGwgPC0gbG0obGlmZUV4cCB+IGxvZyhnZHBQZXJjYXApLCBkYXRhPWhlYWQoZ2FwbWluZGVyKSkKbGFwcGx5KGZpdDFfc21hbGwsIGlkZW50aXR5KSAKYGBgCgpXaHkgaXNuJ3QgUiBwcmludGluZyBvdXQgdGhlIGxpc3QsIHRoZW4/IEJlY2F1c2UgaXQncyBhIHNwZWNpYWwgdHlwZSBvZiBsaXN0IC0tIGl0J3Mgb2YgY2xhc3MgYCJsbSJgLCBzb21ldGhpbmcgdGhhdCB0aGUgbWFrZXJzIG9mIHRoZSBgbG1gIGZ1bmN0aW9uIGRlY2lkZWQuIFdoZW5ldmVyIFIgZW5jb3VudGVycyB0aGlzIG9iamVjdCwgaXQgYWxzbyBoYXMgYSBzcGVjaWFsIHdheSBvZiBwcmludGluZyBpdCB0byBzY3JlZW4uIAoKVGhpcyBpcyB0aGUgaWRlYSBvZiB0aGUgIm9iamVjdCBvcmllbnRlZCIgcGFydCBvZiBSIC0tIHNvbWV0aGluZyBjb3ZlcmVkIG1vcmUgaW4gU1RBVCA1NDcgaW4gdGhlICJSIHBhY2thZ2VzIiBzZWN0aW9uLiAKCiMjIyMgTWFraW5nIHByZWRpY3Rpb25zIGZyb20gdGhlIG1vZGVsCgpUaGUgYHByZWRpY3RgIGZ1bmN0aW9uIHdvcmtzIG9uIGAibG0iYCBvYmplY3RzIHRvIG1ha2UgcHJlZGljdGlvbnMuIElmIHlvdSBkb24ndCBzcGVjaWZ5IG5ldyBkYXRhLCBpdCB3aWxsIG1ha2UgcHJlZGljdGlvbnMgdXNpbmcgdGhlIGV4aXN0aW5nIFggdmFsdWVzLiBMZXQncyBsb29rIGF0IHRoZSBmaXJzdCBzaXg6CgpgYGB7cn0KcHJlZGljdChmaXQxKSAlPiUgaGVhZApgYGAKCkhvdyBhYm91dCBwbG90dGVkIGFnYWluc3QgdGhlIG9yaWdpbmFsIFggdmFsdWVzPyAoV2hpY2ggd2FzIGxvZyBnZHBQZXJjYXApCgpgYGB7cn0KcXBsb3QobG9nKGdhcG1pbmRlciRnZHBQZXJjYXApLCBwcmVkaWN0KGZpdDEpKQpgYGAKCkZvciBmdW4uLi4gbGV0J3MgcHV0IHRoaXMgb3ZlcnRvcCBvZiB0aGUgc2NhdHRlcnBsb3Q6CgpgYGB7cn0KZ2dwbG90KGdhcG1pbmRlciwgYWVzKGdkcFBlcmNhcCwgbGlmZUV4cCkpICsKICAgIGdlb21fcG9pbnQoYWxwaGE9MC4xKSArCiAgICBnZW9tX3BvaW50KHk9cHJlZGljdChmaXQxKSwgY29sb3VyPSJyZWQiKSArCiAgICBzY2FsZV94X2xvZzEwKCkKYGBgCgpZb3UgY2FuIHByZWRpY3Qgd2l0aCBuZXcgZGF0YSwgdG9vLCBhcyBsb25nIGFzIHRoZSBkYXRhIGZyYW1lIHlvdSBlbnRlciBoYXMgdGhlIHNhbWUgY29sdW1uIG5hbWVzIGFzIHlvdXIgWCB2YWx1ZXMuIAoKYGBge3J9CihteV9uZXdkYXRhIDwtIGRhdGEuZnJhbWUoZ2RwUGVyY2FwPWMoMTAwLCA1NDcsIDI4OSkpKQpwcmVkaWN0KGZpdDEsIG5ld2RhdGE9bXlfbmV3ZGF0YSkKcHJlZGljdChmaXQxLCBuZXdkYXRhPWZpbHRlcihnYXBtaW5kZXIsIGNvdW50cnk9PSJDYW5hZGEiKSkKYGBgCgoKIyMjIyBFeHRyYWN0aW5nIG1vZGVsIGNoYXJhY3RlcmlzdGljcwoKV2UgY2FuIGV4dHJhY3QgYSBidW5jaCBvZiB0aGluZ3MgZnJvbSB0aGUgYGxtYCBvdXRwdXQuCgotIFJlZ3Jlc3Npb24gY29lZmZpY2llbnRzPyBUaGV5J3JlIHN0b3JlZCBpbiB0aGUgYCRjb2VmZmljaWVudHNgIHBhcnQgb2YgdGhlIGxpc3QuIE9yLCB1c2UgdGhlIGBjb2VmZmAgZnVuY3Rpb24uCgpgYGB7cn0KZml0MSRjb2VmZmljaWVudHMKY29lZihmaXQxKQpgYGAKCi0gUmVzaWR1YWxzPyBUaGV5J3JlIHN0b3JlZCBpbiB0aGUgYCRyZXNpZHVhbHNgIHBhcnQgb2YgdGhlIGxpc3QuIE9yLCB1c2UgdGhlIGByZXNpZGAgZnVuY3Rpb24uIChMZXQncyBvbmx5IGRpc3BsYXkgdGhlIGZpcnN0IHNpeC4uLiBidXQgcGxvdCBhbGwgb2YgdGhlbSEpCgpgYGB7cn0KZml0MSRyZXNpZHVhbHMgJT4lIGhlYWQKcmVzaWQoZml0MSkgJT4lIGhlYWQKcXBsb3QobG9nKGdhcG1pbmRlciRnZHBQZXJjYXApLCByZXNpZChmaXQxKSkgKwogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0PTAsCiAgICAgICAgICAgICAgIGxpbmV0eXBlPSJkYXNoZWQiKQpgYGAKCmBsbWAgaXMga2luZCBvZiBhbm5veWluZyBpbiB0aGF0IG5vdCBldmVyeXRoaW5nIHlvdSBtaWdodCB3YW50IGlzIHRoZXJlLiBZb3UgY2FuIGFjY2VzcyBtb3JlIHRoaW5ncyB1c2luZyB0aGUgYHN1bW1hcnlgIGZ1bmN0aW9uLiBXaGF0J3MgcHJpbnRlZCB0byBzY3JlZW4gYWZ0ZXIgYHN1bW1hcnlgLCB0aG91Z2gsIF9pc18gcXVpdGUgbmljZSEKCmBgYHtyfQooc3VtbV9maXQxIDwtIHN1bW1hcnkoZml0MSkpCmBgYAoKWW91IGNhbiBzZWUgYWxsIHNvcnRzIG9mIHRoaW5ncywgbGlrZSBwLXZhbHVlcywgJFJeMiQgKGFuZCBhZGp1c3RlZCAkUl4yJCB2YWx1ZXMpLCBhbmQgc3RhbmRhcmQgZXJyb3JzLiAKCkFzIGJlZm9yZSwgdGhpcyBsb29rcyBuaWNlIGFuZCBhbGwuLi4gYnV0IHdoYXQgdGhlIGhlY2sgaXMgdGhpcyBuZXcgb2JqZWN0PyBBZ2FpbiwgaXQncyBhIGxpc3QuIExldCdzIHNlZSBpdHMgY29tcG9uZW50cyAoYWdhaW4sIHdpdGggdGhlIHNtYWxsZXIgZml0LCBzbyB0aGF0IHdlIGRvbid0IHRha2Ugb3ZlciBhbGwgdGhlIHNwYWNlIG9uIHRoZSBzY3JlZW4pLgoKYGBge3J9CnN1bW1fZml0MV9zbWFsbCA8LSBzdW1tYXJ5KGZpdDFfc21hbGwpCnR5cGVvZihzdW1tX2ZpdDFfc21hbGwpCmxhcHBseShzdW1tX2ZpdDFfc21hbGwsIGlkZW50aXR5KSAgIyBQcnkgaXQgb3BlbiEhCmBgYAoKVGhlcmUgd2UgaGF2ZSBpdC4gTm93LCB3aGF0IHdvdWxkIHlvdSBsaWtlIHRvIGV4dHJhY3Q/OgoKLSBSLXNxdWFyZWQ/IFItc3F1YXJlZCBhZGp1c3RlZD8gT2theToKCmBgYHtyfQpzdW1tX2ZpdDEkci5zcXVhcmVkCnN1bW1fZml0MSRhZGouci5zcXVhcmVkCmBgYAoKLSBFc3RpbWF0ZWQgc3RhbmRhcmQgZGV2YWl0aW9uIG9mIHRoZSByYW5kb20gZXJyb3IgdGVybT8gT2theToKCmBgYHtyfQpzdW1tX2ZpdDEkc2lnbWEKYGBgCgpXaGVyZSBjYW4gd2UgZmluZCB0aGUgZG9jdW1lbnRhdGlvbiBmb3IgdGhlIGNvbXBvbmVudHMgb2YgX3RoaXNfIGxpc3QsIHRob3VnaCwgaWYgaXQncyBub3QgaW4gdGhlIGRvY3VtZW50YXRpb24gZm9yIGBsbWA/IExvb2sgYXQgdGhlIGRvY3VtZW50YXRpb24gb2YgYHN1bW1hcnkubG1gLiAKCkJ1dCB3YWl0ISBXaHkgbm90IGp1c3QgbG9vayBhdCB0aGUgZG9jdW1lbnRhdGlvbiBmb3IgYHN1bW1hcnlgPyBJdCdzIGJlY2F1c2UgYHN1bW1hcnlgIGlzIGEgZ2VuZXJpYyBmdW5jdGlvbiwgYW5kIGRlcGVuZHMgb24gdGhlIF9jbGFzc18gb2Ygb2JqZWN0IGl0J3MgYmVpbmcgYXBwbGllZCB0by4gSWYgaXQncyBvZiBjbGFzcyBgImxtImAsIHRoZW4gYHN1bW1hcnkubG1gIGlzIHdoYXQncyBhY3R1YWxseSBzZWNyZXRseSBydW4uIFJ1bm5pbmcgYHN1bW1hcnlgIG9uIGFuIG9iamVjdCBvZiBjbGFzcyBgImdsbSJgPyBSIHdpbGwgc2VjcmV0bHkgcnVuIGBzdW1tYXJ5LmdsbWAgaW5zdGVhZC4gCgpQUzogVGhlIGBicm9vbWAgcGFja2FnZSBtYWtlcyBhIGxvdCBvZiB0aGlzIGVhc2llciBhbmQgbGVzcyBjcnlwdGljLiBXZSB3b24ndCBnbyBvdmVyIGl0IGhlcmUuIENoZWNrIG91dCBbaXRzIHZpZ25ldHRlXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvYnJvb20vdmlnbmV0dGVzL2Jyb29tLmh0bWwpLgoKIyMjIDEuMiBHZW5lcmFsaXplZCBMaW5lYXIgTW9kZWxzIChsaWtlIExvZ2lzdGljIFJlZ3Jlc3Npb24pCgpXZSB3b24ndCBnbyBvdmVyIHRoaXMgaW4gYXMgbXVjaCBkZXRhaWwsIGJlY2F1c2UgaXQncyBxdWl0ZSBzaW1pbGFyIHRvIGBsbWAuIEJ1dCBpZiB5b3Ugd2FudCB0byBydW4gYSBHZW5lcmFsaXplZCBMaW5lYXIgTW9kZWwgKEdMTSkgLS0gc3VjaCBhcyBsb2dpc3RpYy9iaW5vbWlhbCByZWdyZXNzaW9uLCBvciBQb2lzc2lvbiByZWdyZXNzaW9uIC0tIGp1c3QgdXNlIHRoZSBgZ2xtYCBmdW5jdGlvbi4KClByb2JhYmx5IHRoZSBiaWdnZXN0IG5vdGV3b3J0aHkgZGlmZmVyZW5jZSBpcyB0aGUgYGZhbWlseWAgYXJndW1lbnQsIHNwZWNpZnlpbmcgd2hhdCB0eXBlIG9mIHJlZ3Jlc3Npb24geW91IHdhbnQgdG8gZG8uIFN5bnRheDoKCmBgYAojIyBQb2lzc29uIHJlZ3Jlc3Npb246CmdsbSh5IH4geDEgKyB4MiArIC4uLiArIHhwLCBmYW1pbHk9cG9pc3NvbiwgZGF0YT15b3VyX2RhdGFfZnJhbWUpCiMjIExvZ2lzdGljIChha2EgQmlub21pYWwpIHJlZ3Jlc3Npb246CmdsbSh5IH4geDEgKyB4MiArIC4uLiArIHhwLCBmYW1pbHk9Ymlub21pYWwsIGRhdGE9eW91cl9kYXRhX2ZyYW1lKQpgYGAKCkl0cyBvdXRwdXQgbG9va3Mgc2ltaWxhciB0byBgbG1gLiBJdCdzIGFsc28gYSBsaXN0IGRpc2d1aXNlZCBhcyB0ZXh0LiBJdCBhbHNvIHNob3dzIG1vcmUgd2hlbiB5b3UgdXNlIHRoZSBgc3VtbWFyeWAgZnVuY3Rpb24uIEl0IGFsc28gd29ya3Mgd2l0aCB0aGUgYHByZWRpY3RgIGZ1bmN0aW9uLiBJdCBhbHNvIGJlY29tZXMgdGlkaWVyIHdoZW4gdXNlZCBpbiBjb25qdW5jdGlvbiB3aXRoIHRoZSBgYnJvb21gIHBhY2thZ2UuCgojIyMgMS4zIE90aGVycy4uLgoKSGVyZSBhcmUgc29tZSBvdGhlciBwYWNrYWdlcy9mdW5jdGlvbnMgeW91IG1pZ2h0IGZpbmQgdXNlZnVsIHRvIGZpdCBtb2RlbHM6CgotIChHZW5lcmFsaXplZCkgTWl4ZWQgRWZmZWN0cyBNb2RlbHMKICAgIC0gVHdvIFIgcGFja2FnZXMgYXJlIGF2YWlsYWJsZTogYGxtZTRgIGFuZCBgbmxtZWAuIAogICAgICAgIC0gQ2hlY2sgb3V0IFt0aGlzXShodHRwOi8vc3RhdHMuc3RhY2tleGNoYW5nZS5jb20vcXVlc3Rpb25zLzUzNDQvaG93LXRvLWNob29zZS1ubG1lLW9yLWxtZTQtci1saWJyYXJ5LWZvci1taXhlZC1lZmZlY3RzLW1vZGVscykgZGlzY3Vzc2lvbiBvbiBDcm9zcyBWYWxpZGF0ZWQgZm9yIGEgY29tcGFyaXNvbiBvZiB0aGUgdHdvIHBhY2thZ2VzLgogICAgLSBJJ3ZlIGZvdW5kIHRoZSBmdW5jdGlvbiBgZ2xtZXJgIGluIHRoZSBgbG1lNGAgcGFja2FnZSB0byBiZSBmcnVpdGZ1bC4KLSBLZXJuZWwgc21vb3RoaW5nIChpLmUuIGZpdHRpbmcgYSAic21vb3RoZXIiKTogY2hlY2sgb3V0IHRoZSBgbG9lc3NgIGZ1bmN0aW9uLgotIEdlbmVyYWxpemVkIEFkZGl0aXZlIE1vZGVsczogVGhlIGBnYW1gIGZ1bmN0aW9uIGluIF9laXRoZXJfIHRoZSBgZ2FtYCBwYWNrYWdlIG9yIGBtZ2N2YCBwYWNrYWdlLgotIFJvYnVzdCBsaW5lYXIgcmVncmVzc2lvbjogVGhlIGBybG1gIGZ1bmN0aW9uIGluIHRoZSBgTUFTU2AgcGFja2FnZSBpcyB5b3VyIGZyaWVuZC4KLSBSZWd1bGFyaXplZCByZWdyZXNzaW9uIChHTE0pIChsYXNzbywgZWxhc3RpYyBuZXQsIG9yIHJpZGdlIHJlZ3Jlc3Npb24pOiBVc2UgdGhlIGBnbG1uZXRgIGZ1bmN0aW9uIGluIHRoZSBgZ2xtbmV0YCBwYWNrYWdlLgogICAgLSBQUzogSSBfaGlnaGx5XyByZWNvbW1lbmQgdGhpcyBpZiB5b3UgaGF2ZSBtb3JlIHByZWRpY3RvcnMvY292YXJpYXRlcy9mZWF0dXJlcyB0aGFuIHlvdSBrbm93IHdoYXQgdG8gZG8gd2l0aC4uLiB0aGlzIHdpbGwgd2VlZCBvdXQgdGhlIHVubmVjZXNzYXJ5IG9uZXMsIF9hbmRfIHByb2R1Y2UgYSBtb2RlbCB3aXRoIGdvb2QgcHJlZGljdGlvbiBhY2N1cmFjeSBhdCB0aGUgc2FtZSB0aW1lLgoKCiMjIDIuIE1vcmUgYGRwbHlyYAoKVGhlcmUncyBvbmUgbW9yZSBpbXBvcnRhbnQgdGhpbmcgd2l0aCBgZHBseXJgIHRoYXQgeW91IG91Z2h0IHRvIGtub3c6IGFwcGx5aW5nIGBtdXRhdGVgIHRvIGEgZ3JvdXBlZCB0aWJibGUuCgpSZW1lbWJlciBhcHBseWluZyBgc3VtbWFyaXplYCB0byBhIGdyb3VwZWQgdGliYmxlPwoKYGBge3J9CmdhcG1pbmRlciAlPiUKICAgIGdyb3VwX2J5KGNvbnRpbmVudCkgJT4lIAogICAgc3VtbWFyaXplKG1lYW5fZ2RwUGVyY2FwID0gbWVhbihnZHBQZXJjYXApLAogICAgICAgICAgICAgIG5fY291bnRyaWVzICAgID0gbGVuZ3RoKGdkcFBlcmNhcCkpCmBgYAoKV2VsbCwgd2UgY2FuIGFsc28gYXBwbHkgYG11dGF0ZWAgdG8gZWFjaCBncm91cCwgdG9vLiBGb3IgZXhhbXBsZSwgbGV0J3MgY2FsY3VsYXRlIHRoZSBncm93dGggaW4gcG9wdWxhdGlvbiBzaW5jZSB0aGUgZmlyc3QgeWVhciBvbiByZWNvcmQgX2ZvciBlYWNoIGNvdW50cnlfOgoKYGBge3J9CmdhcG1pbmRlciAlPiUgCiAgICBncm91cF9ieShjb3VudHJ5KSAlPiUgCiAgICBtdXRhdGUocG9wX2dyb3d0aCA9IHBvcCAtIHBvcFsxXSkKYGBgCgpOb3RpY2UgdGhhdCBgZHBseXJgIGhhcyByZXRhaW5lZCB0aGUgb3JpZ2luYWwgZ3JvdXBpbmcgLS0gaXQgaGFzbid0IHBlYWxlZCBiYWNrIG9uZSBsZXZlbCBvZiBncm91cGluZy4gVGhhdCdzIGJlY2F1c2UgdGhlcmUncyBzdGlsbCBtb3JlIHRoYW4gb25lIHJvdyBmb3IgZWFjaCBncm91cCEKCkhvdyBhYm91dCBncm93dGggY29tcGFyZWQgdG8gYDE5NzJgPwoKYGBge3J9CmdhcG1pbmRlciAlPiUgCiAgICBncm91cF9ieShjb3VudHJ5KSAlPiUgCiAgICBtdXRhdGUocG9wX2dyb3d0aCA9IHBvcCAtIHBvcFt5ZWFyPT0xOTcyXSkKYGBgCgpJbiBnZW5lcmFsLCB0aGlzIHR5cGUgb2YgImdyb3VwZWQgbXV0YXRpb24iIGlzIHVzZWZ1bCBmb3IgX3dpbmRvdyBmdW5jdGlvbnNfLiBXaGF0J3MgdGhhdD8gV2VsbCwgbGV0J3Mgc2VlIG90aGVyIHR5cGVzIG9mIGZ1bmN0aW9ucyBpbiBSOgoKLSBfX1ZlY3Rvcml6ZWQgRnVuY3Rpb25zX186IFRoZXNlIHRha2UgYSB2ZWN0b3IsIGFuZCBvcGVyYXRlIG9uIGVhY2ggY29tcG9uZW50IGluZGVwZW5kZW50bHkgdG8gcmV0dXJuIGEgdmVjdG9yIG9mIHRoZSBzYW1lIGxlbmd0aC4gSW4gb3RoZXIgd29yZHMsIHRoZXkgd29yayBlbGVtZW50LXdpc2UuIAogICAgLSBFeGFtcGxlcyBhcmUgYGNvc2AsIGBzaW5gLCBgbG9nYCwgYGV4cGAsIGByb3VuZGAuIAogICAgLSBXZSBkb24ndCBuZWVkIHRvIGBncm91cF9ieWAgaW4gb3JkZXIgdG8gYG11dGF0ZWAgd2l0aCB0aGVzZS4gCi0gX19BZ2dyZWdhdGUgRnVuY3Rpb25zX186IFRoZXNlIHRha2UgYSB2ZWN0b3IsIGFuZCByZXR1cm4gYSB2ZWN0b3Igb2YgbGVuZ3RoIDEgLS0gYXMgaWYgImFnZ3JlZ2F0aW5nIiB0aGUgdmFsdWVzIGluIHRoZSB2ZWN0b3IgaW50byBhIHNpbmdsZSB2YWx1ZS4KICAgIC0gRXhhbXBsZXMgYXJlIGBtZWFuYCwgYHNkYCwgYGxlbmd0aGAsIGB0eXBlb2ZgLgogICAgLSBXZSB1c2UgdGhlc2UgaW4gZHBseXIncyBgc3VtbWFyaXNlYCBmdW5jdGlvbi4KLSBfX1dpbmRvdyBGdW5jdGlvbnNfXzogdGhlc2UgdGFrZSBhIHZlY3RvciwgYW5kIHJldHVybiBhIHZlY3RvciBvZiB0aGUgc2FtZSBsZW5ndGggX3RoYXQgZGVwZW5kcyBvbiBvdGhlciB2YWx1ZXMgaW4gdGhlIHZlY3Rvcl8uIAogICAgLSBFeGFtcGxlcyBhcmUgYGxhZ2AsIGByYW5rYCwgYGN1bXN1bWAuIAogICAgLSBTZWUgdGhlIFt3aW5kb3ctZnVuY3Rpb25zXShodHRwczovL2NyYW4uci1wcm9qZWN0Lm9yZy93ZWIvcGFja2FnZXMvZHBseXIvdmlnbmV0dGVzL3dpbmRvdy1mdW5jdGlvbnMuaHRtbCkgdmlnbmV0dGUgZm9yIHRoZSBgZHBseXJgIHBhY2thZ2UuIAoKIyMgMy4gTW9yZSBgZ2dwbG90YAoKKEhhdmUgeW91IHNlZW4gdGhlIFtnZ3Bsb3QyIGNoZWF0c2hlZXRdKGh0dHBzOi8vd3d3LnJzdHVkaW8uY29tL3dwLWNvbnRlbnQvdXBsb2Fkcy8yMDE1LzAzL2dncGxvdDItY2hlYXRzaGVldC5wZGYpPyBJdCBjb250YWlucyBfX2EgbG90X18gb2YgdXNlZnVsIGluZm9ybWF0aW9uIG9uIHR3byBwYWdlcyEpCgojIyMgMy4xIGB0aGVtZWAgbGF5ZXJzCgpZb3UgY2FuIGNoYW5nZSB0aGUgbG9vayBvZiBhIHBsb3QgYnkgYWRkaW5nIGEgYHRoZW1lYCBsYXllciB0byB5b3VyIGdncGxvdCBsYXllcnMuIFRoaXMgZnVuY3Rpb24gZG9lcyBfX25vdF9fIGFjdHVhbGx5IGNoYW5nZSB0aGUgIm5hdHVyZSIgb2YgdGhlIHBsb3QgaXRzZWxmIC0tIG9ubHkgdGhlIGxvb2shIGkuZS4sIHRoZSBzby1jYWxsZWQgIm5vbi1kYXRhIiB0eXBlIGRpc3BsYXlzLiAKCkV4YW1wbGVzOgogICAgLSBmb250CiAgICAtIGp1c3RpZmljYXRpb24gb2YgdGl0bGVzCiAgICAtIHJvdGF0aW9uIG9mIGxhYmVscwogICAgLSBiYWNrZ3JvdW5kIGNvbG91cgogICAgLSBsaW5lIHRoaWNrbmVzcwogICAgLSBldGMuLi4KClRoZXJlIGFyZSBbImNvbXBsZXRlIHRoZW1lcyJdKGh0dHA6Ly9nZ3Bsb3QyLnRpZHl2ZXJzZS5vcmcvcmVmZXJlbmNlL2dndGhlbWUuaHRtbCkgdGhhdCBjb21lIHdpdGggYGdncGxvdDJgLCBteSBmYXZvdXJpdGUgYmVpbmcgYHRoZW1lX2J3YCAoSSd2ZSBncm93biB0aXJlZCBvZiB0aGUgZGVmYXVsdCBncmF5IGJhY2tncm91bmQsIHNvIGB0aGVtZV9id2AgaXMgcmVmcmVzaGluZykuCgpMZXQncyBzZWUgYW4gZXhhbXBsZToKCmBgYHtyfQpwMSA8LSBnZ3Bsb3QoZ2FwbWluZGVyLCBhZXMoZ2RwUGVyY2FwLCBsaWZlRXhwKSkgKwogICAgZmFjZXRfd3JhcCh+IGNvbnRpbmVudCkgKwogICAgZ2VvbV9wb2ludChjb2xvdXI9IiMzODZDQjAiLCBhbHBoYT0wLjIpICsKICAgIHNjYWxlX3hfbG9nMTAoKQpwMSArIHRoZW1lX2J3KCkKYGBgCgpUaGUgZ2VuZXJhbCBgdGhlbWVgIGZ1bmN0aW9uIGdpdmVzIHlvdSB2YXN0IGZ1bmN0aW9uYWxpdHkuLi4ganVzdCBjaGVjayBpdHMgZG9jdW1lbnRhdGlvbiB0byBzZWUgd2hhdCB0aGluZ3MgeW91IGNhbiBjaGFuZ2UuIFRoZSBhcmd1bWVudHMgb2YgYHRoZW1lYCBmb2xsb3cgYSBuYW1pbmcgY29udmVudGlvbjogYGdlbmVyYWwudG8uc3BlY2lmaWNgCgpGb3IgZXhhbXBsZSwKICAgIC0gYGF4aXMudGl0bGVgIHdpbGwgYWxsb3cgeW91IHRvIGNoYW5nZSB0aGUgZm9udCBvZiB0aGUgYXhpcyB0aXRsZXMuCiAgICAtIGBheGlzLnRpdGxlLnhgIGRvZXMgdGhlIHNhbWUsIGJ1dCBmb2N1c3NlcyBvbiB0aGUgYHhgIGF4aXMuCgpfX05vdGVfXzogWW91IF9jYW4ndF8gY2hhbmdlIHRoZSBhY3R1YWwgd29yZHMgdGhpcyB3YXkhIFRoYXQncyBjaGFuZ2luZyB0aGUgbmF0dXJlIG9mIHRoZSBwbG90LCBhbmQgbm90IHRoZSBfbG9va18gb2YgdGhlIHBsb3QuIAoKT25jZSB3ZSd2ZSBjaG9zZW4gYW4gYXJndW1lbnQgbmFtZSwgd2UgbmVlZCB0byBzcGVjaWZ5IGl0cyB2YWx1ZS4gVGhpcyBpcyBhbG1vc3QgYWx3YXlzIHRoZSBvdXRwdXQgb2Ygb25lIG9mIHRoZSBmb2xsb3dpbmcgZnVuY3Rpb25zOgoKLSBgZWxlbWVudF9ibGFua2AgKGJhc2ljYWxseSBtZWFucyByZXBsYWNlIHdpdGggIm5vdGhpbmciKQotIGBlbGVtZW50X3JlY3RgOiBhbGxvd3MgdXMgdG8gc3BlY2lmeSBmZWF0dXJlcyBvZiBhIF9yZWN0YW5nbGVfLgotIGBlbGVtZW50X2xpbmVgOiBhbGxvd3MgdXMgdG8gc3BlY2lmeSBmZWF0dXJlcyBvZiBhIF9saW5lXy4KLSBgZWxlbWVudF90ZXh0YDogYWxsb3dzIHVzIHRvIHNwZWNpZnkgZm9udC4gCgpDaGVjayBvdXQgdGhlaXIgZG9jdW1lbnRhdGlvbiB0byBzZWUgZXhhY3RseSBob3cgZWFjaCBmZWF0dXJlIGlzIG1vZGlmaWVkLgoKLSBfX0V4YW1wbGVfXzogVG8gYHAxYCBhYm92ZSwgZG8gYWxsIG9mIHRoZSBmb2xsb3dpbmcuLi4uCiAgICAtIGNoYW5nZSB0aGUgYmFja2dyb3VuZCBzdHJpcCBjb2xvdXIgdG8gb3JhbmdlLCAKICAgIC0gY2hhbmdlIHRoZSBheGlzIHRpdGxlcycgZm9udCBzaXplcyB0byAxNCwgYW5kIAogICAgLSBjaGFuZ2UgdGhlIHBhbmVsIHRpdGxlcycgZm9udCBzaXplcyB0byAxNCBhbmQgYm9sZGVkLgoKYGBge3J9CnAxICsKICAgIHRoZW1lKHN0cmlwLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbD0ib3JhbmdlIiksCiAgICAgICAgICBheGlzLnRpdGxlID0gZWxlbWVudF90ZXh0KHNpemU9MTQpLAogICAgICAgICAgc3RyaXAudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplPTE0LCBmYWNlPSJib2xkIikpCmBgYAoKX19Bbm90aGVyIGV4YW1wbGVfXzogZG8gdGhlIHNhbWUsIGJ1dCBpbiBjb25qdW5jdGlvbiB3aXRoIHRoZSBgdGhlbWVfYndgIChub3RpY2UgdGhlIG9yZGVyKQoKYGBge3J9CiMjIENvcnJlY3Q6CnAxICsKICAgIHRoZW1lX2J3KCkgKwogICAgdGhlbWUoc3RyaXAuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChmaWxsPSJvcmFuZ2UiKSwKICAgICAgICAgIGF4aXMudGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZT0xNCksCiAgICAgICAgICBzdHJpcC50ZXh0ID0gZWxlbWVudF90ZXh0KHNpemU9MTQsIGZhY2U9ImJvbGQiKSkKIyMgSW5jb3JyZWN0OgpwMSArCiAgICB0aGVtZShzdHJpcC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGZpbGw9Im9yYW5nZSIpLAogICAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplPTE0KSwKICAgICAgICAgIHN0cmlwLnRleHQgPSBlbGVtZW50X3RleHQoc2l6ZT0xNCwgZmFjZT0iYm9sZCIpKSArCiAgICB0aGVtZV9idygpICAjIE92ZXJyaWRlcyB0aGUgcHJldmlvdXMgYHRoZW1lYCBjYWxsIQpgYGAKCgojIyMgMy4yIE1vZGlmeWluZyBzY2FsZXMKClJlY2FsbCB0aGF0IHdlIHVzZSBzb21lIF9zY2FsZV8gdG8gcmVwcmVzZW50IHRoZSBfcmFuZ2Ugb2YgdmFsdWVzIHRoYXQgYSB2YXJpYWJsZSB0YWtlc18gaW4gb3VyIGRhdGEuIGBnZ3Bsb3RgIGNob29zZXMgZGVmYXVsdHMgZm9yIHRoaXMgc2NhbGUsIGJ1dCB3ZSBjYW4gY2hhbmdlIHRob3NlLgoKQ2hlY2sgb3V0IFt0aGlzIHR1dG9yaWFsIGJ5IEhhZGxleSBXaWNraGFtXShodHRwczovL2dpdGh1Yi5jb20vaGFkbGV5L2dncGxvdDItYm9vay9ibG9iL21hc3Rlci9zY2FsZXMucm1kKSBmb3Igc2NhbGVzLiAKCldlIGNhbiBtb2RpZnkgc2NhbGVzIHVzaW5nIGEgc3VpdGUgb2YgZnVuY3Rpb25zIHRoYXQgaGF2ZSB0aGUgZm9sbG93aW5nIG5hbWluZyBjb252ZW50aW9uOiBgc2NhbGVfYV9iYCwgd2hlcmU6CgotIGBhYCBpcyB0aGUgc2NhbGUgeW91IHdhbnQgdG8gY2hhbmdlLiBJcyBpdCBgY29sb3VyYD8gYHNpemVgPyBgeGAgcG9zaXRpb24/Ci0gYGJgIHR5cGljYWxseSBzcGVha3MgdG8gdGhlIG5hdHVyZSBvZiB0aGUgdmFyaWFibGUuIGBjb250aW51b3VzYCBpcyB5b3VyIHZhcmlhYmxlIGlzIGNvbnRpbnVvdXM7IGBkaXNjcmV0ZWAgaWYgZGlzY3JldGUuIEJ1dCwgY291bGQgYmUgb3RoZXIgdGhpbmdzIGluIGNlcnRhaW4gY2FzZXMsIGxpa2UgYGxvZzEwYCwgb3IgYGRhdGVgIGlmIHlvdXIgdmFyaWFibGUgY29uc2lzdHMgb2YgZGF0ZXMuIGBtYW51YWxgIGlzIGFuIG9wdGlvbiB0b28uCgpfX0V4YW1wbGVzX186IGBzY2FsZV94X2NvbnRpbnVvdXNgLCBgc2NhbGVfY29sb3VyX2Rpc2NyZXRlYCwgYHNjYWxlX3lfc3FydGAuCgpBcyB1c3VhbCBpbiBnZ3Bsb3QsIHRoZXNlIGZ1bmN0aW9ucyBhcmUgYWRkZWQgYXMgYSBsYXllci4gCgojIyMjIFVzZWZ1bCBhcmd1bWVudHMKClRoZXJlIGFyZSBtYW55IHVzZWZ1bCBhcmd1bWVudHMgaGVyZS4gU29tZSBhcmUgbW9yZSBzZWxmLWV4cGxhbmF0b3J5IHRoYW4gb3RoZXJzLiAKCi0gYG5hbWVgLiBUaGUgZmlyc3QgYXJndW1lbnQuIEluZGljYXRlIHRoZSBuYW1lIG9mIHRoZSBzY2FsZS9sZWdlbmQgaGVyZS4KICAgIC0gWW91IGNhbiBhbHNvIHVzZSB0aGUgYGxhYnNgIGZ1bmN0aW9uIGZvciBYIGFuZCBZIGF4ZXMsIGFuZCBldmVuIHBsb3QgdGl0bGUuCgpgYGB7cn0KcDEgKyBzY2FsZV95X2NvbnRpbnVvdXMoIkxpZmUgRXhwZWN0YW5jeSIpCnAxICsgbGFicyh4PSJHRFAgcGVyIGNhcGl0YSIsIAogICAgICAgICAgeT0iTGlmZSBFeHBlY3RhbmN5IiwKICAgICAgICAgIHRpdGxlPSJNeSBQbG90IikKCmdncGxvdChnYXBtaW5kZXIsIGFlcyhnZHBQZXJjYXAsIGxpZmVFeHApKSArCiAgICBnZW9tX3BvaW50KGFlcyhjb2xvdXI9Y29udGluZW50KSwKICAgICAgICAgICAgICAgYWxwaGE9MC4yKSArCiAgICBzY2FsZV9jb2xvdXJfZGlzY3JldGUoIkNvbnRpbmVudHMgb2ZcbiB0aGUgV29ybGQiKQpgYGAKCi0gYGJyZWFrc2AgKFR5cGljYWxseSBvZiBhIGNvbnRpbnVvdXMgc2NhbGUpLiBIZXJlLCB5b3UgZ2V0IHRvIHNwZWNpZnkgX3doZXJlXyBhbG9uZyB0aGUgc2NhbGUgeW91J2QgbGlrZSB0byBkaXNwbGF5IGEgdmFsdWUuIAogICAgLSBOdW1iZXJzIGFyZSBvbiB0aGUgc2NhbGUgb2YgdGhlIGRhdGEgKHN1Y2ggYXMgcG9wdWxhdGlvbiksIG5vdCB0aGUgZ2VvbWV0cmljIHNjYWxlIChzdWNoIGFzIGEgaGV4IGNvbG91ciBjb2RlLCBvciBudW1iZXIgb2YgcGl4ZWxzIG92ZXIgaW4gYSBwbG90KS4gCgpgYGB7cn0KIyMgTG9nIGxpbmVzOgpwMSArIHNjYWxlX3hfbG9nMTAoYnJlYWtzPWMoKDE6MTApKjEwMDAsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAoMToxMCkqMTAwMDApKQoKcDIgPC0gZ2dwbG90KGdhcG1pbmRlciwgYWVzKGdkcFBlcmNhcCwgbGlmZUV4cCkpICsKICAgIGdlb21fcG9pbnQoYWVzKGNvbG91cj1wb3AvMTBeOSksCiAgICAgICAgICAgICAgIGFscGhhPTAuMikKIyMgRGVmYXVsdCBicmVha3MKcDIgKyBzY2FsZV9jb2xvdXJfY29udGludW91cygiUG9wdWxhdGlvblxuaW4gYmlsbGlvbnMiKQojIyBOZXcgYnJlYWtzCnAyICsgc2NhbGVfY29sb3VyX2NvbnRpbnVvdXMoIlBvcHVsYXRpb25cbmluIGJpbGxpb25zIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBicmVha3M9c2VxKDAsMixieT0wLjIpKQpgYGAKCi0gYGxhYmVsc2AuIFRleHQgdG8gcmVwbGFjZSB0aGUgZGF0YSB2YWx1ZSBsYWJlbHMuIE1vc3QgdXNlZnVsIGZvciBkaXNjcmV0ZSBkYXRhLgoKYGBge3J9CiMjIE5vdCBhIGdvb2QgaWRlYToKcDIgKyBzY2FsZV9jb2xvdXJfY29udGludW91cygiTXkgb2RkXG5wb3B1bGF0aW9uXG5zY2FsZSIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWtzPWMoMC4yLCAwLjcsIDEuMiksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbGFiZWxzPWMoInNtYWxsIiwgImJpZ2dlciIsICJiaWciKSkKCiMjIERpc2NyZXRlIHNjYWxlOgpnZ3Bsb3QoZ2FwbWluZGVyLCBhZXMoZ2RwUGVyY2FwLCBsaWZlRXhwKSkgKwogICAgZ2VvbV9wb2ludChhZXMoY29sb3VyPWNvbnRpbmVudCksCiAgICAgICAgICAgICAgIGFscGhhPTAuMikgKwogICAgc2NhbGVfY29sb3VyX2Rpc2NyZXRlKGxhYmVscz1jKCJBZiIsICJBbSIsICJBcyIsICJFdSIsICJPYyIpKQpgYGAKCi0gYGxpbWl0c2AuIExvd2VyIGFuZCB1cHBlciBib3VuZHMgb2YgdGhlIGRhdGEgdGhhdCB5b3UnZCBsaWtlIGRpc3BsYXllZC4gTGVhdmUgb25lIGFzIGBOQWAgaWYgeW91IHdhbnQgdG8gdXNlIHRoZSBkZWZhdWx0LgoKYGBge3J9CnAxICsgc2NhbGVfeV9jb250aW51b3VzKGxpbWl0cz1jKDYwLE5BKSkKYGBgCgotIGBwb3NpdGlvbmAuIFBvc2l0aW9uIG9mIHRoZSBzY2FsZS4gQWxzbyBjb250cm9sbGFibGUgdXNpbmcgYHRoZW1lYCBmb3IgdGhlIGxlZ2VuZC4KCmBgYHtyfQpwMSArIHNjYWxlX3lfY29udGludW91cyhwb3NpdGlvbj0icmlnaHQiKQpwMiArIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iKQpgYGAKCiMjIDQuIEV4ZXJjaXNlcy4gCgpQcmFjdGljZSB0aGVzZSBjb25jZXB0cyBpbiB0aGUgZm9sbG93aW5nIGV4ZXJjaXNlcy4KCl9fRXhlcmNpc2UgMV9fOiBTdXBwb3NlIHdlIHdhbnQgdG8gY2FsY3VsYXRlIHNvbWUgcXVhbnRpdHkgZm9yIGVhY2ggY291bnRyeSBpbiB0aGUgYGdhcG1pbmRlcmAgZGF0YSBzZXQuIEZvciBlYWNoIG9mIHRoZSBmb2xsb3dpbmcgcXVhbnRpdGllcywgaW5kaWNhdGUgd2hldGhlciB0aGUgZnVuY3Rpb24gaXMgX3ZlY3Rvcml6ZWRfLCBfYWdncmVnYXRlXywgb3IgX3dpbmRvd18sIGFuZCB1c2UgYGRwbHlyYCBmdW5jdGlvbnMgdG8gY2FsY3VsYXRlIHRoZSBzcGVjaWZpZWQgdmFyaWFibGUuCgotIFRoZSBjaGFuZ2UgaW4gcG9wdWxhdGlvbiBmcm9tIDE5NjIgdG8gMTk3Mi4KCmBgYHtyfQojIyBJdCdzIGFuIEFnZ3JlZ2F0ZSBmdW5jdGlvbi4KZ2FwbWluZGVyICU+JSAKICAgIGZpbHRlcih5ZWFyICVpbiUgYygxOTYyLCAxOTcyKSkgJT4lIAogICAgYXJyYW5nZSh5ZWFyKSAlPiUgCiAgICBncm91cF9ieShjb3VudHJ5KSAlPiUgCiAgICBzdW1tYXJpemUocG9wX2NoZz1kaWZmKHBvcCkpCmdhcG1pbmRlciAlPiUgCiAgICBncm91cF9ieShjb3VudHJ5KSAlPiUgCiAgICBzdW1tYXJpc2UocG9wX2NoZz1wb3BbeWVhcj09MTk3Ml0tcG9wW3llYXI9PTE5NjJdKQpgYGAKCgotIFRoZSBwb3B1bGF0aW9uLCBpbiBiaWxsaW9ucy4KCmBgYHtyfQojIyBJdCdzIGEgdmVjdG9yaXplZCBmdW5jdGlvbi4KZ2FwbWluZGVyICU+JQogICAgbXV0YXRlKHBvcF9pbl9iaWxsID0gcG9wLzEwXjkpCmBgYAoKCi0gVGhlIGxhZ2dlZCBnZHBQZXJjYXAKICAgIC0gaS5lLiwgdGhlIHZhbHVlIHRoYXQgYXBwZWFycyBmb3IgMTk2MiB3b3VsZCBiZSB0aGUgZ2RwUGVyY2FwIGluIDE5NTcgKHRoZSBwcmV2aW91cyBlbnRyeSkuCiAgICAtIEhpbnQ6IHVzZSB0aGUgYGxhZ2AgZnVuY3Rpb24sIHRoZW4gZmlsdGVyIG91dCB0aGUgYE5BYCdzIGNyZWF0ZWQgd2l0aCB0aGUgYGlzLm5hYCBmdW5jdGlvbi4KCmBgYHtyfQojIyBJdCdzIGEgd2luZG93IGZ1bmN0aW9uLgpnYXBtaW5kZXIgJT4lIAogICAgZ3JvdXBfYnkoY291bnRyeSkgJT4lIAogICAgYXJyYW5nZSh5ZWFyKSAlPiUgCiAgICBtdXRhdGUobGFnX2dkcFBlcmNhcD1sYWcoZ2RwUGVyY2FwKSkgJT4lIAogICAgZmlsdGVyKCFpcy5uYShsYWdfZ2RwUGVyY2FwKSkKYGBgCgoKCl9fRXhlcmNpc2UgMl9fOiBGb3IgdGhlIGBnYXBtaW5kZXJgIGRhdGFzZXQsIG1ha2UgYSBzcGFnaGV0dGkgcGxvdCBzaG93aW5nIHRoZSBwb3B1bGF0aW9uIHRyZW5kIChpbiBtaWxsaW9ucykgb3ZlciB0aW1lIGZvciBlYWNoIGNvdW50cnksIGZhY2V0dGVkIGJ5IGNvbnRpbmVudC4gTWFrZSBhcyBtYW55IG9mIHRoZSBmb2xsb3dpbmcgbW9kaWZpY2F0aW9ucyBhcyB5b3UgY2FuOgoKLSBDb2xvdXIgZWFjaCBsaW5lIGJ5IHRoZSBsb2cgbWF4aW11bSBnZHBQZXJjYXAgZXhwZXJpZW5jZWQgYnkgdGhlIGNvdW50cnkuCi0gUm90YXRlIHRoZSB4LWF4aXMgbGFiZWxzIHRvIGJlIHZlcnRpY2FsLgotIFJlbW92ZSB0aGUgeC1heGlzIHRpdGxlLgotIEdpdmUgdGhlIGxlZ2VuZCBhbiBhcHByb3ByaWF0ZSB0aXRsZS4KLSBQdXQgdGhlIHktYXhpcyBvbiBhIGxvZy1zY2FsZS4gCi0gUmVuYW1lIHRoZSB5LWF4aXMgdGl0bGUuCi0gQWRkIG1vcmUgbnVtYmVycyBhbG9uZyB0aGUgeS1heGlzLgotIEdpdmUgdGhlIHBsb3QgYSB0aXRsZSwgYW5kIGNlbnRlciB0aGUgdGl0bGUuIAotIE9ubHkgbGFiZWwgdGhlIHggYXhpcyB3aXRoIHllYXJzIDE5NTAsIDE5NzUsIGFuZCAyMDAwLgotIE1vdmUgdGhlIGNvbG91ciBzY2FsZSB0byB0aGUgYm90dG9tLgotIFJlbmFtZSB0aGUgY29sb3VyIGxlZ2VuZAoKYGBge3J9CmdhcG1pbmRlciAlPiUgCiAgICBncm91cF9ieShjb3VudHJ5KSAlPiUgCiAgICBtdXRhdGUobWF4X2dkcFBlcmNhcD1tYXgoZ2RwUGVyY2FwKSkgJT4lIAogICAgZ2dwbG90KGFlcyh5ZWFyLCBwb3AvMTBeNikpICsgCiAgICBmYWNldF93cmFwKH4gY29udGluZW50KSArCiAgICBnZW9tX2xpbmUoYWVzKGdyb3VwPWNvdW50cnksCiAgICAgICAgICAgICAgICAgIGNvbG91cj1sb2cobWF4X2dkcFBlcmNhcCkpLAogICAgICAgICAgICAgIGFscGhhPTAuMjUpICsKICAgIHRoZW1lX2J3KCkgKyAjIEkgYWRkZWQgdGhpcyBiZWNhdXNlIEkgbGlrZSB0aGlzIHRoZW1lLgogICAgbGFicyh0aXRsZT0iUG9wdWxhdGlvbiBUcmVuZHMiKSArCiAgICBzY2FsZV95X2xvZzEwKCJQb3B1bGF0aW9uIChtaWxsaW9ucykiLAogICAgICAgICAgICAgICAgICBicmVha3M9YygwLjEsIDEsIDEwLCAxMDAsIDEwMDApLAogICAgICAgICAgICAgICAgICBsYWJlbHM9YygwLjEsIDEsIDEwLCAxMDAsIDEwMDApKSArCiAgICBzY2FsZV94X2NvbnRpbnVvdXMoIiIsIGJyZWFrcz1jKDE5NTAsIDE5NzUsIDIwMDApKSArCiAgICBzY2FsZV9jb2xvdXJfY29udGludW91cygibG9nIE1heGltdW1cbkdEUCBwZXIgY2FwLiIpICsKICAgIHRoZW1lKGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KGFuZ2xlPTkwKSwKICAgICAgICAgIHBsb3QudGl0bGUgPSBlbGVtZW50X3RleHQoaGp1c3Q9MC41KSwKICAgICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJib3R0b20iKQpgYGAK