In this session, we will look at basic functions in R that will help us in running some inferential statistics. These will help us to evaluate the relationship between one (or more) predictor(s) and an outcome. We will start with a basic example. Using the dataframe “cars”.

2 Basic statistics

2.1 Built-in datasets

We will use one of the built in R. You can check all available datasets in R using the following:

datasets have been moved from package 'base' to package 'datasets'datasets have been moved from package 'stats' to package 'datasets'

We will use the iris dataset from the package MASS

2.2 Checking structure and summaries

2.2.1 Structure

'data.frame':   150 obs. of  5 variables:
 $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
 $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
 $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
 $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
 $ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...

We have a dataframe with 150 observations and 5 variables; 4 numeric and 1 factor with 3 levels.

2.2.2 Summary

We summarise the data to see the trends:

  Sepal.Length    Sepal.Width     Petal.Length    Petal.Width   
 Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100  
 1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  
 Median :5.800   Median :3.000   Median :4.350   Median :1.300  
 Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199  
 3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800  
 Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500  
       Species  
 setosa    :50  
 versicolor:50  
 virginica :50

So we have an equal dataframe (50 observations under each level of the factor Species), with no missing values (aka NA).

2.2.3 Advanced

What if you want to summarise the data and get the mean, SD, by each level of the factor?

2.2.4 Up to you

Do some additional summaries

2.3 Plot

We can make a boxplot of the data and add a trend line. This allows us to visualise the median, and quantiles in addition to the standard deviation and any outliers… All in the same plot!

Here I have used the variable Sepal.Length. You can use any of the additional variables to plot the data.

2.4 Correlation tests

2.4.1 Basic correlations

We use the function cor to obtain the pearson correlation and cor.test to run a basic correlation test on our data with significance testing

[1] 0.8717538

    Pearson's product-moment correlation

data:  iris$Sepal.Length and iris$Petal.Length
t = 21.646, df = 148, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.8270363 0.9055080
sample estimates:
      cor 
0.8717538

2.4.1.1 Up to you

Can you check whether there is a correlation between the Sepal.Length and the Petal.Width? What about Petal.Length and Petal.Width?

2.4.2 Using the package psycho

Above, we did a correlation test on two predictors. We run multiple correlations on multiple predictors with corrections. You can get a table, and a plot. (see here https://www.r-bloggers.com/beautiful-and-powerful-correlation-tables-in-r/amp/)

2.4.2.1 Defaults

Pearson Full correlation (p value correction: holm):

   - Sepal.Length / Sepal.Width:   Results of the Pearson correlation showed a non significant small, and negative association between Sepal.Length and Sepal.Width (r(148) = -0.12, p > .1).
   - Sepal.Length / Petal.Length:   Results of the Pearson correlation showed a significant large, and positive association between Sepal.Length and Petal.Length (r(148) = 0.87, p < .001***).
   - Sepal.Width / Petal.Length:   Results of the Pearson correlation showed a significant moderate, and negative association between Sepal.Width and Petal.Length (r(148) = -0.43, p < .001***).
   - Sepal.Length / Petal.Width:   Results of the Pearson correlation showed a significant large, and positive association between Sepal.Length and Petal.Width (r(148) = 0.82, p < .001***).
   - Sepal.Width / Petal.Width:   Results of the Pearson correlation showed a significant moderate, and negative association between Sepal.Width and Petal.Width (r(148) = -0.37, p < .001***).
   - Petal.Length / Petal.Width:   Results of the Pearson correlation showed a significant large, and positive association between Petal.Length and Petal.Width (r(148) = 0.96, p < .001***).

2.4.2.2 Changing parameters

You can change the following if you are interested in getting a spearman correlation compared to a pearson correlation: method = "spearman" (method = “pearson”, “spearman”, “kendall”).

The defaults have corrections for multiple comparisons using the holm method; you can use adjust="bonferroni" to use a bonferroni correction (adjust= “holm” (default), “bonferroni”, “fdr”, “none”)

2.4.3 Up to you

Run multiple correlations below by changing the method, or the adjustments. Any comments?

Up to now, we have done some basic summaries and checked the correlations in the data. The pearson correlations we have done provided us with significance levels related to the correlations between two numeric outcomes We do not know if the levels of our variable Species are statistically different based on one of the predictors. We use inferential statistics here.

2.5 First step into statistics

2.5.1 Basic t-tests

T.test are used when the outcome (DV) is numeric and the predictor is either numeric or categorical (with two levels only).

2.5.1.1 Numeric predictor

We can run a basic t-test on our data, using the function t-test. This looks at the two numeric outcomes and sees if they are different from each other


    Welch Two Sample t-test

data:  iris$Sepal.Length and iris$Petal.Length
t = 13.098, df = 211.54, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 1.771500 2.399166
sample estimates:
mean of x mean of y 
 5.843333  3.758000

2.5.1.2 Categorical predictor

In the iris dataset, we have a categorical predictor: Species which has three levels

[1] "setosa"     "versicolor" "virginica"

Let’s subset the data and evaluate the differences between setosa and versicolor using a t-test and a linear model. We will also check the normality and homogeneity of variance in the data

2.5.1.2.1 Subset
[1] "setosa"     "versicolor" "virginica"

But wait a minute… We just subsetted the data and our levels show three levels factor.. why? Let’s run a summary

    setosa versicolor  virginica 
        50         50          0

Aha! So the subsetting worked as the virginica cases are 0. We need to tell R to change the factor levels

[1] "setosa"     "versicolor"

Now this works better

2.5.1.2.2 Normality of distribution
2.5.1.2.2.1 Shapiro test

To check normality of distribution, we use the shapiro.test on the numeric outcome. Given that our predictor now has two levels. We need to subset the data again to check normality of the outcome Sepal.Length for each level of our factor Species


    Shapiro-Wilk normality test

data:  irisSubSet$Sepal.Length
W = 0.9777, p-value = 0.4595

    Shapiro-Wilk normality test

data:  irisSubVers$Sepal.Length
W = 0.97784, p-value = 0.4647

How to interpret this non-statistically significant result? This tells us that the distribution of the data is not statistically different from a normal distribution.

2.5.1.2.2.2 Density plot

We can also use a density plot to evaluate normality. The results show that both levels have bell shaped distributions.

2.5.1.2.3 Homogeneity of variance

Because our data is normally distributed, we can use the barlett test. If our data were non-normally distributed, we would use the leveneTest. We can check both.

2.5.1.2.3.1 Barlett test

    Bartlett test of homogeneity of variances

data:  Sepal.Length by Species
Bartlett's K-squared = 6.8917, df = 1, p-value = 0.00866
2.5.1.2.3.2 Levene test

    F test to compare two variances

data:  Sepal.Length by Species
F = 0.46634, num df = 49, denom df = 49, p-value = 0.008657
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
 0.2646385 0.8217841
sample estimates:
ratio of variances 
         0.4663429 
Levene's Test for Homogeneity of Variance (center = median)
      Df F value   Pr(>F)   
group  1  8.1727 0.005196 **
      98                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

In all cases, the statistically significant result indicates that there is evidence that the variance of two levels of the factor Species is statistically significant; i.e., the variances are not equal

2.5.1.2.4 T-test

We then run a t-test on the data. We specify the formula as y ~ x and add var.equal = FALSE


    Welch Two Sample t-test

data:  Sepal.Length by Species
t = -10.521, df = 86.538, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.1057074 -0.7542926
sample estimates:
    mean in group setosa mean in group versicolor 
                   5.006                    5.936

To interpret the t-test, we say that there is evidence for a statistically significant difference between the two groups: setosa and versicolor: t(86) = -10.521, p < 2.2e-16. The mean of setosa is significantly lower than that of versicolor.

2.5.1.2.5 Linear Model

Let us run a linear model on the same data


Call:
lm(formula = Sepal.Length ~ Species, data = irisSub)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.0360 -0.3135 -0.0060  0.2715  1.0640 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)        5.00600    0.06250   80.09   <2e-16 ***
Speciesversicolor  0.93000    0.08839   10.52   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.442 on 98 degrees of freedom
Multiple R-squared:  0.5304,    Adjusted R-squared:  0.5256 
F-statistic: 110.7 on 1 and 98 DF,  p-value: < 2.2e-16

Any comments? discuss with your neighbour.

The results of the linear model are exactly the same, albeit with a difference in the sign of the difference. This indicates that the \(\beta\) coefficient for versicolor is significantly higher than that of setosa by 0.93000: t(98) = 10.52, p < <2e-16.

The dataset iris contains three species. We will run an ANOVA and a linear model on the data.

2.5.2 Basic ANOVA

We can use the function aov to run an Analysis of Variance

             Df Sum Sq Mean Sq F value Pr(>F)    
Species       2  63.21  31.606   119.3 <2e-16 ***
Residuals   147  38.96   0.265                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

2.5.3 Linear model

We can use the function lm to run a linear model


Call:
lm(formula = Sepal.Length ~ Species, data = iris)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.6880 -0.3285 -0.0060  0.3120  1.3120 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)         5.0060     0.0728  68.762  < 2e-16 ***
Speciesversicolor   0.9300     0.1030   9.033 8.77e-16 ***
Speciesvirginica    1.5820     0.1030  15.366  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.5148 on 147 degrees of freedom
Multiple R-squared:  0.6187,    Adjusted R-squared:  0.6135 
F-statistic: 119.3 on 2 and 147 DF,  p-value: < 2.2e-16

But wait… How is the linear model comparable to the analysis of variance we ran above? This linear model derives the analysis of variance we saw above, use anova on your linear model..

Here are the results of the initial Analysis of variance:

             Df Sum Sq Mean Sq F value Pr(>F)    
Species       2  63.21  31.606   119.3 <2e-16 ***
Residuals   147  38.96   0.265                   
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

And here are the results of the linear model with the anova function

Analysis of Variance Table

Response: Sepal.Length
           Df Sum Sq Mean Sq F value    Pr(>F)    
Species     2 63.212  31.606  119.26 < 2.2e-16 ***
Residuals 147 38.956   0.265                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

They are exactly the same… The underlying of an Analysis of variance is a linear model. We will continue with a linear model to understand it better

2.6 Looking again at our Linear Model

The basic assumption of a Linear model is to create a regression analysis on the data. We have an outcome (or dependent variable) and a predictor (or an independent variable). The formula of a linear model is as follows outcome~predictor that can be read as “outcome as a function of the predictor”. We can add “1” to specify an intercept, but this is by default added to the model

2.6.1 Model estimation


Call:
lm(formula = Sepal.Length ~ Species, data = iris)

Coefficients:
      (Intercept)  Speciesversicolor   Speciesvirginica  
            5.006              0.930              1.582  

Call:
lm(formula = Sepal.Length ~ Species, data = iris)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.6880 -0.3285 -0.0060  0.3120  1.3120 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)         5.0060     0.0728  68.762  < 2e-16 ***
Speciesversicolor   0.9300     0.1030   9.033 8.77e-16 ***
Speciesvirginica    1.5820     0.1030  15.366  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.5148 on 147 degrees of freedom
Multiple R-squared:  0.6187,    Adjusted R-squared:  0.6135 
F-statistic: 119.3 on 2 and 147 DF,  p-value: < 2.2e-16

2.6.2 Tidying the output

To interpret the model, we need look at the coefficients. The “Intercept” (=Setosa) is 5.006 and the coefficients for Versicolor and for Virginica are respectively 5.936 and 6.588 This tells us that compared to Setosa, moving from this category to Versicolor leads to a significant increase by 5.936, and for Virginica, there is a significant increase by 6.588.

2.6.3 Obtaining our “true” coefficients

But where are our actual values based on the means in the table above? We run a model that suppresses the intercept (i.e., adding 0 instead of 1) and this will allow us to obtain the “true” coefficients for each level of our predictor


Call:
lm(formula = Sepal.Length ~ 0 + Species, data = iris)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.6880 -0.3285 -0.0060  0.3120  1.3120 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
Speciessetosa       5.0060     0.0728   68.76   <2e-16 ***
Speciesversicolor   5.9360     0.0728   81.54   <2e-16 ***
Speciesvirginica    6.5880     0.0728   90.49   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.5148 on 147 degrees of freedom
Multiple R-squared:  0.9925,    Adjusted R-squared:  0.9924 
F-statistic:  6522 on 3 and 147 DF,  p-value: < 2.2e-16

This matches the original data. Setosa has a mean of 5.006, Versicolor 10.942, and Virginica 10.942. See table above and coefficients below

[1] 5.006
[1] 5.936
[1] 6.588

The same as

[1] 5.006
[1] 5.936
[1] 6.588

2.6.4 Nice table of our model summary

We can also obtain a nice table of our model summary. We can use the package knitr or xtable

2.6.4.1 Directly from model summary

Estimate Std. Error t value Pr(>|t|)
(Intercept) 5.006 0.073 68.762 0
Speciesversicolor 0.930 0.103 9.033 0
Speciesvirginica 1.582 0.103 15.366 0

2.6.4.2 From the tidy output

term estimate std.error statistic p.value
(Intercept) 5.006 0.073 68.762 0
Speciesversicolor 0.930 0.103 9.033 0
Speciesvirginica 1.582 0.103 15.366 0

2.6.5 Dissecting the model

Let us dissect the model. If you use “str”, you will be able to see what is available under our linear model. To access some info from the model

2.6.5.1 “str” and “coef”

List of 13
 $ coefficients : Named num [1:3] 5.01 0.93 1.58
  ..- attr(*, "names")= chr [1:3] "(Intercept)" "Speciesversicolor" "Speciesvirginica"
 $ residuals    : Named num [1:150] 0.094 -0.106 -0.306 -0.406 -0.006 ...
  ..- attr(*, "names")= chr [1:150] "1" "2" "3" "4" ...
 $ effects      : Named num [1:150] -71.5659 0.8025 7.91 -0.3826 0.0174 ...
  ..- attr(*, "names")= chr [1:150] "(Intercept)" "Speciesversicolor" "Speciesvirginica" "" ...
 $ rank         : int 3
 $ fitted.values: Named num [1:150] 5.01 5.01 5.01 5.01 5.01 ...
  ..- attr(*, "names")= chr [1:150] "1" "2" "3" "4" ...
 $ assign       : int [1:3] 0 1 1
 $ qr           :List of 5
  ..$ qr   : num [1:150, 1:3] -12.2474 0.0816 0.0816 0.0816 0.0816 ...
  .. ..- attr(*, "dimnames")=List of 2
  .. .. ..$ : chr [1:150] "1" "2" "3" "4" ...
  .. .. ..$ : chr [1:3] "(Intercept)" "Speciesversicolor" "Speciesvirginica"
  .. ..- attr(*, "assign")= int [1:3] 0 1 1
  .. ..- attr(*, "contrasts")=List of 1
  .. .. ..$ Species: chr "contr.treatment"
  ..$ qraux: num [1:3] 1.08 1.05 1.09
  ..$ pivot: int [1:3] 1 2 3
  ..$ tol  : num 1e-07
  ..$ rank : int 3
  ..- attr(*, "class")= chr "qr"
 $ df.residual  : int 147
 $ contrasts    :List of 1
  ..$ Species: chr "contr.treatment"
 $ xlevels      :List of 1
  ..$ Species: chr [1:3] "setosa" "versicolor" "virginica"
 $ call         : language lm(formula = Sepal.Length ~ Species, data = iris)
 $ terms        :Classes 'terms', 'formula'  language Sepal.Length ~ Species
  .. ..- attr(*, "variables")= language list(Sepal.Length, Species)
  .. ..- attr(*, "factors")= int [1:2, 1] 0 1
  .. .. ..- attr(*, "dimnames")=List of 2
  .. .. .. ..$ : chr [1:2] "Sepal.Length" "Species"
  .. .. .. ..$ : chr "Species"
  .. ..- attr(*, "term.labels")= chr "Species"
  .. ..- attr(*, "order")= int 1
  .. ..- attr(*, "intercept")= int 1
  .. ..- attr(*, "response")= int 1
  .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> 
  .. ..- attr(*, "predvars")= language list(Sepal.Length, Species)
  .. ..- attr(*, "dataClasses")= Named chr [1:2] "numeric" "factor"
  .. .. ..- attr(*, "names")= chr [1:2] "Sepal.Length" "Species"
 $ model        :'data.frame':  150 obs. of  2 variables:
  ..$ Sepal.Length: num [1:150] 5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
  ..$ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...
  ..- attr(*, "terms")=Classes 'terms', 'formula'  language Sepal.Length ~ Species
  .. .. ..- attr(*, "variables")= language list(Sepal.Length, Species)
  .. .. ..- attr(*, "factors")= int [1:2, 1] 0 1
  .. .. .. ..- attr(*, "dimnames")=List of 2
  .. .. .. .. ..$ : chr [1:2] "Sepal.Length" "Species"
  .. .. .. .. ..$ : chr "Species"
  .. .. ..- attr(*, "term.labels")= chr "Species"
  .. .. ..- attr(*, "order")= int 1
  .. .. ..- attr(*, "intercept")= int 1
  .. .. ..- attr(*, "response")= int 1
  .. .. ..- attr(*, ".Environment")=<environment: R_GlobalEnv> 
  .. .. ..- attr(*, "predvars")= language list(Sepal.Length, Species)
  .. .. ..- attr(*, "dataClasses")= Named chr [1:2] "numeric" "factor"
  .. .. .. ..- attr(*, "names")= chr [1:2] "Sepal.Length" "Species"
 - attr(*, "class")= chr "lm"
      (Intercept) Speciesversicolor  Speciesvirginica 
            5.006             0.930             1.582

2.6.5.2 “coef” and “coefficients”

What if I want to obtain the “Intercept”? Or the coefficient for distance? What if I want the full row for distance?

(Intercept) 
      5.006 
Speciesversicolor 
             0.93 
    Estimate   Std. Error      t value     Pr(>|t|) 
9.300000e-01 1.029579e-01 9.032819e+00 8.770194e-16 
[1] 8.770194e-16

2.6.5.3 Up to you

Play around with the model summary and obtain the t values for the three levels. You can do this by referring to the coefficient as above

2.6.5.4 Residuals

What about residuals (difference between the observed value and the estimated value of the quantity) and fitted values?

2.6.5.5 Goodness of fit?

[1] 231.452
[1] 243.4945
'log Lik.' -111.726 (df=4)

Or use the following from broom

2.6.5.6 Significance testing

Are the above informative? of course not directly. If we want to test for overall significance of model. We run a null model (aka intercept only) and compare models.

Analysis of Variance Table

Model 1: Sepal.Length ~ 1
Model 2: Sepal.Length ~ Species
  Res.Df     RSS Df Sum of Sq      F    Pr(>F)    
1    149 102.168                                  
2    147  38.956  2    63.212 119.26 < 2.2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The results show that adding the factor “Species” improves the model fit. We can write this as follows: Model comparison showed that the addition of Species improved the model fit when compared with an intercept only model (\(F\)(2) = 119.26, p < 1.669669210^{-31})

2.6.5.7 Plotting fitted values

2.6.5.7.1 Trend line

Let’s plot our fitted values but only for the trend line

This allows us to plot the fitted values from our model with the predicted linear trend. This is exactly the same as our original data.

2.6.6 What about pairwise comparison?

Based on our model’s summary, can you tell me if there is a difference between Versicolor and Virginica?


Call:
lm(formula = Sepal.Length ~ Species, data = iris)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.6880 -0.3285 -0.0060  0.3120  1.3120 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)         5.0060     0.0728  68.762  < 2e-16 ***
Speciesversicolor   0.9300     0.1030   9.033 8.77e-16 ***
Speciesvirginica    1.5820     0.1030  15.366  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.5148 on 147 degrees of freedom
Multiple R-squared:  0.6187,    Adjusted R-squared:  0.6135 
F-statistic: 119.3 on 2 and 147 DF,  p-value: < 2.2e-16
$emmeans
 Species    emmean     SE  df lower.CL upper.CL
 setosa       5.01 0.0728 147     4.86     5.15
 versicolor   5.94 0.0728 147     5.79     6.08
 virginica    6.59 0.0728 147     6.44     6.73

Confidence level used: 0.95 

$contrasts
 contrast               estimate    SE  df t.ratio p.value
 setosa - versicolor      -0.930 0.103 147  -9.033 <.0001 
 setosa - virginica       -1.582 0.103 147 -15.366 <.0001 
 versicolor - virginica   -0.652 0.103 147  -6.333 <.0001 

P value adjustment: fdr method for 3 tests

How to interpret the output? Discuss with your neighbour and share with the group.

Hint… Look at the emmeans values for each level of our factor “Species” and the contrasts.

2.7 What about the other outcomes?

So far, we only looked at “Sepal.Length”. What about the other outcomes? how informative are they? Do we have statistical difference between the three levels of our predictor? You can do this in your spare time.

2.8 Conclusion

We have so far looked at the Linear Model. The underlying assumption about linear models is that we have a normal (Gaussian) distribution. This is the model to be used when we have a numeric outcome. What if our outcome is not numeric? What if we have two categories, i.e., black vs white? correct vs incorrect? yes vs no? These are categorical binary outcome. We look in the next section at Logistic Regression

3 From Linear to Logistic models

Here we will look at an example when the outcome is binary. This simulated data is structured as follows. We asked one participant to listen to 165 sentences, and to judge whether these are “grammatical” or “ungrammatical”. There were 105 sentences that were “grammatical” and 60 “ungrammatical”. This fictitious example can apply in any other situation. Let’s think Geography: 165 lands: 105 “flat” and 60 “non-flat”, etc. This applies to any case where you need to “categorise” the outcome into two groups.

3.1 Load and summaries

Let’s load in the data and do some basic summaries

'data.frame':   165 obs. of  2 variables:
 $ grammaticality: Factor w/ 2 levels "grammatical",..: 1 1 1 1 1 1 1 1 1 1 ...
 $ response      : Factor w/ 2 levels "no","yes": 2 2 2 2 2 2 2 2 2 2 ...
               
                yes  no
  grammatical   100   5
  ungrammatical  10  50

3.2 Accuracy and Signal Detection Theory

3.2.1 Rationale

We are generally interested in performance, i.e., whether the we have “accurately” categorised the outcome or not and at the same time want to evaluate our biases in responses. When deciding on categories, we are usually biased in our selection.

Let’s ask the question: How many of you have a Mac laptop and how many a Windows laptop? For those with a Mac, what was the main reason for choosing it? Are you biased in anyway by your decision?

To correct for these biases, we use some variants from Signal Detection Theory to obtain the true estimates without being influenced by the biases.

3.2.2 Running stats

Let’s do some stats on this

Yes No Total
Grammatical (Yes Actual) TP = 100 FN = 5 (Yes Actual) 105
Ungrammatical (No Actual) FP = 10 TN = 50 (No Actual) 60
Total (Yes Response) 110 (No Response) 55 165 
[1] 0.9090909
[1] 0.09090909
[1] 0.952381
[1] 0.1666667
[1] 0.8333333
[1] 0.9090909
$dprime
[1] 2.635813

$beta
[1] 0.3970026

$aprime
[1] 0.9419643

$bppd
[1] -0.5076923

$c
[1] -0.3504848

The most important from above, is d-prime. This is modelling the difference between the rate of “True Positive” responses and “False Positive” responses in standard unit (or z-scores). The formula can be written as:

d' (d prime) = Z(True Positive Rate) - Z(False Positive Rate)

3.3 GLM

Let’s run a first GLM (Generalised Linear Model). A GLM uses a special family “binomial” as it assumes the outcome has a binomial distribution. In general, results from a Logistic Regression are close to what we get from SDT (see above).

To run the results, we will change the reference level for both response and grammaticality. The basic assumption about GLM is that we start with our reference level being the “no” responses to the “ungrammatical” category. Any changes to this reference will be seen in the coefficients as “yes” responses to the “grammatical” category.

3.3.1 Model estimation and results

The results below show the logodds for our model.

[1] "yes" "no" 
[1] "grammatical"   "ungrammatical"

Call:
glm(formula = response ~ grammaticality, family = binomial, data = grammatical)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.4676  -0.6039   0.3124   0.3124   1.8930  

Coefficients:
                          Estimate Std. Error z value Pr(>|z|)    
(Intercept)                -1.6094     0.3464  -4.646 3.38e-06 ***
grammaticalitygrammatical   4.6052     0.5744   8.017 1.08e-15 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 210.050  on 164  degrees of freedom
Residual deviance:  94.271  on 163  degrees of freedom
AIC: 98.271

Number of Fisher Scoring iterations: 5

The results show that for one unit increase in the response (i.e., from no to yes), the logodds of being “grammatical” is increased by 4.6051702 (the intercept shows that when the response is “no”, the logodds are -1.6094379). The actual logodds for the response “yes” to grammatical is 2.9957323

3.3.2 Getting “true” coefficients from GLM

How to get the true coefficients? How to suppress the Intercept?

Here we will run the same GLM model above but by suppressing the “Intercept”. The idea here is to get the “true” coefficient for a “grammatical” and a “yes” response. We can use the above code, i.e., “mycoef2[1]+mycoef2[2]” or as below


Call:
glm(formula = response ~ 0 + grammaticality, family = binomial, 
    data = grammatical)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.4676  -0.6039   0.3124   0.3124   1.8930  

Coefficients:
                            Estimate Std. Error z value Pr(>|z|)
grammaticalityungrammatical  -1.6094     0.3464  -4.646 3.38e-06
grammaticalitygrammatical     2.9957     0.4582   6.538 6.25e-11
                               
grammaticalityungrammatical ***
grammaticalitygrammatical   ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 228.739  on 165  degrees of freedom
Residual deviance:  94.271  on 163  degrees of freedom
AIC: 98.271

Number of Fisher Scoring iterations: 5

3.3.3 Logodds to Odd ratios

Logodds can be modified to talk about the odds of an event. For our model above, the odds of “grammatical” receiving a “yes” response increase by 100; whereas receiving a “no” is a mere 0.2. Using the second model (i.e., without an Intercept) allows us to get the odd ratios for each of responses to “Ungrammatical” and “Grammatical”, which is still 0.2 for “ungrammatical” and 20 for “grammatical”, i.e., 20 times more

              (Intercept) grammaticalitygrammatical 
                      0.2                     100.0 
grammaticalityungrammatical   grammaticalitygrammatical 
                        0.2                        20.0

3.3.4 LogOdds to proportions

If you want to talk about the percentage “accuracy” of our model, then we can transform our loggodds into proportions. This shows that the proportion of “grammatical” receiving a “yes” response increases by 99% (or 95% based on our “true” coefficients)

              (Intercept) grammaticalitygrammatical 
                0.1666667                 0.9900990 
grammaticalityungrammatical   grammaticalitygrammatical 
                  0.1666667                   0.9523810

3.3.5 GLM as a classification tool

The code below demonstrates the links between our GLM model and what we had obtained above from SDT. The predictions’ table shows that our GLM was successful at obtaining prediction that are identical to our initial data setup. Look at the table here and the table above. Once we have created our table of outcome, we can compute percent correct, the specificity, the sensitivity, the Kappa score, etc.. this yields the actual value with the SD that is related to variations in responses.

     
      grammatical ungrammatical
  yes         100            10
  no            5            50

If you look at the results from SDT above, these results are the same as the following

Accuracy: (TP+TN)/Total (0.9090909)

True Positive Rate (or Specificity) TP/(TP+FN) (0.952381)

True Negative Rate (or Sensitivity) TN/(FP+TN) (0.8333333)

3.3.7 GLM and d prime

The values obtained here match those obtained from SDT. For d prime, the difference stems from the use of the logit variant of the Binomial family. By using a probit variant, one obtains the same values (see here for more details). A probit variant models the z-score differences in the outcome and is evaluated in change in 1-standard unit. This is modelling the change from “ungrammatical” “no” responses into “grammatical” “yes” responses in z-scores. The same conceptual underpinnings of d-prime from Signal Detection Theory.

[1] 2.635813
grammaticalityungrammatical 
                   2.635813

3.4 GLM: Other distributions

If your data does not fit a binomial distribution, and is a multinomial (i.e., three or more response categories) or poisson (count data), then you need to use the glm function with a specific family function.

LS0tDQp0aXRsZTogIlNlc3Npb25fNC1BbmFseXNpbmdEYXRhIg0Kb3V0cHV0OiANCiAgaHRtbF9ub3RlYm9vazoNCiAgICBoaWdobGlnaHQ6IHB5Z21lbnRzDQogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMNCiAgICB0b2M6IHllcw0KICAgIHRvY19kZXB0aDogNg0KICAgIHRvY19mbG9hdDoNCiAgICAgIGNvbGxhcHNlZDogeWVzDQogIA0KLS0tDQoNCkluIHRoaXMgc2Vzc2lvbiwgd2Ugd2lsbCBsb29rIGF0IGJhc2ljIGZ1bmN0aW9ucyBpbiBSIHRoYXQgd2lsbCBoZWxwIHVzIGluIHJ1bm5pbmcgc29tZSBpbmZlcmVudGlhbCBzdGF0aXN0aWNzLiBUaGVzZSB3aWxsIGhlbHAgdXMgdG8gZXZhbHVhdGUgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIG9uZSAob3IgbW9yZSkgcHJlZGljdG9yKHMpIGFuZCBhbiBvdXRjb21lLiBXZSB3aWxsIHN0YXJ0IHdpdGggYSBiYXNpYyBleGFtcGxlLiBVc2luZyB0aGUgZGF0YWZyYW1lICJjYXJzIi4gDQoNCg0KIyBMb2FkaW5nIHBhY2thZ2VzIA0KYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgZXJyb3I9RkFMU0V9DQojIyBVc2UgdGhlIGNvZGUgYmVsb3cgdG8gY2hlY2sgaWYgeW91IGhhdmUgYWxsIHJlcXVpcmVkIHBhY2thZ2VzIGluc3RhbGxlZC4gSWYgc29tZSBhcmUgbm90IGluc3RhbGxlZCBhbHJlYWR5LCB0aGUgY29kZSBiZWxvdyB3aWxsIGluc3RhbGwgdGhlc2UuIElmIHlvdSBoYXZlIGFsbCBwYWNrYWdlcyBpbnN0YWxsZWQsIHRoZW4geW91IGNvdWxkIGxvYWQgdGhlbSB3aXRoIHRoZSBzZWNvbmQgY29kZS4NCnJlcXVpcmVkUGFja2FnZXMgPSBjKCd0aWR5dmVyc2UnLCAncHN5Y2hvJywgJ29yZGluYWwnLCAnUHJlc2VuY2VBYnNlbmNlJywgJ2Jyb29tJywgJ2VtbWVhbnMnLCAnY2FyJywna25pdHInKQ0KZm9yKHAgaW4gcmVxdWlyZWRQYWNrYWdlcyl7DQogIGlmKCFyZXF1aXJlKHAsY2hhcmFjdGVyLm9ubHkgPSBUUlVFKSkgaW5zdGFsbC5wYWNrYWdlcyhwKQ0KICBsaWJyYXJ5KHAsY2hhcmFjdGVyLm9ubHkgPSBUUlVFKQ0KfQ0KYGBgDQoNCiMgQmFzaWMgc3RhdGlzdGljcyANCg0KIyMgQnVpbHQtaW4gZGF0YXNldHMNCg0KV2Ugd2lsbCB1c2Ugb25lIG9mIHRoZSBidWlsdCBpbiBgUmAuIFlvdSBjYW4gY2hlY2sgYWxsIGF2YWlsYWJsZSBkYXRhc2V0cyBpbiBgUmAgdXNpbmcgdGhlIGZvbGxvd2luZzoNCg0KYGBge3J9DQpkYXRhKCkNCiMgb3IgYmVsb3cgZm9yIGFsbCBkYXRhc2V0cyBhdmFpbGFibGUgaW4gYWxsIGluc3RhbGxlZCBwYWNrYWdlcw0KZGF0YShwYWNrYWdlID0gLnBhY2thZ2VzKGFsbC5hdmFpbGFibGUgPSBUUlVFKSkNCmBgYA0KDQpXZSB3aWxsIHVzZSB0aGUgYGlyaXNgIGRhdGFzZXQgZnJvbSB0aGUgcGFja2FnZSBgTUFTU2ANCg0KIyMgQ2hlY2tpbmcgc3RydWN0dXJlIGFuZCBzdW1tYXJpZXMNCg0KIyMjIFN0cnVjdHVyZQ0KDQpgYGB7cn0NCnN0cihpcmlzKQ0KYGBgDQoNCldlIGhhdmUgYSBkYXRhZnJhbWUgd2l0aCAxNTAgb2JzZXJ2YXRpb25zIGFuZCA1IHZhcmlhYmxlczsgNCBudW1lcmljIGFuZCAxIGZhY3RvciB3aXRoIDMgbGV2ZWxzLiANCg0KIyMjIFN1bW1hcnkNCg0KV2Ugc3VtbWFyaXNlIHRoZSBkYXRhIHRvIHNlZSB0aGUgdHJlbmRzOg0KDQpgYGB7cn0NCnN1bW1hcnkoaXJpcykNCmBgYA0KDQpTbyB3ZSBoYXZlIGFuIGVxdWFsIGRhdGFmcmFtZSAoNTAgb2JzZXJ2YXRpb25zIHVuZGVyIGVhY2ggbGV2ZWwgb2YgdGhlIGZhY3RvciBgU3BlY2llc2ApLCB3aXRoIG5vIG1pc3NpbmcgdmFsdWVzIChha2EgYE5BYCkuDQoNCg0KIyMjIEFkdmFuY2VkDQoNCldoYXQgaWYgeW91IHdhbnQgdG8gc3VtbWFyaXNlIHRoZSBkYXRhIGFuZCBnZXQgdGhlIG1lYW4sIFNELCBieSBlYWNoIGxldmVsIG9mIHRoZSBmYWN0b3I/IA0KDQpgYGB7cn0NCmlyaXMgJT4lIA0KICBncm91cF9ieShTcGVjaWVzKSAlPiUgDQogIHN1bW1hcmlzZSgNCiAgU0wuTWVhbiA9IG1lYW4oU2VwYWwuTGVuZ3RoKSwNCiAgU0wuU0QgPSBzZChTZXBhbC5MZW5ndGgpDQogICkNCmBgYA0KDQojIyMgVXAgdG8geW91DQoNCkRvIHNvbWUgYWRkaXRpb25hbCBzdW1tYXJpZXMNCg0KYGBge3J9DQoNCmBgYA0KDQoNCiMjIFBsb3QNCg0KV2UgY2FuIG1ha2UgYSBib3hwbG90IG9mIHRoZSBkYXRhIGFuZCBhZGQgYSB0cmVuZCBsaW5lLiBUaGlzIGFsbG93cyB1cyB0byB2aXN1YWxpc2UgdGhlIG1lZGlhbiwgYW5kIHF1YW50aWxlcyBpbiBhZGRpdGlvbiB0byB0aGUgc3RhbmRhcmQgZGV2aWF0aW9uIGFuZCBhbnkgb3V0bGllcnMuLi4gQWxsIGluIHRoZSBzYW1lIHBsb3QhDQoNCmBgYHtyLHdhcm5pbmc9RkFMU0UsbWVzc2FnZT1GQUxTRX0NCmlyaXMgJT4lIA0KICBnZ3Bsb3QoYWVzKHggPSBTcGVjaWVzLCB5ID0gU2VwYWwuTGVuZ3RoKSkrDQogIGdlb21fYm94cGxvdCgpICsNCiAgbGFicyh4ID0gIlNwZWNpZXMiLCB5ID0gIkxlbmd0aCIsIHRpdGxlID0gIkJveHBsb3QgYW5kIHRyZW5kIGxpbmUiLCBzdWJ0aXRsZT0id2l0aCBnZ3Bsb3QyIikgKyANCiAgdGhlbWVfYncoKSArIHRoZW1lKHRleHQgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDE1KSkrDQogIGdlb21fc21vb3RoKGFlcyh4ID0gYXMubnVtZXJpYyhTcGVjaWVzKSwgeSA9IFNlcGFsLkxlbmd0aCksIG1ldGhvZCA9ICJsbSIpDQpgYGANCg0KSGVyZSBJIGhhdmUgdXNlZCB0aGUgdmFyaWFibGUgYFNlcGFsLkxlbmd0aGAuIFlvdSBjYW4gdXNlIGFueSBvZiB0aGUgYWRkaXRpb25hbCB2YXJpYWJsZXMgdG8gcGxvdCB0aGUgZGF0YS4NCg0KIyMgQ29ycmVsYXRpb24gdGVzdHMNCg0KIyMjIEJhc2ljIGNvcnJlbGF0aW9ucw0KDQpXZSB1c2UgdGhlIGZ1bmN0aW9uIGBjb3JgIHRvIG9idGFpbiB0aGUgcGVhcnNvbiBjb3JyZWxhdGlvbiBhbmQgYGNvci50ZXN0YCB0byBydW4gYSBiYXNpYyBjb3JyZWxhdGlvbiB0ZXN0IG9uIG91ciBkYXRhIHdpdGggc2lnbmlmaWNhbmNlIHRlc3RpbmcNCg0KYGBge3J9DQpjb3IoaXJpcyRTZXBhbC5MZW5ndGgsIGlyaXMkUGV0YWwuTGVuZ3RoLCBtZXRob2QgPSAicGVhcnNvbiIpDQpjb3IudGVzdChpcmlzJFNlcGFsLkxlbmd0aCwgaXJpcyRQZXRhbC5MZW5ndGgpDQpgYGANCg0KIyMjIyBVcCB0byB5b3UNCg0KQ2FuIHlvdSBjaGVjayB3aGV0aGVyIHRoZXJlIGlzIGEgY29ycmVsYXRpb24gYmV0d2VlbiB0aGUgYFNlcGFsLkxlbmd0aGAgYW5kIHRoZSBgUGV0YWwuV2lkdGhgPyBXaGF0IGFib3V0IGBQZXRhbC5MZW5ndGhgIGFuZCBgUGV0YWwuV2lkdGhgPyANCg0KDQpgYGB7cn0NCg0KYGBgDQoNCg0KDQojIyMgVXNpbmcgdGhlIHBhY2thZ2UgYHBzeWNob2ANCg0KQWJvdmUsIHdlIGRpZCBhIGNvcnJlbGF0aW9uIHRlc3Qgb24gdHdvIHByZWRpY3RvcnMuIFdlIHJ1biBtdWx0aXBsZSBjb3JyZWxhdGlvbnMgb24gbXVsdGlwbGUgcHJlZGljdG9ycyB3aXRoIGNvcnJlY3Rpb25zLiBZb3UgY2FuIGdldCBhIHRhYmxlLCBhbmQgYSBwbG90LiAoc2VlIGhlcmUgaHR0cHM6Ly93d3cuci1ibG9nZ2Vycy5jb20vYmVhdXRpZnVsLWFuZC1wb3dlcmZ1bC1jb3JyZWxhdGlvbi10YWJsZXMtaW4tci9hbXAvKQ0KDQojIyMjIERlZmF1bHRzDQoNCmBgYHtyfQ0KIyMgY29ycmVsYXRpb24gdXNpbmcgInBzeWNobyINCiMjIHVzZSAiQWx0IiArICItIiB0byBhZGQgIjwtIg0KIyMgdXNlICJDdHJsIiArICJTaGlmdCIgKyAiTSIgdG8gb2J0YWluICIlPiUiIA0KY29yIDwtIGlyaXMgJT4lIA0KICBjb3JyZWxhdGlvbigpDQpzdW1tYXJ5KGNvcikNCnBsb3QoY29yKQ0KcHJpbnQoY29yKQ0KYGBgDQoNCiMjIyMgQ2hhbmdpbmcgcGFyYW1ldGVycw0KDQpZb3UgY2FuIGNoYW5nZSB0aGUgZm9sbG93aW5nIGlmIHlvdSBhcmUgaW50ZXJlc3RlZCBpbiBnZXR0aW5nIGEgYHNwZWFybWFuYCBjb3JyZWxhdGlvbiBjb21wYXJlZCB0byBhIGBwZWFyc29uYCBjb3JyZWxhdGlvbjogDQpgbWV0aG9kID0gInNwZWFybWFuImAgKG1ldGhvZCA9ICJwZWFyc29uIiwgInNwZWFybWFuIiwgImtlbmRhbGwiKS4NCg0KVGhlIGRlZmF1bHRzIGhhdmUgY29ycmVjdGlvbnMgZm9yIG11bHRpcGxlIGNvbXBhcmlzb25zIHVzaW5nIHRoZSBgaG9sbWAgbWV0aG9kOyB5b3UgY2FuIHVzZSBgYWRqdXN0PSJib25mZXJyb25pImAgdG8gdXNlIGEgYm9uZmVycm9uaSBjb3JyZWN0aW9uIChhZGp1c3Q9ICJob2xtIiAoZGVmYXVsdCksICJib25mZXJyb25pIiwgImZkciIsICJub25lIikNCg0KIyMjIFVwIHRvIHlvdQ0KDQpSdW4gbXVsdGlwbGUgY29ycmVsYXRpb25zIGJlbG93IGJ5IGNoYW5naW5nIHRoZSBtZXRob2QsIG9yIHRoZSBhZGp1c3RtZW50cy4gQW55IGNvbW1lbnRzPyANCg0KYGBge3J9DQoNCmBgYA0KDQoNClVwIHRvIG5vdywgd2UgaGF2ZSBkb25lIHNvbWUgYmFzaWMgc3VtbWFyaWVzIGFuZCBjaGVja2VkIHRoZSBjb3JyZWxhdGlvbnMgaW4gdGhlIGRhdGEuIFRoZSBwZWFyc29uIGNvcnJlbGF0aW9ucyB3ZSBoYXZlIGRvbmUgcHJvdmlkZWQgdXMgd2l0aCBzaWduaWZpY2FuY2UgbGV2ZWxzIHJlbGF0ZWQgdG8gdGhlIGNvcnJlbGF0aW9ucyBiZXR3ZWVuIHR3byAqbnVtZXJpYyogb3V0Y29tZXMgV2UgZG8gbm90IGtub3cgaWYgdGhlIGxldmVscyBvZiBvdXIgdmFyaWFibGUgYFNwZWNpZXNgIGFyZSBzdGF0aXN0aWNhbGx5IGRpZmZlcmVudCBiYXNlZCBvbiBvbmUgb2YgdGhlIHByZWRpY3RvcnMuIFdlIHVzZSBpbmZlcmVudGlhbCBzdGF0aXN0aWNzIGhlcmUuDQoNCiMjIEZpcnN0IHN0ZXAgaW50byBzdGF0aXN0aWNzDQoNCiMjIyBCYXNpYyB0LXRlc3RzDQoNClQudGVzdCBhcmUgdXNlZCB3aGVuIHRoZSBvdXRjb21lIChEVikgaXMgbnVtZXJpYyBhbmQgdGhlIHByZWRpY3RvciBpcyBlaXRoZXIgbnVtZXJpYyBvciBjYXRlZ29yaWNhbCAod2l0aCB0d28gbGV2ZWxzIG9ubHkpLiANCg0KIyMjIyBOdW1lcmljIHByZWRpY3Rvcg0KDQpXZSBjYW4gcnVuIGEgYmFzaWMgdC10ZXN0IG9uIG91ciBkYXRhLCB1c2luZyB0aGUgZnVuY3Rpb24gYHQtdGVzdGAuIFRoaXMgbG9va3MgYXQgdGhlIHR3byBudW1lcmljIG91dGNvbWVzIGFuZCBzZWVzIGlmIHRoZXkgYXJlIGRpZmZlcmVudCBmcm9tIGVhY2ggb3RoZXINCg0KYGBge3J9DQp0LnRlc3QoaXJpcyRTZXBhbC5MZW5ndGgsIGlyaXMkUGV0YWwuTGVuZ3RoKQ0KYGBgDQoNCiMjIyMgQ2F0ZWdvcmljYWwgcHJlZGljdG9yDQoNCkluIHRoZSBgaXJpc2AgZGF0YXNldCwgd2UgaGF2ZSBhIGNhdGVnb3JpY2FsIHByZWRpY3RvcjogYFNwZWNpZXNgIHdoaWNoIGhhcyB0aHJlZSBsZXZlbHMNCg0KYGBge3J9DQpsZXZlbHMoaXJpcyRTcGVjaWVzKQ0KYGBgDQoNCkxldCdzIHN1YnNldCB0aGUgZGF0YSBhbmQgZXZhbHVhdGUgdGhlIGRpZmZlcmVuY2VzIGJldHdlZW4gYHNldG9zYWAgYW5kIGB2ZXJzaWNvbG9yYCB1c2luZyBhIHQtdGVzdCBhbmQgYSBsaW5lYXIgbW9kZWwuIFdlIHdpbGwgYWxzbyBjaGVjayB0aGUgbm9ybWFsaXR5IGFuZCBob21vZ2VuZWl0eSBvZiB2YXJpYW5jZSBpbiB0aGUgZGF0YQ0KDQojIyMjIyBTdWJzZXQNCg0KYGBge3J9DQppcmlzU3ViIDwtIGlyaXMgJT4lIA0KICBmaWx0ZXIoU3BlY2llcyA9PSAic2V0b3NhIiB8DQogICAgICAgICAgIFNwZWNpZXMgPT0gInZlcnNpY29sb3IiKQ0KbGV2ZWxzKGlyaXNTdWIkU3BlY2llcykNCmBgYA0KDQpCdXQgd2FpdCBhIG1pbnV0ZS4uLiBXZSBqdXN0IHN1YnNldHRlZCB0aGUgZGF0YSBhbmQgb3VyIGBsZXZlbHNgIHNob3cgdGhyZWUgbGV2ZWxzIGZhY3Rvci4uIHdoeT8gTGV0J3MgcnVuIGEgc3VtbWFyeQ0KDQpgYGB7cn0NCnN1bW1hcnkoaXJpc1N1YiRTcGVjaWVzKQ0KYGBgDQoNCkFoYSEgU28gdGhlIHN1YnNldHRpbmcgd29ya2VkIGFzIHRoZSBgdmlyZ2luaWNhYCBjYXNlcyBhcmUgMC4gV2UgbmVlZCB0byB0ZWxsIGBSYCB0byBjaGFuZ2UgdGhlIGZhY3RvciBsZXZlbHMNCg0KYGBge3J9DQppcmlzU3ViJFNwZWNpZXMgPC0gZmFjdG9yKGlyaXNTdWIkU3BlY2llcykNCmxldmVscyhpcmlzU3ViJFNwZWNpZXMpDQpgYGANCg0KTm93IHRoaXMgd29ya3MgYmV0dGVyDQoNCiMjIyMjIE5vcm1hbGl0eSBvZiBkaXN0cmlidXRpb24NCg0KIyMjIyMjIFNoYXBpcm8gdGVzdA0KDQpUbyBjaGVjayBub3JtYWxpdHkgb2YgZGlzdHJpYnV0aW9uLCB3ZSB1c2UgdGhlIGBzaGFwaXJvLnRlc3RgIG9uIHRoZSBudW1lcmljIG91dGNvbWUuIEdpdmVuIHRoYXQgb3VyIHByZWRpY3RvciBub3cgaGFzIHR3byBsZXZlbHMuIFdlIG5lZWQgdG8gc3Vic2V0IHRoZSBkYXRhIGFnYWluIHRvIGNoZWNrIG5vcm1hbGl0eSBvZiB0aGUgb3V0Y29tZSBgU2VwYWwuTGVuZ3RoYCBmb3IgZWFjaCBsZXZlbCBvZiBvdXIgZmFjdG9yIGBTcGVjaWVzYA0KDQpgYGB7cn0NCmlyaXNTdWJTZXQgPC0gaXJpcyAlPiUgDQogIGZpbHRlcihTcGVjaWVzID09ICJzZXRvc2EiKQ0KaXJpc1N1YlZlcnMgPC0gaXJpcyAlPiUgDQogIGZpbHRlcihTcGVjaWVzID09ICJ2ZXJzaWNvbG9yIikNCiAgDQpzaGFwaXJvLnRlc3QoaXJpc1N1YlNldCRTZXBhbC5MZW5ndGgpDQpzaGFwaXJvLnRlc3QoaXJpc1N1YlZlcnMkU2VwYWwuTGVuZ3RoKQ0KYGBgDQoNCkhvdyB0byBpbnRlcnByZXQgdGhpcyBub24tc3RhdGlzdGljYWxseSBzaWduaWZpY2FudCByZXN1bHQ/IFRoaXMgdGVsbHMgdXMgdGhhdCB0aGUgZGlzdHJpYnV0aW9uIG9mIHRoZSBkYXRhIGlzIG5vdCBzdGF0aXN0aWNhbGx5IGRpZmZlcmVudCBmcm9tIGEgbm9ybWFsIGRpc3RyaWJ1dGlvbi4NCg0KIyMjIyMjIERlbnNpdHkgcGxvdA0KDQpXZSBjYW4gYWxzbyB1c2UgYSBkZW5zaXR5IHBsb3QgdG8gZXZhbHVhdGUgbm9ybWFsaXR5LiBUaGUgcmVzdWx0cyBzaG93IHRoYXQgYm90aCBsZXZlbHMgaGF2ZSBiZWxsIHNoYXBlZCBkaXN0cmlidXRpb25zLg0KDQpgYGB7cn0NCmlyaXNTdWIgJT4lIA0KICBnZ3Bsb3QoYWVzKHggPSBTZXBhbC5MZW5ndGgpKSsNCiAgZ2VvbV9kZW5zaXR5KCkrDQogIGZhY2V0X3dyYXAoflNwZWNpZXMsIHNjYWxlcyA9ICJmcmVlX3giKQ0KYGBgDQoNCg0KIyMjIyMgSG9tb2dlbmVpdHkgb2YgdmFyaWFuY2UNCg0KQmVjYXVzZSBvdXIgZGF0YSBpcyBub3JtYWxseSBkaXN0cmlidXRlZCwgd2UgY2FuIHVzZSB0aGUgYGJhcmxldHRgIHRlc3QuIElmIG91ciBkYXRhIHdlcmUgbm9uLW5vcm1hbGx5IGRpc3RyaWJ1dGVkLCB3ZSB3b3VsZCB1c2UgdGhlIGxldmVuZVRlc3QuIFdlIGNhbiBjaGVjayBib3RoLg0KDQojIyMjIyMgQmFybGV0dCB0ZXN0DQoNCmBgYHtyfQ0KYmFydGxldHQudGVzdChTZXBhbC5MZW5ndGggfiBTcGVjaWVzLCBkYXRhID0gaXJpc1N1YikNCmBgYA0KDQojIyMjIyMgTGV2ZW5lIHRlc3QNCg0KYGBge3J9DQp2YXIudGVzdChTZXBhbC5MZW5ndGggfiBTcGVjaWVzLCBkYXRhID0gaXJpc1N1YikNCmNhcjo6bGV2ZW5lVGVzdChTZXBhbC5MZW5ndGggfiBTcGVjaWVzLCBkYXRhID0gaXJpc1N1YikNCmBgYA0KDQpJbiBhbGwgY2FzZXMsIHRoZSBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50IHJlc3VsdCBpbmRpY2F0ZXMgdGhhdCB0aGVyZSBpcyBldmlkZW5jZSB0aGF0IHRoZSB2YXJpYW5jZSBvZiB0d28gbGV2ZWxzIG9mIHRoZSBmYWN0b3IgYFNwZWNpZXNgIGlzIHN0YXRpc3RpY2FsbHkgc2lnbmlmaWNhbnQ7IGkuZS4sIHRoZSB2YXJpYW5jZXMgYXJlIG5vdCBlcXVhbA0KDQojIyMjIyBULXRlc3QNCg0KV2UgdGhlbiBydW4gYSB0LXRlc3Qgb24gdGhlIGRhdGEuIFdlIHNwZWNpZnkgdGhlIGZvcm11bGEgYXMgYHkgfiB4YCBhbmQgYWRkIGB2YXIuZXF1YWwgPSBGQUxTRWANCg0KYGBge3J9DQp0LnRlc3QoU2VwYWwuTGVuZ3RoIH4gU3BlY2llcywgZGF0YSA9IGlyaXNTdWIsIHZhci5lcXVhbCA9IEZBTFNFKQ0KYGBgDQoNClRvIGludGVycHJldCB0aGUgdC10ZXN0LCB3ZSBzYXkgdGhhdCB0aGVyZSBpcyBldmlkZW5jZSBmb3IgYSBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50IGRpZmZlcmVuY2UgYmV0d2VlbiB0aGUgdHdvIGdyb3VwczogYHNldG9zYWAgYW5kIGB2ZXJzaWNvbG9yYDogYHQoODYpID0gLTEwLjUyMSwgcCA8IDIuMmUtMTZgLiBUaGUgbWVhbiBvZiBgc2V0b3NhYCBpcyBzaWduaWZpY2FudGx5IGxvd2VyIHRoYW4gdGhhdCBvZiBgdmVyc2ljb2xvcmAuDQoNCiMjIyMjIExpbmVhciBNb2RlbA0KDQpMZXQgdXMgcnVuIGEgbGluZWFyIG1vZGVsIG9uIHRoZSBzYW1lIGRhdGENCg0KDQpgYGB7cn0NCnN1bW1hcnkobG0oU2VwYWwuTGVuZ3RoIH4gU3BlY2llcywgZGF0YSA9IGlyaXNTdWIpKQ0KYGBgDQoNCkFueSBjb21tZW50cz8gZGlzY3VzcyB3aXRoIHlvdXIgbmVpZ2hib3VyLg0KDQpUaGUgcmVzdWx0cyBvZiB0aGUgbGluZWFyIG1vZGVsIGFyZSBleGFjdGx5IHRoZSBzYW1lLCBhbGJlaXQgd2l0aCBhIGRpZmZlcmVuY2UgaW4gdGhlIHNpZ24gb2YgdGhlIGRpZmZlcmVuY2UuIFRoaXMgaW5kaWNhdGVzIHRoYXQgdGhlICRcYmV0YSQgY29lZmZpY2llbnQgZm9yIGB2ZXJzaWNvbG9yYCBpcyBzaWduaWZpY2FudGx5IGhpZ2hlciB0aGFuIHRoYXQgb2YgYHNldG9zYWAgYnkgYDAuOTMwMDBgOiBgdCg5OCkgPSAxMC41MiwgcCA8IDwyZS0xNmAuDQoNCg0KVGhlIGRhdGFzZXQgYGlyaXNgIGNvbnRhaW5zIHRocmVlIHNwZWNpZXMuIFdlIHdpbGwgcnVuIGFuIEFOT1ZBIGFuZCBhIGxpbmVhciBtb2RlbCBvbiB0aGUgZGF0YS4NCg0KIyMjIEJhc2ljIEFOT1ZBDQoNCldlIGNhbiB1c2UgdGhlIGZ1bmN0aW9uIGBhb3ZgIHRvIHJ1biBhbiBBbmFseXNpcyBvZiBWYXJpYW5jZQ0KDQpgYGB7cn0NCm1kbC5hb3YgPC0gYW92KFNlcGFsLkxlbmd0aCB+IFNwZWNpZXMsIGRhdGEgPSBpcmlzKQ0Kc3VtbWFyeShtZGwuYW92KQ0KYGBgDQoNCiMjIyBMaW5lYXIgbW9kZWwNCg0KV2UgY2FuIHVzZSB0aGUgZnVuY3Rpb24gYGxtYCB0byBydW4gYSBsaW5lYXIgbW9kZWwNCg0KYGBge3J9DQptZGwubG0gPC0gbG0oU2VwYWwuTGVuZ3RoIH4gU3BlY2llcywgZGF0YSA9IGlyaXMpDQpzdW1tYXJ5KG1kbC5sbSkNCmBgYA0KDQpCdXQgd2FpdC4uLiBIb3cgaXMgdGhlIGxpbmVhciBtb2RlbCBjb21wYXJhYmxlIHRvIHRoZSBhbmFseXNpcyBvZiB2YXJpYW5jZSB3ZSByYW4gYWJvdmU/IFRoaXMgbGluZWFyIG1vZGVsIGRlcml2ZXMgdGhlIGFuYWx5c2lzIG9mIHZhcmlhbmNlIHdlIHNhdyBhYm92ZSwgdXNlIGBhbm92YWAgb24geW91ciBsaW5lYXIgbW9kZWwuLg0KDQpIZXJlIGFyZSB0aGUgcmVzdWx0cyBvZiB0aGUgaW5pdGlhbCBBbmFseXNpcyBvZiB2YXJpYW5jZToNCg0KYGBge3J9DQpzdW1tYXJ5KG1kbC5hb3YpDQpgYGANCg0KQW5kIGhlcmUgYXJlIHRoZSByZXN1bHRzIG9mIHRoZSBsaW5lYXIgbW9kZWwgd2l0aCB0aGUgYGFub3ZhYCBmdW5jdGlvbg0KDQpgYGB7cn0NCmFub3ZhKG1kbC5sbSkNCmBgYA0KDQpUaGV5IGFyZSBleGFjdGx5IHRoZSBzYW1lLi4uIFRoZSB1bmRlcmx5aW5nIG9mIGFuIEFuYWx5c2lzIG9mIHZhcmlhbmNlIGlzIGEgbGluZWFyIG1vZGVsLiBXZSB3aWxsIGNvbnRpbnVlIHdpdGggYSBsaW5lYXIgbW9kZWwgdG8gdW5kZXJzdGFuZCBpdCBiZXR0ZXINCg0KIyMgTG9va2luZyBhZ2FpbiBhdCBvdXIgTGluZWFyIE1vZGVsDQoNClRoZSBiYXNpYyBhc3N1bXB0aW9uIG9mIGEgTGluZWFyIG1vZGVsIGlzIHRvIGNyZWF0ZSBhIHJlZ3Jlc3Npb24gYW5hbHlzaXMgb24gdGhlIGRhdGEuIFdlIGhhdmUgYW4gb3V0Y29tZSAob3IgZGVwZW5kZW50IHZhcmlhYmxlKSBhbmQgYSBwcmVkaWN0b3IgKG9yIGFuIGluZGVwZW5kZW50IHZhcmlhYmxlKS4gVGhlIGZvcm11bGEgb2YgYSBsaW5lYXIgbW9kZWwgaXMgYXMgZm9sbG93cyBgb3V0Y29tZX5wcmVkaWN0b3JgIHRoYXQgY2FuIGJlIHJlYWQgYXMgIm91dGNvbWUgYXMgYSBmdW5jdGlvbiBvZiB0aGUgcHJlZGljdG9yIi4gV2UgY2FuIGFkZCAiMSIgdG8gc3BlY2lmeSBhbiBpbnRlcmNlcHQsIGJ1dCB0aGlzIGlzIGJ5IGRlZmF1bHQgYWRkZWQgdG8gdGhlIG1vZGVsDQoNCiMjIyBNb2RlbCBlc3RpbWF0aW9uDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KbWRsLmxtIDwtIGxtKFNlcGFsLkxlbmd0aCB+IFNwZWNpZXMsIGRhdGEgPSBpcmlzKQ0KIyBzYW1lIGFzIGJlbG93Lg0KI21kbC5sbSA8LSBsbShTZXBhbC5MZW5ndGh+MStTcGVjaWVzLCBkYXRhPWlyaXMpDQptZGwubG0jYWxzbyBwcmludChtZGwubG0pDQpzdW1tYXJ5KG1kbC5sbSkNCmBgYA0KDQojIyMgVGlkeWluZyB0aGUgb3V0cHV0DQoNCmBgYHtyfQ0KIyBmcm9tIGxpYnJhcnkoYnJvb20pDQp0aWR5KG1kbC5sbSkgJT4lIA0KICBzZWxlY3QodGVybSwgZXN0aW1hdGUpICU+JSANCiAgbXV0YXRlKGVzdGltYXRlID0gcm91bmQoZXN0aW1hdGUsIDMpKQ0KbXljb2VmRSA8LSB0aWR5KG1kbC5sbSkgJT4lIHB1bGwoZXN0aW1hdGUpDQoNCmBgYA0KDQoNCg0KVG8gaW50ZXJwcmV0IHRoZSBtb2RlbCwgd2UgbmVlZCBsb29rIGF0IHRoZSBjb2VmZmljaWVudHMuIFRoZSAiSW50ZXJjZXB0IiAoPVNldG9zYSkgaXMgYHIgbXljb2VmRVsxXWAgYW5kIHRoZSBjb2VmZmljaWVudHMgZm9yIFZlcnNpY29sb3IgYW5kIGZvciBWaXJnaW5pY2EgYXJlIHJlc3BlY3RpdmVseSBgciBteWNvZWZFWzJdYCBhbmQgYHIgbXljb2VmRVszXWAgVGhpcyB0ZWxscyB1cyB0aGF0IGNvbXBhcmVkIHRvIFNldG9zYSwgbW92aW5nIGZyb20gdGhpcyBjYXRlZ29yeSB0byBWZXJzaWNvbG9yIGxlYWRzIHRvIGEgc2lnbmlmaWNhbnQgaW5jcmVhc2UgYnkgYHIgbXljb2VmRVsyXWAsIGFuZCBmb3IgVmlyZ2luaWNhLCB0aGVyZSBpcyBhIHNpZ25pZmljYW50IGluY3JlYXNlIGJ5IGByIG15Y29lZkVbM11gLiANCg0KIyMjIE9idGFpbmluZyBvdXIgInRydWUiIGNvZWZmaWNpZW50cw0KDQpCdXQgd2hlcmUgYXJlIG91ciBhY3R1YWwgdmFsdWVzIGJhc2VkIG9uIHRoZSBtZWFucyBpbiB0aGUgdGFibGUgYWJvdmU/DQpXZSBydW4gYSBtb2RlbCB0aGF0IHN1cHByZXNzZXMgdGhlIGludGVyY2VwdCAoaS5lLiwgYWRkaW5nIDAgaW5zdGVhZCBvZiAxKSBhbmQgdGhpcyB3aWxsIGFsbG93IHVzIHRvIG9idGFpbiB0aGUgInRydWUiIGNvZWZmaWNpZW50cyBmb3IgZWFjaCBsZXZlbCBvZiBvdXIgcHJlZGljdG9yDQoNCmBgYHtyfQ0KbWRsLmxtLjIgPC0gbG0oU2VwYWwuTGVuZ3RoIH4gMCArIFNwZWNpZXMsIGRhdGEgPSBpcmlzKQ0Kc3VtbWFyeShtZGwubG0uMikNCmBgYA0KDQpUaGlzIG1hdGNoZXMgdGhlIG9yaWdpbmFsIGRhdGEuIFNldG9zYSBoYXMgYSBtZWFuIG9mIGByIG15Y29lZkVbMV1gLCBWZXJzaWNvbG9yIGByIG15Y29lZkVbMV0gKyBteWNvZWZFWzJdYCwgYW5kIFZpcmdpbmljYSBgciBteWNvZWZFWzFdICsgbXljb2VmRVsyXWAuIFNlZSB0YWJsZSBhYm92ZSBhbmQgY29lZmZpY2llbnRzIGJlbG93IA0KDQpgYGB7cn0NCiNTZXRvc2ENCm15Y29lZkVbMV0NCiNWZXJzaWNvbG9yDQpteWNvZWZFWzFdICsgbXljb2VmRVsyXQ0KI1ZpcmdpbmljYQ0KbXljb2VmRVsxXSArIG15Y29lZkVbM10NCmBgYA0KDQpUaGUgc2FtZSBhcw0KDQpgYGB7cn0NCnRpZHkobWRsLmxtLjIpICU+JSANCiAgc2VsZWN0KHRlcm0sIGVzdGltYXRlKSAlPiUgDQogIG11dGF0ZShlc3RpbWF0ZSA9IHJvdW5kKGVzdGltYXRlLCAzKSkNCm15Y29lZkUgPC0gdGlkeShtZGwubG0uMikgJT4lDQogIHB1bGwoZXN0aW1hdGUpDQoNCiNTZXRvc2ENCm15Y29lZkVbMV0NCiNWZXJzaWNvbG9yDQpteWNvZWZFWzJdDQojVmlyZ2luaWNhDQpteWNvZWZFWzNdDQpgYGANCg0KDQojIyMgTmljZSB0YWJsZSBvZiBvdXIgbW9kZWwgc3VtbWFyeQ0KDQpXZSBjYW4gYWxzbyBvYnRhaW4gYSBuaWNlIHRhYmxlIG9mIG91ciBtb2RlbCBzdW1tYXJ5LiBXZSBjYW4gdXNlIHRoZSBwYWNrYWdlIGBrbml0cmAgb3IgYHh0YWJsZWANCg0KIyMjIyBEaXJlY3RseSBmcm9tIG1vZGVsIHN1bW1hcnkNCg0KYGBge3J9DQprYWJsZShzdW1tYXJ5KG1kbC5sbSkkY29lZiwgZGlnaXRzPTMpDQoNCmBgYA0KDQojIyMjIEZyb20gdGhlIGB0aWR5YCBvdXRwdXQNCg0KYGBge3J9DQptZGwubG1UIDwtIHRpZHkobWRsLmxtKQ0Ka2FibGUobWRsLmxtVCwgZGlnaXRzPTMpDQpgYGANCg0KDQojIyMgRGlzc2VjdGluZyB0aGUgbW9kZWwNCg0KTGV0IHVzIGRpc3NlY3QgdGhlIG1vZGVsLiBJZiB5b3UgdXNlICJzdHIiLCB5b3Ugd2lsbCBiZSBhYmxlIHRvIHNlZSB3aGF0IGlzIGF2YWlsYWJsZSB1bmRlciBvdXIgbGluZWFyIG1vZGVsLiBUbyBhY2Nlc3Mgc29tZSBpbmZvIGZyb20gdGhlIG1vZGVsDQoNCiMjIyMgInN0ciIgYW5kICJjb2VmIg0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCnN0cihtZGwubG0pDQpjb2VmKG1kbC5sbSkNCiMjIHNhbWUgYXMgDQojIyBtZGwubG0kY29lZmZpY2llbnRzDQpgYGANCg0KIyMjIyAiY29lZiIgYW5kICJjb2VmZmljaWVudHMiDQoNCldoYXQgaWYgSSB3YW50IHRvIG9idGFpbiB0aGUgIkludGVyY2VwdCI/IE9yIHRoZSBjb2VmZmljaWVudCBmb3IgZGlzdGFuY2U/IFdoYXQgaWYgSSB3YW50IHRoZSBmdWxsIHJvdyBmb3IgZGlzdGFuY2U/DQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KY29lZihtZGwubG0pWzFdICMgc2FtZSBhcyBtZGwubG0kY29lZmZpY2llbnRzWzFdDQpjb2VmKG1kbC5sbSlbMl0gIyBzYW1lIGFzIG1kbC5sbSRjb2VmZmljaWVudHNbMl0NCg0Kc3VtbWFyeShtZGwubG0pJGNvZWZmaWNpZW50c1syLCBdICMgZnVsbCByb3cNCnN1bW1hcnkobWRsLmxtKSRjb2VmZmljaWVudHNbMiwgNF0gI2ZvciBwIHZhbHVlDQoNCmBgYA0KDQoNCiMjIyMgVXAgdG8geW91DQoNClBsYXkgYXJvdW5kIHdpdGggdGhlIG1vZGVsIHN1bW1hcnkgYW5kIG9idGFpbiB0aGUgdCB2YWx1ZXMgZm9yIHRoZSB0aHJlZSBsZXZlbHMuIFlvdSBjYW4gZG8gdGhpcyBieSByZWZlcnJpbmcgdG8gdGhlIGNvZWZmaWNpZW50IGFzIGFib3ZlDQoNCmBgYHtyfQ0KDQpgYGANCg0KDQojIyMjIFJlc2lkdWFscw0KDQpXaGF0IGFib3V0IHJlc2lkdWFscyAoZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSBvYnNlcnZlZCB2YWx1ZSBhbmQgdGhlIGVzdGltYXRlZCB2YWx1ZSBvZiB0aGUgcXVhbnRpdHkpIGFuZCBmaXR0ZWQgdmFsdWVzPw0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCmhpc3QocmVzaWR1YWxzKG1kbC5sbSkpDQpxcW5vcm0ocmVzaWR1YWxzKG1kbC5sbSkpOyBxcWxpbmUocmVzaWR1YWxzKG1kbC5sbSkpDQpwbG90KGZpdHRlZChtZGwubG0pLHJlc2lkdWFscyhtZGwubG0pLCBjZXggPSA0KQ0KYGBgDQoNCiMjIyMgR29vZG5lc3Mgb2YgZml0Pw0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCkFJQyhtZGwubG0pCSMgQWthaWtlJ3MgSW5mb3JtYXRpb24gQ3JpdGVyaW9uLCBsb3dlciB2YWx1ZXMgYXJlIGJldHRlcg0KQklDKG1kbC5sbSkJIyBCYXllc2lhbiBBSUMNCmxvZ0xpayhtZGwubG0pCSMgbG9nIGxpa2VsaWhvb2QNCmBgYA0KDQoNCk9yIHVzZSB0aGUgZm9sbG93aW5nIGZyb20gYGJyb29tYA0KDQpgYGB7cn0NCmdsYW5jZShtZGwubG0pDQpgYGANCg0KDQojIyMjIFNpZ25pZmljYW5jZSB0ZXN0aW5nDQoNCkFyZSB0aGUgYWJvdmUgaW5mb3JtYXRpdmU/IG9mIGNvdXJzZSBub3QgZGlyZWN0bHkuIElmIHdlIHdhbnQgdG8gdGVzdCBmb3Igb3ZlcmFsbCBzaWduaWZpY2FuY2Ugb2YgbW9kZWwuIFdlIHJ1biBhIG51bGwgbW9kZWwgKGFrYSBpbnRlcmNlcHQgb25seSkgYW5kIGNvbXBhcmUgbW9kZWxzLg0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCm1kbC5sbS5OdWxsIDwtIGxtKFNlcGFsLkxlbmd0aCB+IDEsIGRhdGE9aXJpcykNCm1kbC5jb21wIDwtIGFub3ZhKG1kbC5sbS5OdWxsLCBtZGwubG0pDQptZGwuY29tcA0KYGBgDQoNClRoZSByZXN1bHRzIHNob3cgdGhhdCBhZGRpbmcgdGhlIGZhY3RvciAiU3BlY2llcyIgaW1wcm92ZXMgdGhlIG1vZGVsIGZpdC4gV2UgY2FuIHdyaXRlIHRoaXMgYXMgZm9sbG93czogTW9kZWwgY29tcGFyaXNvbiBzaG93ZWQgdGhhdCB0aGUgYWRkaXRpb24gb2YgU3BlY2llcyBpbXByb3ZlZCB0aGUgbW9kZWwgZml0IHdoZW4gY29tcGFyZWQgd2l0aCBhbiBpbnRlcmNlcHQgb25seSBtb2RlbCAoJEYkKGByIG1kbC5jb21wWzIsM11gKSA9IGByIHJvdW5kKG1kbC5jb21wWzIsNV0sMilgLCAqcCogPCBgciBtZGwuY29tcFsyLDZdYCkgDQoNCiMjIyMgUGxvdHRpbmcgZml0dGVkIHZhbHVlcw0KDQojIyMjIyBUcmVuZCBsaW5lDQoNCkxldCdzIHBsb3Qgb3VyIGZpdHRlZCB2YWx1ZXMgYnV0IG9ubHkgZm9yIHRoZSB0cmVuZCBsaW5lDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KaXJpcyAlPiUgDQogIGdncGxvdChhZXMoeCA9IFNwZWNpZXMsIHkgPSBTZXBhbC5MZW5ndGgpKSsNCiAgZ2VvbV9ib3hwbG90KCkgKw0KICBsYWJzKHggPSAiU3BlY2llcyIsIHkgPSAiTGVuZ3RoIiwgdGl0bGUgPSAiQm94cGxvdCBhbmQgcHJlZGljdGVkIHRyZW5kIGxpbmUiLCBzdWJ0aXRsZSA9ICJ3aXRoIGdncGxvdDIiKSArIA0KICB0aGVtZV9idygpICsgdGhlbWUodGV4dCA9IGVsZW1lbnRfdGV4dChzaXplID0gMTUpKSsNCiAgZ2VvbV9zbW9vdGgoYWVzKHggPSBhcy5udW1lcmljKFNwZWNpZXMpLCB5ID0gcHJlZGljdChtZGwubG0pKSwgbWV0aG9kID0gImxtIiwgY29sb3IgPSAiYmx1ZSIpDQpgYGANCg0KVGhpcyBhbGxvd3MgdXMgdG8gcGxvdCB0aGUgZml0dGVkIHZhbHVlcyBmcm9tIG91ciBtb2RlbCB3aXRoIHRoZSBwcmVkaWN0ZWQgbGluZWFyIHRyZW5kLiBUaGlzIGlzIGV4YWN0bHkgdGhlIHNhbWUgYXMgb3VyIG9yaWdpbmFsIGRhdGEuDQoNCiMjIyMjIFByZWRpY3RlZCBtZWFucyBhbmQgdGhlIHRyZW5kIGxpbmUNCg0KV2UgY2FuIGFsc28gcGxvdCB0aGUgcHJlZGljdGVkIG1lYW5zIGFuZCBsaW5lYXIgdHJlbmQNCg0KYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgZXJyb3I9RkFMU0V9DQppcmlzICU+JSANCiAgZ2dwbG90KGFlcyh4ID0gU3BlY2llcywgeSA9IHByZWRpY3QobWRsLmxtKSkpKw0KICBnZW9tX2JveHBsb3QoY29sb3IgPSAiYmx1ZSIpICsNCiAgbGFicyh4ID0gIlNwZWNpZXMiLCB5ID0gIkxlbmd0aCIsIHRpdGxlID0gIlByZWRpY3RlZCBtZWFucyBhbmQgdHJlbmQgbGluZSIsIHN1YnRpdGxlID0gIndpdGggZ2dwbG90MiIpICsgDQogIHRoZW1lX2J3KCkgKyB0aGVtZSh0ZXh0ID0gZWxlbWVudF90ZXh0KHNpemUgPSAxNSkpKw0KICBnZW9tX3Ntb290aChhZXMoeCA9IGFzLm51bWVyaWMoU3BlY2llcyksIHkgPSBwcmVkaWN0KG1kbC5sbSkpLCBtZXRob2QgPSAibG0iLCBjb2xvciA9ICJibHVlIikNCmBgYA0KDQoNCiMjIyMjIFJhdyBkYXRhLCBwcmVkaWN0ZWQgbWVhbnMgYW5kIHRoZSB0cmVuZCBsaW5lDQoNCldlIGNhbiBhbHNvIHBsb3QgdGhlIGFjdHVhbCBkYXRhLCB0aGUgcHJlZGljdGVkIG1lYW5zIGFuZCBsaW5lYXIgdHJlbmQNCg0KYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgZXJyb3I9RkFMU0V9DQppcmlzICU+JSANCiAgZ2dwbG90KGFlcyh4ID0gU3BlY2llcywgeSA9IFNlcGFsLkxlbmd0aCkpKw0KICBnZW9tX2JveHBsb3QoKSsNCiAgZ2VvbV9ib3hwbG90KGFlcyh4ID0gU3BlY2llcywgeSA9IHByZWRpY3QobWRsLmxtKSksIGNvbG9yID0gImJsdWUiKSArDQogIGxhYnMoeCA9ICJTcGVjaWVzIiwgeSA9ICJMZW5ndGgiLCB0aXRsZSA9ICJCb3hwbG90IHJhdyBkYXRhLCBwcmVkaWN0ZWQgbWVhbnMgKGluIGJsdWUpIGFuZCB0cmVuZCBsaW5lIiwgc3VidGl0bGUgPSAid2l0aCBnZ3Bsb3QyIikgKyANCiAgdGhlbWVfYncoKSArIHRoZW1lKHRleHQgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDE1KSkrDQogIGdlb21fc21vb3RoKGFlcyh4ID0gYXMubnVtZXJpYyhTcGVjaWVzKSwgeSA9IHByZWRpY3QobWRsLmxtKSksIG1ldGhvZCA9ICJsbSIsIGNvbG9yID0gImJsdWUiKQ0KYGBgDQoNCg0KIyMjIFdoYXQgYWJvdXQgcGFpcndpc2UgY29tcGFyaXNvbj8NCg0KQmFzZWQgb24gb3VyIG1vZGVsJ3Mgc3VtbWFyeSwgY2FuIHlvdSB0ZWxsIG1lIGlmIHRoZXJlIGlzIGEgZGlmZmVyZW5jZSBiZXR3ZWVuIFZlcnNpY29sb3IgYW5kIFZpcmdpbmljYT8NCg0KYGBge3J9DQpzdW1tYXJ5KG1kbC5sbSkNCmBgYA0KDQoNCmBgYHtyfQ0KZW1tZWFucyhtZGwubG0sIHBhaXJ3aXNlIH4gU3BlY2llcywgYWRqdXN0ID0gImZkciIpDQpgYGANCg0KSG93IHRvIGludGVycHJldCB0aGUgb3V0cHV0PyBEaXNjdXNzIHdpdGggeW91ciBuZWlnaGJvdXIgYW5kIHNoYXJlIHdpdGggdGhlIGdyb3VwLg0KDQpIaW50Li4uIExvb2sgYXQgdGhlIGVtbWVhbnMgdmFsdWVzIGZvciBlYWNoIGxldmVsIG9mIG91ciBmYWN0b3IgIlNwZWNpZXMiIGFuZCB0aGUgY29udHJhc3RzLiANCg0KIyMgV2hhdCBhYm91dCB0aGUgb3RoZXIgb3V0Y29tZXM/DQoNClNvIGZhciwgd2Ugb25seSBsb29rZWQgYXQgIlNlcGFsLkxlbmd0aCIuIFdoYXQgYWJvdXQgdGhlIG90aGVyIG91dGNvbWVzPyBob3cgaW5mb3JtYXRpdmUgYXJlIHRoZXk/IERvIHdlIGhhdmUgc3RhdGlzdGljYWwgZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSB0aHJlZSBsZXZlbHMgb2Ygb3VyIHByZWRpY3Rvcj8gWW91IGNhbiBkbyB0aGlzIGluIHlvdXIgc3BhcmUgdGltZS4NCg0KIyMgQ29uY2x1c2lvbg0KDQpXZSBoYXZlIHNvIGZhciBsb29rZWQgYXQgdGhlIExpbmVhciBNb2RlbC4gVGhlIHVuZGVybHlpbmcgYXNzdW1wdGlvbiBhYm91dCBsaW5lYXIgbW9kZWxzIGlzIHRoYXQgd2UgaGF2ZSBhIG5vcm1hbCAoR2F1c3NpYW4pIGRpc3RyaWJ1dGlvbi4gVGhpcyBpcyB0aGUgbW9kZWwgdG8gYmUgdXNlZCB3aGVuIHdlIGhhdmUgYSBudW1lcmljIG91dGNvbWUuIFdoYXQgaWYgb3VyIG91dGNvbWUgaXMgbm90IG51bWVyaWM/IFdoYXQgaWYgd2UgaGF2ZSB0d28gY2F0ZWdvcmllcywgaS5lLiwgYmxhY2sgdnMgd2hpdGU/IGNvcnJlY3QgdnMgaW5jb3JyZWN0PyB5ZXMgdnMgbm8/IFRoZXNlIGFyZSBjYXRlZ29yaWNhbCAqKmJpbmFyeSoqIG91dGNvbWUuIFdlIGxvb2sgaW4gdGhlIG5leHQgc2VjdGlvbiBhdCBMb2dpc3RpYyBSZWdyZXNzaW9uDQoNCiMgRnJvbSBMaW5lYXIgdG8gTG9naXN0aWMgbW9kZWxzDQoNCkhlcmUgd2Ugd2lsbCBsb29rIGF0IGFuIGV4YW1wbGUgd2hlbiB0aGUgb3V0Y29tZSBpcyBiaW5hcnkuIFRoaXMgc2ltdWxhdGVkIGRhdGEgaXMgc3RydWN0dXJlZCBhcyBmb2xsb3dzLiBXZSBhc2tlZCBvbmUgcGFydGljaXBhbnQgdG8gbGlzdGVuIHRvIDE2NSBzZW50ZW5jZXMsIGFuZCB0byBqdWRnZSB3aGV0aGVyIHRoZXNlIGFyZSAiZ3JhbW1hdGljYWwiIG9yICJ1bmdyYW1tYXRpY2FsIi4gVGhlcmUgd2VyZSAxMDUgc2VudGVuY2VzIHRoYXQgd2VyZSAiZ3JhbW1hdGljYWwiIGFuZCA2MCAidW5ncmFtbWF0aWNhbCIuIFRoaXMgZmljdGl0aW91cyBleGFtcGxlIGNhbiBhcHBseSBpbiBhbnkgb3RoZXIgc2l0dWF0aW9uLiBMZXQncyB0aGluayBHZW9ncmFwaHk6IDE2NSBsYW5kczogMTA1ICJmbGF0IiBhbmQgNjAgIm5vbi1mbGF0IiwgZXRjLiBUaGlzIGFwcGxpZXMgdG8gYW55IGNhc2Ugd2hlcmUgeW91IG5lZWQgdG8gImNhdGVnb3Jpc2UiIHRoZSBvdXRjb21lIGludG8gdHdvIGdyb3Vwcy4gDQoNCiMjIExvYWQgYW5kIHN1bW1hcmllcw0KDQpMZXQncyBsb2FkIGluIHRoZSBkYXRhIGFuZCBkbyBzb21lIGJhc2ljIHN1bW1hcmllcw0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCmdyYW1tYXRpY2FsIDwtIHJlYWQuY3N2KCJncmFtbWF0aWNhbC5jc3YiKQ0Kc3RyKGdyYW1tYXRpY2FsKQ0KaGVhZChncmFtbWF0aWNhbCkNCmdyYW1tYXRpY2FsJHJlc3BvbnNlIDwtIHJlbGV2ZWwoZ3JhbW1hdGljYWwkcmVzcG9uc2UsIHJlZiA9ICJ5ZXMiKQ0KdGFibGUoZ3JhbW1hdGljYWwkZ3JhbW1hdGljYWxpdHksIGdyYW1tYXRpY2FsJHJlc3BvbnNlKQ0KYGBgDQoNCiMjIEFjY3VyYWN5IGFuZCBTaWduYWwgRGV0ZWN0aW9uIFRoZW9yeQ0KDQojIyMgUmF0aW9uYWxlDQoNCldlIGFyZSBnZW5lcmFsbHkgaW50ZXJlc3RlZCBpbiBwZXJmb3JtYW5jZSwgaS5lLiwgd2hldGhlciB0aGUgd2UgaGF2ZSAiYWNjdXJhdGVseSIgY2F0ZWdvcmlzZWQgdGhlIG91dGNvbWUgb3Igbm90IGFuZCBhdCB0aGUgc2FtZSB0aW1lIHdhbnQgdG8gZXZhbHVhdGUgb3VyIGJpYXNlcyBpbiByZXNwb25zZXMuIFdoZW4gZGVjaWRpbmcgb24gY2F0ZWdvcmllcywgd2UgYXJlIHVzdWFsbHkgYmlhc2VkIGluIG91ciBzZWxlY3Rpb24uIA0KDQpMZXQncyBhc2sgdGhlIHF1ZXN0aW9uOiBIb3cgbWFueSBvZiB5b3UgaGF2ZSBhIE1hYyBsYXB0b3AgYW5kIGhvdyBtYW55IGEgV2luZG93cyBsYXB0b3A/IEZvciB0aG9zZSB3aXRoIGEgTWFjLCB3aGF0IHdhcyB0aGUgbWFpbiByZWFzb24gZm9yIGNob29zaW5nIGl0PyBBcmUgeW91IGJpYXNlZCBpbiBhbnl3YXkgYnkgeW91ciBkZWNpc2lvbj8gDQoNClRvIGNvcnJlY3QgZm9yIHRoZXNlIGJpYXNlcywgd2UgdXNlIHNvbWUgdmFyaWFudHMgZnJvbSBTaWduYWwgRGV0ZWN0aW9uIFRoZW9yeSB0byBvYnRhaW4gdGhlIHRydWUgZXN0aW1hdGVzIHdpdGhvdXQgYmVpbmcgaW5mbHVlbmNlZCBieSB0aGUgYmlhc2VzLiANCg0KIyMjIFJ1bm5pbmcgc3RhdHMNCg0KTGV0J3MgZG8gc29tZSBzdGF0cyBvbiB0aGlzIA0KDQp8ICB8IFllcyB8IE5vIHwgVG90YWwgfA0KfC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tLS18LS0tLS0tLS0tLS0tLS0tLS0tfC0tLS0tLS0tLS0tLS0tLS0tLXwNCnwgR3JhbW1hdGljYWwgKFllcyBBY3R1YWwpIHwgVFAgPSAxMDAgfCBGTiA9IDUgfCAoWWVzIEFjdHVhbCkgMTA1IHwNCnwgVW5ncmFtbWF0aWNhbCAoTm8gQWN0dWFsKSAgfCBGUCA9IDEwIHwgVE4gPSA1MCB8IChObyBBY3R1YWwpIDYwIHwNCnwgVG90YWwgfCAoWWVzIFJlc3BvbnNlKSAxMTAgfCAoTm8gUmVzcG9uc2UpIDU1IHwgMTY1IHwNCg0KYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgZXJyb3I9RkFMU0V9DQojIyBUUCA9IFRydWUgUG9zaXRpdmUgKEhpdCk7IEZQID0gRmFsc2UgUG9zaXRpdmU7IEZOID0gRmFsc2UgTmVnYXRpdmU7IFROID0gVHJ1ZSBOZWdhdGl2ZQ0KDQoNClRQID0gMTAwDQpGUCA9IDEwDQpGTiA9IDUNClROID0gNTANClRvdGFsID0gMTY1DQoNCihUUCtUTikvVG90YWwgIyBhY2N1cmFjeQ0KKEZQK0ZOKS9Ub3RhbCAjIGVycm9yLCBhbHNvIDEtYWNjdXJhY3kNCg0KIyBXaGVuIHN0aW11bHVzID0geWVzLCBob3cgbWFueSB0aW1lcyByZXNwb25zZSA9IHllcz8NClRQLyhUUCtGTikgIyBhbHNvIFRydWUgUG9zaXRpdmUgUmF0ZSBvciBTcGVjaWZpY2l0eQ0KDQojIFdoZW4gc3RpbXVsdXMgPSBubywgaG93IG1hbnkgdGltZXMgcmVzcG9uc2UgPSB5ZXM/DQpGUC8oRlArVE4pICMgRmFsc2UgUG9zaXRpdmUgUmF0ZSwgDQoNCiMgV2hlbiBzdGltdWx1cyA9IG5vLCBob3cgbWFueSB0aW1lcyByZXNwb25zZSA9IG5vPw0KVE4vKEZQK1ROKSAjIFRydWUgTmVnYXRpdmUgUmF0ZSBvciBTZW5zaXRpdml0eSANCg0KIyBXaGVuIHN1YmplY3QgcmVzcG9uZHMgInllcyIgaG93IG1hbnkgdGltZXMgaXMgKHMpaGUgY29ycmVjdD8NClRQLyhUUCtGUCkgIyBwcmVjaXNpb24NCg0KIyBnZXR0aW5nIGRwcmltZSAob3IgdGhlIHNlbnNldGl2aXR5IGluZGV4KTsgYmV0YSAoYmlhcyBjcml0ZXJpb24sIDAtMSwgbG93ZXI9aW5jcmVhc2UgaW4gInllcyIpOyBBcHJpbWUgKGVzdGltYXRlIG9mIGRpc2NyaW1pbmFiaWxpdHksIDAtMSwgMT1nb29kIGRpc2NyaW1pbmF0aW9uOyAwIGF0IGNoYW5jZSk7IGJwcGQgKGIgcHJpbWUgcHJpbWUgZCwgLTEgdG8gMTsgMCA9IG5vIGJpYXMsIG5lZ2F0aXZlID0gdGVuZGVuY3kgdG8gcmVzcG9uZCAieWVzIiwgcG9zaXRpdmUgPSB0ZW5kZW5jeSB0byByZXNwb25kICJubyIpOyBjIChpbmRleCBvZiBiaWFzLCBlcXVhbHMgdG8gU0QpDQojKHNlZSBhbHNvIGh0dHBzOi8vd3d3LnItYmxvZ2dlcnMuY29tL2NvbXB1dGUtc2lnbmFsLWRldGVjdGlvbi10aGVvcnktaW5kaWNlcy13aXRoLXIvYW1wLykgDQpwc3ljaG86OmRwcmltZShUUCwgRlAsIEZOLCBUTiwgDQogICAgICAgICAgICAgICBuX3RhcmdldHMgPSBUUCtGTiwgDQogICAgICAgICAgICAgICBuX2Rpc3RyYWN0b3JzID0gRlArVE4sDQogICAgICAgICAgICAgICBhZGp1c3Q9RikNCg0KYGBgDQoNClRoZSBtb3N0IGltcG9ydGFudCBmcm9tIGFib3ZlLCBpcyBkLXByaW1lLiBUaGlzIGlzIG1vZGVsbGluZyB0aGUgZGlmZmVyZW5jZSBiZXR3ZWVuIHRoZSByYXRlIG9mICJUcnVlIFBvc2l0aXZlIiByZXNwb25zZXMgYW5kICJGYWxzZSBQb3NpdGl2ZSIgcmVzcG9uc2VzIGluIHN0YW5kYXJkIHVuaXQgKG9yIHotc2NvcmVzKS4gVGhlIGZvcm11bGEgY2FuIGJlIHdyaXR0ZW4gYXM6DQoNCmBkJyAoZCBwcmltZSkgPSBaKFRydWUgUG9zaXRpdmUgUmF0ZSkgLSBaKEZhbHNlIFBvc2l0aXZlIFJhdGUpYA0KDQoNCiMjIEdMTQ0KDQpMZXQncyBydW4gYSBmaXJzdCBHTE0gKEdlbmVyYWxpc2VkIExpbmVhciBNb2RlbCkuIEEgR0xNIHVzZXMgYSBzcGVjaWFsIGZhbWlseSAiYmlub21pYWwiIGFzIGl0IGFzc3VtZXMgdGhlIG91dGNvbWUgaGFzIGEgYmlub21pYWwgZGlzdHJpYnV0aW9uLiBJbiBnZW5lcmFsLCByZXN1bHRzIGZyb20gYSBMb2dpc3RpYyBSZWdyZXNzaW9uIGFyZSBjbG9zZSB0byB3aGF0IHdlIGdldCBmcm9tIFNEVCAoc2VlIGFib3ZlKS4NCg0KVG8gcnVuIHRoZSByZXN1bHRzLCB3ZSB3aWxsIGNoYW5nZSB0aGUgcmVmZXJlbmNlIGxldmVsIGZvciBib3RoIHJlc3BvbnNlIGFuZCBncmFtbWF0aWNhbGl0eS4gVGhlIGJhc2ljIGFzc3VtcHRpb24gYWJvdXQgR0xNIGlzIHRoYXQgd2Ugc3RhcnQgd2l0aCBvdXIgcmVmZXJlbmNlIGxldmVsIGJlaW5nIHRoZSAibm8iIHJlc3BvbnNlcyB0byB0aGUgInVuZ3JhbW1hdGljYWwiIGNhdGVnb3J5LiBBbnkgY2hhbmdlcyB0byB0aGlzIHJlZmVyZW5jZSB3aWxsIGJlIHNlZW4gaW4gdGhlIGNvZWZmaWNpZW50cyBhcyAieWVzIiByZXNwb25zZXMgdG8gdGhlICJncmFtbWF0aWNhbCIgY2F0ZWdvcnkuDQoNCiMjIyBNb2RlbCBlc3RpbWF0aW9uIGFuZCByZXN1bHRzDQoNClRoZSByZXN1bHRzIGJlbG93IHNob3cgdGhlIGxvZ29kZHMgZm9yIG91ciBtb2RlbC4gDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KbGV2ZWxzKGdyYW1tYXRpY2FsJHJlc3BvbnNlKQ0KbGV2ZWxzKGdyYW1tYXRpY2FsJGdyYW1tYXRpY2FsaXR5KQ0KDQpncmFtbWF0aWNhbCRyZXNwb25zZSA8LSByZWxldmVsKGdyYW1tYXRpY2FsJHJlc3BvbnNlLCByZWYgPSAibm8iKQ0KZ3JhbW1hdGljYWwkZ3JhbW1hdGljYWxpdHkgPC0gcmVsZXZlbChncmFtbWF0aWNhbCRncmFtbWF0aWNhbGl0eSwgcmVmID0gInVuZ3JhbW1hdGljYWwiKQ0KDQptZGwuZ2xtIDwtIGdsbShyZXNwb25zZSB+IGdyYW1tYXRpY2FsaXR5LCBkYXRhID0gZ3JhbW1hdGljYWwsIGZhbWlseSA9IGJpbm9taWFsKQ0Kc3VtbWFyeShtZGwuZ2xtKQ0KDQp0aWR5KG1kbC5nbG0pICU+JSANCiAgc2VsZWN0KHRlcm0sIGVzdGltYXRlKSAlPiUgDQogIG11dGF0ZShlc3RpbWF0ZSA9IHJvdW5kKGVzdGltYXRlLCAzKSkNCiMgdG8gb25seSBnZXQgdGhlIGNvZWZmaWNpZW50cw0KbXljb2VmMiA8LSB0aWR5KG1kbC5nbG0pICU+JSBwdWxsKGVzdGltYXRlKQ0KYGBgDQoNCg0KVGhlIHJlc3VsdHMgc2hvdyB0aGF0IGZvciBvbmUgdW5pdCBpbmNyZWFzZSBpbiB0aGUgcmVzcG9uc2UgKGkuZS4sIGZyb20gbm8gdG8geWVzKSwgdGhlIGxvZ29kZHMgb2YgYmVpbmcgImdyYW1tYXRpY2FsIiBpcyBpbmNyZWFzZWQgYnkgYHIgbXljb2VmMlsyXWAgKHRoZSBpbnRlcmNlcHQgc2hvd3MgdGhhdCB3aGVuIHRoZSByZXNwb25zZSBpcyAibm8iLCB0aGUgbG9nb2RkcyBhcmUgYHIgbXljb2VmMlsxXWApLiBUaGUgYWN0dWFsIGxvZ29kZHMgZm9yIHRoZSByZXNwb25zZSAieWVzIiB0byBncmFtbWF0aWNhbCBpcyBgciBteWNvZWYyWzFdK215Y29lZjJbMl1gIA0KDQojIyMgR2V0dGluZyAidHJ1ZSIgY29lZmZpY2llbnRzIGZyb20gR0xNDQoNCg0KSG93IHRvIGdldCB0aGUgdHJ1ZSBjb2VmZmljaWVudHM/IEhvdyB0byBzdXBwcmVzcyB0aGUgSW50ZXJjZXB0Pw0KDQpIZXJlIHdlIHdpbGwgcnVuIHRoZSBzYW1lIEdMTSBtb2RlbCBhYm92ZSBidXQgYnkgc3VwcHJlc3NpbmcgdGhlICJJbnRlcmNlcHQiLiBUaGUgaWRlYSBoZXJlIGlzIHRvIGdldCB0aGUgInRydWUiIGNvZWZmaWNpZW50IGZvciBhICJncmFtbWF0aWNhbCIgYW5kIGEgInllcyIgcmVzcG9uc2UuIFdlIGNhbiB1c2UgdGhlIGFib3ZlIGNvZGUsIGkuZS4sICJteWNvZWYyWzFdK215Y29lZjJbMl0iIG9yIGFzIGJlbG93DQoNCg0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCiMgQW4gaW50ZXJjZXB0IGlzIGFsd2F5cyBpbmNsdWRlZCBpbiBhbnkgcmVncmVzc2lvbiwgYnV0IHlvdSBjYW4gc3BlY2lmeSBpdCB3aXRoICIxIg0KIyMgZ2xtKHJlc3BvbnNlfjErZ3JhbW1hdGljYWxpdHksZGF0YT1ncmFtbWF0aWNhbCxmYW1pbHkgPSBiaW5vbWlhbCkNCg0KbWRsLmdsbTIgPC0gZ2xtKHJlc3BvbnNlIH4gMCArIGdyYW1tYXRpY2FsaXR5LCBkYXRhID0gZ3JhbW1hdGljYWwsIGZhbWlseSA9IGJpbm9taWFsKQ0Kc3VtbWFyeShtZGwuZ2xtMikNCg0KDQpgYGANCg0KIyMjIExvZ29kZHMgdG8gT2RkIHJhdGlvcw0KDQpMb2dvZGRzIGNhbiBiZSBtb2RpZmllZCB0byB0YWxrIGFib3V0IHRoZSBvZGRzIG9mIGFuIGV2ZW50LiBGb3Igb3VyIG1vZGVsIGFib3ZlLCB0aGUgb2RkcyBvZiAiZ3JhbW1hdGljYWwiIHJlY2VpdmluZyBhICJ5ZXMiIHJlc3BvbnNlIGluY3JlYXNlIGJ5IDEwMDsgd2hlcmVhcyByZWNlaXZpbmcgYSAibm8iIGlzIGEgbWVyZSAwLjIuIFVzaW5nIHRoZSBzZWNvbmQgbW9kZWwgKGkuZS4sIHdpdGhvdXQgYW4gSW50ZXJjZXB0KSBhbGxvd3MgdXMgdG8gZ2V0IHRoZSBvZGQgcmF0aW9zIGZvciBlYWNoIG9mIHJlc3BvbnNlcyB0byAiVW5ncmFtbWF0aWNhbCIgYW5kICJHcmFtbWF0aWNhbCIsIHdoaWNoIGlzIHN0aWxsIDAuMiBmb3IgInVuZ3JhbW1hdGljYWwiIGFuZCAyMCBmb3IgImdyYW1tYXRpY2FsIiwgaS5lLiwgMjAgdGltZXMgbW9yZQ0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCmV4cChjb2VmKG1kbC5nbG0pKQ0KZXhwKGNvZWYobWRsLmdsbTIpKQ0KDQpgYGANCg0KIyMjIExvZ09kZHMgdG8gcHJvcG9ydGlvbnMNCg0KSWYgeW91IHdhbnQgdG8gdGFsayBhYm91dCB0aGUgcGVyY2VudGFnZSAiYWNjdXJhY3kiIG9mIG91ciBtb2RlbCwgdGhlbiB3ZSBjYW4gdHJhbnNmb3JtIG91ciBsb2dnb2RkcyBpbnRvIHByb3BvcnRpb25zLiBUaGlzIHNob3dzIHRoYXQgdGhlIHByb3BvcnRpb24gb2YgImdyYW1tYXRpY2FsIiByZWNlaXZpbmcgYSAieWVzIiByZXNwb25zZSBpbmNyZWFzZXMgYnkgOTklIChvciA5NSUgYmFzZWQgb24gb3VyICJ0cnVlIiBjb2VmZmljaWVudHMpDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KcGxvZ2lzKGNvZWYobWRsLmdsbSkpDQpwbG9naXMoY29lZihtZGwuZ2xtMikpDQpgYGANCg0KIyMjIEdMTSBhcyBhIGNsYXNzaWZpY2F0aW9uIHRvb2wNCg0KVGhlIGNvZGUgYmVsb3cgZGVtb25zdHJhdGVzIHRoZSBsaW5rcyBiZXR3ZWVuIG91ciBHTE0gbW9kZWwgYW5kIHdoYXQgd2UgaGFkIG9idGFpbmVkIGFib3ZlIGZyb20gU0RULiBUaGUgcHJlZGljdGlvbnMnIHRhYmxlIHNob3dzIHRoYXQgb3VyIEdMTSB3YXMgc3VjY2Vzc2Z1bCBhdCBvYnRhaW5pbmcgcHJlZGljdGlvbiB0aGF0IGFyZSBpZGVudGljYWwgdG8gb3VyIGluaXRpYWwgZGF0YSBzZXR1cC4gTG9vayBhdCB0aGUgdGFibGUgaGVyZSBhbmQgdGhlIHRhYmxlIGFib3ZlLiBPbmNlIHdlIGhhdmUgY3JlYXRlZCBvdXIgdGFibGUgb2Ygb3V0Y29tZSwgd2UgY2FuIGNvbXB1dGUgcGVyY2VudCBjb3JyZWN0LCB0aGUgc3BlY2lmaWNpdHksIHRoZSBzZW5zaXRpdml0eSwgdGhlIEthcHBhIHNjb3JlLCBldGMuLiB0aGlzIHlpZWxkcyB0aGUgYWN0dWFsIHZhbHVlIHdpdGggdGhlIFNEIHRoYXQgaXMgcmVsYXRlZCB0byB2YXJpYXRpb25zIGluIHJlc3BvbnNlcy4gDQoNCmBgYHtyfQ0KIyMgcHJlZGljdChtZGwuZ2xtKT4wLjUgaXMgaWRlbnRpY2FsIHRvIA0KIyMgcHJlZGljdChnbG0ocmVzcG9uc2V+Z3JhbW1hdGljYWxpdHksZGF0YT1ncmFtbWF0aWNhbCxmYW1pbHkgPSBiaW5vbWlhbCksdHlwZT0icmVzcG9uc2UiKQ0KZ3JhbW1hdGljYWwkcmVzcG9uc2UgPC0gcmVsZXZlbChncmFtbWF0aWNhbCRyZXNwb25zZSwgInllcyIpDQpncmFtbWF0aWNhbCRncmFtbWF0aWNhbGl0eSA8LSByZWxldmVsKGdyYW1tYXRpY2FsJGdyYW1tYXRpY2FsaXR5LCAiZ3JhbW1hdGljYWwiKQ0KbWRsLmdsbS5DIDwtIGdsbShyZXNwb25zZSB+IGdyYW1tYXRpY2FsaXR5LCBkYXRhID0gZ3JhbW1hdGljYWwsZmFtaWx5ID0gYmlub21pYWwpDQp0YmwuZ2xtIDwtIHRhYmxlKGdyYW1tYXRpY2FsJHJlc3BvbnNlLCBwcmVkaWN0KG1kbC5nbG0uQywgdHlwZSA9ICJyZXNwb25zZSIpPjAuNSkNCmNvbG5hbWVzKHRibC5nbG0pIDwtIGMoImdyYW1tYXRpY2FsIiwgInVuZ3JhbW1hdGljYWwiKQ0KdGJsLmdsbQ0KUHJlc2VuY2VBYnNlbmNlOjpwY2ModGJsLmdsbSkNClByZXNlbmNlQWJzZW5jZTo6c3BlY2lmaWNpdHkodGJsLmdsbSkNClByZXNlbmNlQWJzZW5jZTo6c2Vuc2l0aXZpdHkodGJsLmdsbSkNCiMjI2V0Yy4uDQpgYGANCg0KSWYgeW91IGxvb2sgYXQgdGhlIHJlc3VsdHMgZnJvbSBTRFQgYWJvdmUsIHRoZXNlIHJlc3VsdHMgYXJlIHRoZSBzYW1lIGFzDQp0aGUgZm9sbG93aW5nDQoNCkFjY3VyYWN5OiAoVFArVE4pL1RvdGFsIChgciAoVFArVE4pL1RvdGFsYCkgDQoNClRydWUgUG9zaXRpdmUgUmF0ZSAob3IgU3BlY2lmaWNpdHkpIFRQLyhUUCtGTikgKGByIFRQLyhUUCtGTilgKQ0KDQpUcnVlIE5lZ2F0aXZlIFJhdGUgKG9yIFNlbnNpdGl2aXR5KSBUTi8oRlArVE4pIChgciBUTi8oRlArVE4pYCkgDQoNCg0KIyMjIFBsb3R0aW5nDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KZ3JhbW1hdGljYWwkcHJvYiA8LSBwcmVkaWN0KGdsbShyZXNwb25zZSB+IGdyYW1tYXRpY2FsaXR5LCBkYXRhID0gZ3JhbW1hdGljYWwsIGZhbWlseSA9IGJpbm9taWFsKSwgdHlwZSA9ICJyZXNwb25zZSIpDQpncmFtbWF0aWNhbCAlPiUgDQogIGdncGxvdChhZXMoeCA9IGFzLm51bWVyaWMoZ3JhbW1hdGljYWxpdHkpLCB5ID0gcHJvYikpICsNCiAgZ2VvbV9wb2ludCgpICsNCiAgZ2VvbV9zbW9vdGgobWV0aG9kID0gImdsbSIsIA0KICAgIG1ldGhvZC5hcmdzID0gbGlzdChmYW1pbHkgPSAiYmlub21pYWwiKSwgDQogICAgc2UgPSBUKSArIHRoZW1lX2J3KGJhc2Vfc2l6ZSA9IDIwKSsNCiAgICBsYWJzKHkgPSAiUHJvYmFiaWxpdHkiLCB4ID0gIiIpKw0KICAgIGNvb3JkX2NhcnRlc2lhbih5bGltID0gYygwLDEpKSsNCiAgICBzY2FsZV94X2Rpc2NyZXRlKGxpbWl0cyA9IGMoIlVuZ3JhbW1hdGljYWwiLCAiR3JhbW1hdGljYWwiKSkNCmBgYA0KDQojIyMgR0xNIGFuZCBkIHByaW1lDQoNClRoZSB2YWx1ZXMgb2J0YWluZWQgaGVyZSBtYXRjaCB0aG9zZSBvYnRhaW5lZCBmcm9tIFNEVC4gRm9yIGQgcHJpbWUsIHRoZSBkaWZmZXJlbmNlIHN0ZW1zIGZyb20gdGhlIHVzZSBvZiB0aGUgbG9naXQgdmFyaWFudCBvZiB0aGUgQmlub21pYWwgZmFtaWx5LiBCeSB1c2luZyBhIHByb2JpdCB2YXJpYW50LCBvbmUgb2J0YWlucyB0aGUgc2FtZSB2YWx1ZXMgKFtzZWUgaGVyZV0oaHR0cHM6Ly9zdGF0cy5pZHJlLnVjbGEuZWR1L3IvZGFlL3Byb2JpdC1yZWdyZXNzaW9uLykgZm9yIG1vcmUgZGV0YWlscykuIEEgcHJvYml0IHZhcmlhbnQgbW9kZWxzIHRoZSB6LXNjb3JlIGRpZmZlcmVuY2VzIGluIHRoZSBvdXRjb21lIGFuZCBpcyBldmFsdWF0ZWQgaW4gY2hhbmdlIGluIDEtc3RhbmRhcmQgdW5pdC4gVGhpcyBpcyBtb2RlbGxpbmcgdGhlIGNoYW5nZSBmcm9tICJ1bmdyYW1tYXRpY2FsIiAibm8iIHJlc3BvbnNlcyBpbnRvICJncmFtbWF0aWNhbCIgInllcyIgcmVzcG9uc2VzIGluIHotc2NvcmVzLiBUaGUgc2FtZSBjb25jZXB0dWFsIHVuZGVycGlubmluZ3Mgb2YgZC1wcmltZSBmcm9tIFNpZ25hbCBEZXRlY3Rpb24gVGhlb3J5Lg0KDQpgYGB7cn0NCiMjIGQgcHJpbWUNCnBzeWNobzo6ZHByaW1lKFRQLCBGUCwgRk4sIFROLCANCiAgICAgICAgICAgICAgIG5fdGFyZ2V0cyA9IFRQK0ZOLCANCiAgICAgICAgICAgICAgIG5fZGlzdHJhY3RvcnMgPSBGUCtUTiwNCiAgICAgICAgICAgICAgIGFkanVzdD1GKSRkcHJpbWUNCg0KIyMgR0xNIHdpdGggcHJvYml0DQpjb2VmKGdsbShyZXNwb25zZSB+IGdyYW1tYXRpY2FsaXR5LCBkYXRhID0gZ3JhbW1hdGljYWwsIGZhbWlseSA9IGJpbm9taWFsKHByb2JpdCkpKVsyXQ0KDQpgYGANCg0KDQojIyBHTE06IE90aGVyIGRpc3RyaWJ1dGlvbnMNCg0KSWYgeW91ciBkYXRhIGRvZXMgbm90IGZpdCBhIGJpbm9taWFsIGRpc3RyaWJ1dGlvbiwgYW5kIGlzIGEgbXVsdGlub21pYWwgKGkuZS4sIHRocmVlIG9yIG1vcmUgcmVzcG9uc2UgY2F0ZWdvcmllcykgb3IgcG9pc3NvbiAoY291bnQgZGF0YSksIHRoZW4geW91IG5lZWQgdG8gdXNlIHRoZSBnbG0gZnVuY3Rpb24gd2l0aCBhIHNwZWNpZmljIGZhbWlseSBmdW5jdGlvbi4gDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFLCBlY2hvPUZBTFNFfQ0KIyMgRm9yIGEgbXVsdGlub21pYWwgKDMgb3IgbW9yZSByZXNwb25zZSBjYXRlZ29yaWVzKSwgc2VlIGJlbG93IGFuZCB1c2UgdGhlIGZvbGxvd2luZyBzcGVjaWZpY2F0aW9uDQojIyBodHRwczovL3N0YXRzLmlkcmUudWNsYS5lZHUvci9kYWUvbXVsdGlub21pYWwtbG9naXN0aWMtcmVncmVzc2lvbi8NCiMjIG1kbC5tdWx0aSA8LSBubmV0OjptdWx0aW5vbShvdXRjb21lfnByZWRpY3RvciwgZGF0YT1kYXRhKQ0KDQojIyBGb3IgYSBwb2lzc29uIChjb3VudCBkYXRhKSwgc2VlIGJlbG93IGFuZCB1c2UgdGhlIGZvbGxvd2luZyBzcGVjaWZpY2F0aW9uDQojIyBodHRwczovL3N0YXRzLmlkcmUudWNsYS5lZHUvci9kYWUvcG9pc3Nvbi1yZWdyZXNzaW9uLw0KDQojIyBtZGwucG9pc3NvbiA8LSBnbG0ob3V0Y29tZX5wcmVkaWN0b3IsIGRhdGE9ZGF0YSxmYW1pbHk9InBvaXNzb24iKQ0KDQoNCmBgYA0KDQoNCiMgQ3VtdWxhdGl2ZSBMaW5rIE1vZGVscw0KDQpUaGVzZSBtb2RlbHMgd29yayBwZXJmZWN0bHkgd2l0aCByYXRpbmcgZGF0YS4gUmF0aW5ncyBhcmUgaW5oZXJlbnRseSBvcmRlcmVkLCAxLCAyLCAuLi4gbiwgYW5kIGV4cGVjdCB0byBvYnNlcnZlIGFuIGluY3JlYXNlIChvciBkZWNyZWFzZSkgaW4gb3ZlcmFsbCByYXRpbmdzIGZyb20gMSB0byBuLiBUbyBkZW1vbnN0cmF0ZSB0aGlzLCB3ZSB3aWxsIHVzZSBhbiBleGFtcGxlIHVzaW5nIHRoZSBwYWNrYWdlICJvcmRpbmFsIi4gRGF0YSB3ZXJlIGZyb20gYSByYXRpbmcgZXhwZXJpbWVudCB3aGVyZSBzaXggcGFydGljaXBhbnRzIHJhdGVkIHRoZSBwZXJjZXB0IG9mIG5hc2FsaXR5IGluIHRoZSBwcm9kdWN0aW9uIG9mIHBhcnRpY3VsYXIgY29uc29uYW50cyBpbiBBcmFiaWMuIFRoZSBkYXRhIGNhbWUgZnJvbSBuaW5lIHByb2R1Y2luZyBzdWJqZWN0cy4gVGhlIHJhdGluZ3Mgd2VyZSBmcm9tIDEgdG8gNS4gVGhpcyBleGFtcGxlIGNhbiBhcHBseSB0byBhbnkgc3R1ZHksIGUuZy4sIHJhdGluZyBncmFtbWF0aWNhbGl0eSBvZiBzZW50ZW5jZXMsIHJhdGluZyBob3cgcG9zaXRpdmUgdGhlIHNlbnRpbWVudHMgYXJlIGluIGEgYXJ0aWNsZSwgaW50ZXJ2aWV3IHJlc3BvbnNlcywgZXRjLg0KDQojIyBJbXBvcnRpbmcgYW5kIHByZS1wcm9jZXNzaW5nDQoNCldlIHN0YXJ0IGJ5IGltcG9ydGluZyB0aGUgZGF0YSBhbmQgcHJvY2VzcyBpdC4gV2UgY2hhbmdlIHRoZSByZWZlcmVuY2UgbGV2ZWwgaW4gdGhlIHByZWRpY3Rvcg0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCnJhdGluZyA8LSByZWFkLmNzdigicmF0aW5nLmNzdiIpDQpzdHIocmF0aW5nKQ0KIyMgd2UgbmVlZCB0byBjb252ZXJ0ICJSZXNwb25zZSIgdG8gYSBmYWN0b3INCnJhdGluZyRSZXNwb25zZSA8LSBhcy5mYWN0b3IocmF0aW5nJFJlc3BvbnNlKQ0KcmF0aW5nJENvbnRleHQgPC0gcmVsZXZlbChyYXRpbmckQ29udGV4dCwgcmVmPSJpc29sYXRpb24iKQ0KYGBgDQoNCiMjIE91ciBmaXJzdCBtb2RlbA0KDQpXZSBydW4gb3VyIGZpcnN0IGNsbSBtb2RlbCBhcyBhIHNpbXBsZSwgaS5lLiwgd2l0aCBubyByYW5kb20gZWZmZWN0cw0KDQpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCBlcnJvcj1GQUxTRX0NCm1kbC5jbG0gPC0gY2xtKFJlc3BvbnNlIH4gQ29udGV4dCwgZGF0YSA9IHJhdGluZykNCnN1bW1hcnkobWRsLmNsbSkNCmBgYA0KDQoNCiMjIFRlc3Rpbmcgc2lnbmlmaWNhbmNlIA0KDQpXZSBjYW4gZXZhbHVhdGUgd2hldGhlciAiQ29udGV4dCIgaW1wcm92ZXMgdGhlIG1vZGVsIGZpdCwgYnkgY29tcGFyaW5nIGEgbnVsbCBtb2RlbCB3aXRoIG91ciBtb2RlbC4gT2YgY291cnNlICJDb250ZXh0IiBpcyBpbXByb3ZpbmcgdGhlIG1vZGVsIGZpdC4NCg0KYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgZXJyb3I9RkFMU0V9DQptZGwuY2xtLk51bGwgPC0gY2xtKFJlc3BvbnNlIH4gMSwgZGF0YSA9IHJhdGluZykNCmFub3ZhKG1kbC5jbG0sIG1kbC5jbG0uTnVsbCkNCg0KYGBgDQoNCiMjIEludGVycHJldGluZyBhIGN1bXVsYXRpdmUgbW9kZWwNCg0KQXMgYSB3YXkgdG8gaW50ZXJwcmV0IHRoZSBtb2RlbCwgd2UgY2FuIGxvb2sgYXQgdGhlIGNvZWZmaWNpZW50cyBhbmQgbWFrZSBzZW5zZSBvZiB0aGUgcmVzdWx0cy4gQSBDTE0gbW9kZWwgaXMgYSBMb2dpc3RpYyBtb2RlbCB3aXRoIGEgY3VtdWxhdGl2ZSBlZmZlY3QuIFRoZSAiQ29lZmZpY2llbnRzIiBhcmUgdGhlIGVzdGltYXRlcyBmb3IgZWFjaCBsZXZlbCBvZiB0aGUgZml4ZWQgZWZmZWN0OyB0aGUgIlRocmVzaG9sZCBjb2VmZmljaWVudHMiIGFyZSB0aG9zZSBvZiB0aGUgcmVzcG9uc2UuIEZvciB0aGUgZm9ybWVyLCBhIG5lZ2F0aXZlIGNvZWZmaWNpZW50IGluZGljYXRlcyBhIG5lZ2F0aXZlIGFzc29jaWF0aW9uIHdpdGggdGhlIHJlc3BvbnNlOyBhbmQgYSBwb3NpdGl2ZSBpcyBwb3NpdGl2ZWx5IGFzc29jaWF0ZWQgd2l0aCB0aGUgcmVzcG9uc2UuIFRoZSBwIHZhbHVlcyBhcmUgaW5kaWNhdGluZyB0aGUgc2lnbmlmaWNhbmNlIG9mIGVhY2ggbGV2ZWwuIEZvciB0aGUgIlRocmVzaG9sZCBjb2VmZmljaWVudHMiLCB3ZSBjYW4gc2VlIHRoZSBjdW11bGF0aXZlIGVmZmVjdHMgb2YgcmF0aW5ncyAxfDIsIDJ8MywgM3w0IGFuZCA0fDUgd2hpY2ggaW5kaWNhdGUgYW4gb3ZlcmFsbCBpbmNyZWFzZSBpbiB0aGUgcmF0aW5ncyBmcm9tIDEgdG8gNS4gDQoNCiMjIFBsb3R0aW5nIA0KDQpXZSB1c2UgYSBtb2RpZmllZCB2ZXJzaW9uIG9mIGEgcGxvdHRpbmcgZnVuY3Rpb24gdGhhdCBhbGxvd3MgdXMgdG8gdmlzdWFsaXNlIHRoZSBlZmZlY3RzLiBGb3IgdGhpcywgd2UgdXNlIHRoZSBiYXNlIFIgcGxvdHRpbmcgZnVuY3Rpb25zDQoNCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIGVycm9yPUZBTFNFfQ0KcGFyKG9tYT1jKDEsIDAsIDAsIDMpLG1ncD1jKDIsIDEsIDApKQ0KeGxpbU5hcyA9IGMobWluKG1kbC5jbG0kYmV0YSksIG1heChtZGwuY2xtJGJldGEpKQ0KeWxpbU5hcyA9IGMoMCwxKQ0KcGxvdCgwLDAseGxpbT14bGltTmFzLCB5bGltPXlsaW1OYXMsIHR5cGU9Im4iLCB5bGFiPWV4cHJlc3Npb24oUHJvYmFiaWxpdHkpLCB4bGFiPSIiLCB4YXh0ID0gIm4iLG1haW49IlByZWRpY3RlZCBjdXJ2ZXMgLSBOYXNhbGlzYXRpb24iLGNleD0yLGNleC5sYWI9MS41LGNleC5tYWluPTEuNSxjZXguYXhpcz0xLjUpDQpheGlzKHNpZGUgPSAxLCBhdCA9IGMoMCxtZGwuY2xtJGJldGEpLGxhYmVscyA9IGxldmVscyhyYXRpbmckQ29udGV4dCksIGxhcz0yLGNleD0yLGNleC5sYWI9MS41LGNleC5heGlzPTEuNSkNCnhzTmFzID0gc2VxKHhsaW1OYXNbMV0sIHhsaW1OYXNbMl0sIGxlbmd0aC5vdXQ9MTAwKQ0KbGluZXMoeHNOYXMsIHBsb2dpcyhtZGwuY2xtJFRoZXRhWzFdIC0geHNOYXMpLCBjb2w9J2JsYWNrJykNCmxpbmVzKHhzTmFzLCBwbG9naXMobWRsLmNsbSRUaGV0YVsyXSAtIHhzTmFzKS1wbG9naXMobWRsLmNsbSRUaGV0YVsxXSAtIHhzTmFzKSwgY29sPSdyZWQnKQ0KbGluZXMoeHNOYXMsIHBsb2dpcyhtZGwuY2xtJFRoZXRhWzNdIC0geHNOYXMpLXBsb2dpcyhtZGwuY2xtJFRoZXRhWzJdIC0geHNOYXMpLCBjb2w9J2dyZWVuJykNCmxpbmVzKHhzTmFzLCBwbG9naXMobWRsLmNsbSRUaGV0YVs0XSAtIHhzTmFzKS1wbG9naXMobWRsLmNsbSRUaGV0YVszXSAtIHhzTmFzKSwgY29sPSdvcmFuZ2UnKQ0KbGluZXMoeHNOYXMsIDEtKHBsb2dpcyhtZGwuY2xtJFRoZXRhWzRdIC0geHNOYXMpKSwgY29sPSdibHVlJykNCmFibGluZSh2PWMoMCxtZGwuY2xtJGJldGEpLGx0eT0zKQ0KYWJsaW5lKGg9MCwgbHR5PSJkYXNoZWQiKQ0KYWJsaW5lKGg9MSwgbHR5PSJkYXNoZWQiKQ0KbGVnZW5kKHBhcigndXNyJylbMl0sIHBhcigndXNyJylbNF0sIGJ0eT0nbicsIHhwZD1OQSxsdHk9MSwgY29sPWMoImJsYWNrIiwgInJlZCIsICJncmVlbiIsICJvcmFuZ2UiLCAiYmx1ZSIpLCANCiAgICAgICBsZWdlbmQ9YygiT3JhbCIsICIyIiwgIjMiLCAiNCIsICJOYXNhbCIpLGNleD0wLjc1KQ0KDQpgYGANCg0KDQo=