Voicing and Gemination - Random Forests
Jalal Al-Tamimi
Newcastle University17 July 2018
1 Introduction
This analysis accompanies the article “Al-Tamimi, J. and Khattab, G., (2018). Acoustic correlates of the voicing contrast in Lebanese Arabic singleton and geminate plosives. Invited manuscript for the special issue of Journal of Phonetics,”Marking 50 Years of Research on Voice Onset Time and the Voicing Contrast in the World’s Languages" (eds., T. Cho, G. Docherty & D. Whalen). DOI: https://doi.org/10.1016/j.wocn.2018.09.010
This notebook presents the analyses and results of the Random Forests classification. The aim was to evaluate the robustness of the results obtained for each of the 19 acoustic correlates used, (6 durational, three voicing, and 10 non-temporal including f0, F1 and harmonic differences). These acoustic correlates were used to evaluate the relationship between them and the four-way “contrast” observed between voicing and gemination, i.e., Voiced Singleton, Voiceless Singleton, Voiced Geminate and Voiceless Geminate. The previous notebook looked specifically at how each individual acoustic correlate was associated with the predictors. This was done on an individual basis see notebook. This section extends the analysis by looking at the combination of these acoustic correlates and their predictive power in discriminating between the four categories above.
2 Loading required packages
We start by loading the required packages.
requiredPackages = c('dplyr','DMwR','party','pROC','psycho','ggplot2','reshape','foreach','doSNOW','parallel')
for(p in requiredPackages){
if(!require(p,character.only = TRUE)) install.packages(p)
library(p,character.only = TRUE)
}3 Preprocessing the data
3.1 Reading in the data
We start reading in the data and checking the structure. The data contains some missing data. For the Random Forests to work appropriately, we replace the missing values by centrally imputed values.
ResultsFullOriginal <- read.csv("ResultsFullOriginalData.csv")
str(ResultsFullOriginal)'data.frame': 1793 obs. of 36 variables:
$ x : int 1 2 3 4 5 6 7 8 9 10 ...
$ file_name : Factor w/ 68 levels "01_F_W_main_1.wav",..: 1 1 1 1 1 1 1 1 1 1 ...
$ speaker : Factor w/ 20 levels "sp1","sp10","sp11",..: 1 1 1 1 1 1 1 1 1 1 ...
$ sex : Factor w/ 2 levels "F","M": 1 1 1 1 1 1 1 1 1 1 ...
$ Number : int 8 38 68 98 371 395 419 449 743 767 ...
$ wordTarget : Factor w/ 102 levels "2aabbeh","2aabeD",..: 83 9 28 52 47 38 34 90 6 27 ...
$ word : Factor w/ 331 levels "2AAbaD","2aabbeh",..: 269 31 89 160 151 126 112 281 23 88 ...
$ singGem : Factor w/ 2 levels "geminate","singleton": 2 2 2 2 2 2 2 2 2 2 ...
$ syllType : Factor w/ 2 levels "iambic","trochaic": 2 2 2 2 2 2 2 2 2 2 ...
$ vowelLength : Factor w/ 2 levels "longV1","shortV1": 2 2 2 2 2 2 2 2 2 2 ...
$ syllStruct : Factor w/ 5 levels "V1C2V2","V1CC2V2",..: 1 1 1 1 1 1 1 1 1 1 ...
$ c.v : Factor w/ 2 levels "C2","CC2": 1 1 1 1 1 1 1 1 1 1 ...
$ phoneme : Factor w/ 13 levels "b","bb","d","D",..: 1 3 12 8 1 10 12 8 1 10 ...
$ modeArticulation4 : Factor w/ 1 level "stop": 1 1 1 1 1 1 1 1 1 1 ...
$ placeArticulation : Factor w/ 4 levels "alveolar","bilabial",..: 2 1 3 4 2 1 3 4 2 1 ...
$ voiced.unvoiced : Factor w/ 2 levels "voiced","voiceless": 1 1 2 2 1 2 2 2 1 2 ...
$ placeVoicing : Factor w/ 6 levels "alveolar voiced",..: 3 1 5 6 3 2 5 6 3 2 ...
$ durationC2 : num 60.2 87 106.9 109.7 77 ...
$ durationVoicedPercC2 : num 100 57.5 43.9 32.7 67.5 ...
$ durationVOTAll : num 19.2 11.3 21.4 22 16.4 ...
$ VOTDec : num -60.2 -87 21 22 -77 ...
$ durationBurst : num 6 8 7 11 9 11 7 12 9 7 ...
$ durationVOT : num 13 3 14 11 7 18 14 15 14 14 ...
$ durationVoicedPercVOTAll: num 100 0 0 0 41.2 ...
$ intensityOnsetVOTAll : num 52.6 56.9 52.6 55.3 53.4 ...
$ intensityOffsetVOTAll : num 64.5 63.4 59.7 57.7 64.2 ...
$ durationV2 : num 89 155.2 86.3 113.3 76.4 ...
$ f0OnsetV2 : num 247 241 245 239 244 ...
$ intensityOnsetV2 : num 67.3 66.6 61.1 60.9 64.5 ...
$ f1OnsetV2 : num 699 542 659 444 679 ...
$ h1mnh2OnsetNormV2 : num -2.08 5.31 11.84 17.52 13.02 ...
$ durationV1 : num 90.5 48.4 52 63.2 61.5 ...
$ f0OffsetV1 : num 239 239 241 238 205 ...
$ intensityOffsetV1 : num 63.8 70.4 68.7 66.4 65.9 ...
$ f1OffsetV1 : num 573 644 942 661 637 ...
$ h1mnh2OffsetNorm : num 9.123 NA 0.981 3.576 0.861 ...summary(ResultsFullOriginal) x file_name speaker sex
Min. : 1 01_F_W_main_1.wav: 55 sp1 : 97 F:940
1st Qu.: 449 06_F_W_main_1.wav: 55 sp10 : 97 M:853
Median : 897 02_F_W_main_1.wav: 53 sp6 : 97
Mean : 897 10_F_W_main_1.wav: 52 sp17 : 96
3rd Qu.:1345 08_F_W_main_1.wav: 51 sp9 : 96
Max. :1793 01_M_W_main_1.wav: 50 sp20 : 95
(Other) :1477 (Other):1215
Number wordTarget word singGem
Min. : 8 2adar : 20 3atab : 20 geminate :1007
1st Qu.: 9584 3aadal : 20 fatal : 20 singleton: 786
Median :19555 3atab : 20 saba2 : 20
Mean :19497 3attaal: 20 2atal : 19
3rd Qu.:29525 fatal : 20 3addad : 19
Max. :39279 Haba2 : 20 badal : 19
(Other):1673 (Other):1676
syllType vowelLength syllStruct c.v
iambic : 356 longV1 : 594 V1C2V2 :433 C2 : 786
trochaic:1437 shortV1:1199 V1CC2V2 :410 CC2:1007
V1CC2VV2:356
VV1C2V2 :353
VV1CC2V2:241
phoneme modeArticulation4 placeArticulation
dd :285 stop:1793 alveolar :625
d :242 bilabial :426
bb :235 pharyngealised:449
b :191 velar :293
kk :160
k :133
(Other):547
voiced.unvoiced placeVoicing durationC2
voiced :1059 alveolar voiced :420 Min. : 27.09
voiceless: 734 alveolar voiceless :205 1st Qu.: 78.43
bilabial :426 Median :139.81
pharyngealised voiced :213 Mean :131.23
pharyngealised voiceless:236 3rd Qu.:175.43
velar :293 Max. :315.77
durationVoicedPercC2 durationVOTAll VOTDec
Min. : 0.00 Min. : 0.712 Min. :-299.69
1st Qu.: 23.08 1st Qu.:10.478 1st Qu.:-142.10
Median : 69.12 Median :15.796 Median : -62.73
Mean : 61.60 Mean :18.600 Mean : -61.12
3rd Qu.:100.00 3rd Qu.:23.386 3rd Qu.: 19.00
Max. :100.00 Max. :89.065 Max. : 80.00
durationBurst durationVOT durationVoicedPercVOTAll
Min. : 0.500 Min. : 0.10 Min. : 0.00
1st Qu.: 4.000 1st Qu.: 5.00 1st Qu.: 0.00
Median : 6.000 Median : 9.00 Median : 0.00
Mean : 7.236 Mean :11.94 Mean : 41.02
3rd Qu.: 9.000 3rd Qu.:15.00 3rd Qu.:100.00
Max. :38.000 Max. :97.00 Max. :100.00
NA's :62 NA's :62
intensityOnsetVOTAll intensityOffsetVOTAll durationV2
Min. :41.11 Min. :42.47 Min. : 37.52
1st Qu.:54.91 1st Qu.:62.46 1st Qu.:113.30
Median :58.95 Median :65.72 Median :143.69
Mean :59.42 Mean :65.37 Mean :156.24
3rd Qu.:64.08 3rd Qu.:68.80 3rd Qu.:193.76
Max. :77.19 Max. :82.02 Max. :381.26
f0OnsetV2 intensityOnsetV2 f1OnsetV2
Min. : 80.93 Min. :45.35 Min. : 209.9
1st Qu.:116.99 1st Qu.:64.50 1st Qu.: 431.2
Median :162.49 Median :68.27 Median : 485.1
Mean :162.64 Mean :67.80 Mean : 499.6
3rd Qu.:199.27 3rd Qu.:71.83 3rd Qu.: 555.4
Max. :308.96 Max. :84.14 Max. :1082.2
h1mnh2OnsetNormV2 durationV1 f0OffsetV1
Min. :-23.341 Min. : 4.31 Min. : 82.23
1st Qu.: -7.210 1st Qu.: 57.45 1st Qu.:119.48
Median : -4.097 Median : 78.51 Median :172.18
Mean : -2.665 Mean : 94.63 Mean :171.86
3rd Qu.: 0.814 3rd Qu.:131.74 3rd Qu.:215.80
Max. : 22.130 Max. :264.37 Max. :367.78
NA's :10
intensityOffsetV1 f1OffsetV1 h1mnh2OffsetNorm
Min. :48.25 Min. : 246.6 Min. :-13.019
1st Qu.:65.71 1st Qu.: 470.5 1st Qu.: -6.394
Median :70.19 Median : 534.2 Median : -3.136
Mean :69.53 Mean : 553.6 Mean : -2.269
3rd Qu.:73.82 3rd Qu.: 622.4 3rd Qu.: 1.192
Max. :84.98 Max. :1081.3 Max. : 19.413
NA's :226 3.2 Central imputations
As can be seen from the summary above, the data contains some missing data. For the Random Forests to work appropriately, we replace the missing values by centrally imputed ones
# Central imputation using the package "DMwR".
ResultsFullOriginalCentral <- ResultsFullOriginal # create copy
ResultsFullOriginalCentral[, 18:36] <- centralImputation(ResultsFullOriginalCentral[, 18:36])
str(ResultsFullOriginalCentral)'data.frame': 1793 obs. of 36 variables:
$ x : int 1 2 3 4 5 6 7 8 9 10 ...
$ file_name : Factor w/ 68 levels "01_F_W_main_1.wav",..: 1 1 1 1 1 1 1 1 1 1 ...
$ speaker : Factor w/ 20 levels "sp1","sp10","sp11",..: 1 1 1 1 1 1 1 1 1 1 ...
$ sex : Factor w/ 2 levels "F","M": 1 1 1 1 1 1 1 1 1 1 ...
$ Number : int 8 38 68 98 371 395 419 449 743 767 ...
$ wordTarget : Factor w/ 102 levels "2aabbeh","2aabeD",..: 83 9 28 52 47 38 34 90 6 27 ...
$ word : Factor w/ 331 levels "2AAbaD","2aabbeh",..: 269 31 89 160 151 126 112 281 23 88 ...
$ singGem : Factor w/ 2 levels "geminate","singleton": 2 2 2 2 2 2 2 2 2 2 ...
$ syllType : Factor w/ 2 levels "iambic","trochaic": 2 2 2 2 2 2 2 2 2 2 ...
$ vowelLength : Factor w/ 2 levels "longV1","shortV1": 2 2 2 2 2 2 2 2 2 2 ...
$ syllStruct : Factor w/ 5 levels "V1C2V2","V1CC2V2",..: 1 1 1 1 1 1 1 1 1 1 ...
$ c.v : Factor w/ 2 levels "C2","CC2": 1 1 1 1 1 1 1 1 1 1 ...
$ phoneme : Factor w/ 13 levels "b","bb","d","D",..: 1 3 12 8 1 10 12 8 1 10 ...
$ modeArticulation4 : Factor w/ 1 level "stop": 1 1 1 1 1 1 1 1 1 1 ...
$ placeArticulation : Factor w/ 4 levels "alveolar","bilabial",..: 2 1 3 4 2 1 3 4 2 1 ...
$ voiced.unvoiced : Factor w/ 2 levels "voiced","voiceless": 1 1 2 2 1 2 2 2 1 2 ...
$ placeVoicing : Factor w/ 6 levels "alveolar voiced",..: 3 1 5 6 3 2 5 6 3 2 ...
$ durationC2 : num 60.2 87 106.9 109.7 77 ...
$ durationVoicedPercC2 : num 100 57.5 43.9 32.7 67.5 ...
$ durationVOTAll : num 19.2 11.3 21.4 22 16.4 ...
$ VOTDec : num -60.2 -87 21 22 -77 ...
$ durationBurst : num 6 8 7 11 9 11 7 12 9 7 ...
$ durationVOT : num 13 3 14 11 7 18 14 15 14 14 ...
$ durationVoicedPercVOTAll: num 100 0 0 0 41.2 ...
$ intensityOnsetVOTAll : num 52.6 56.9 52.6 55.3 53.4 ...
$ intensityOffsetVOTAll : num 64.5 63.4 59.7 57.7 64.2 ...
$ durationV2 : num 89 155.2 86.3 113.3 76.4 ...
$ f0OnsetV2 : num 247 241 245 239 244 ...
$ intensityOnsetV2 : num 67.3 66.6 61.1 60.9 64.5 ...
$ f1OnsetV2 : num 699 542 659 444 679 ...
$ h1mnh2OnsetNormV2 : num -2.08 5.31 11.84 17.52 13.02 ...
$ durationV1 : num 90.5 48.4 52 63.2 61.5 ...
$ f0OffsetV1 : num 239 239 241 238 205 ...
$ intensityOffsetV1 : num 63.8 70.4 68.7 66.4 65.9 ...
$ f1OffsetV1 : num 573 644 942 661 637 ...
$ h1mnh2OffsetNorm : num 9.123 -3.136 0.981 3.576 0.861 ...summary(ResultsFullOriginalCentral) x file_name speaker sex
Min. : 1 01_F_W_main_1.wav: 55 sp1 : 97 F:940
1st Qu.: 449 06_F_W_main_1.wav: 55 sp10 : 97 M:853
Median : 897 02_F_W_main_1.wav: 53 sp6 : 97
Mean : 897 10_F_W_main_1.wav: 52 sp17 : 96
3rd Qu.:1345 08_F_W_main_1.wav: 51 sp9 : 96
Max. :1793 01_M_W_main_1.wav: 50 sp20 : 95
(Other) :1477 (Other):1215
Number wordTarget word singGem
Min. : 8 2adar : 20 3atab : 20 geminate :1007
1st Qu.: 9584 3aadal : 20 fatal : 20 singleton: 786
Median :19555 3atab : 20 saba2 : 20
Mean :19497 3attaal: 20 2atal : 19
3rd Qu.:29525 fatal : 20 3addad : 19
Max. :39279 Haba2 : 20 badal : 19
(Other):1673 (Other):1676
syllType vowelLength syllStruct c.v
iambic : 356 longV1 : 594 V1C2V2 :433 C2 : 786
trochaic:1437 shortV1:1199 V1CC2V2 :410 CC2:1007
V1CC2VV2:356
VV1C2V2 :353
VV1CC2V2:241
phoneme modeArticulation4 placeArticulation
dd :285 stop:1793 alveolar :625
d :242 bilabial :426
bb :235 pharyngealised:449
b :191 velar :293
kk :160
k :133
(Other):547
voiced.unvoiced placeVoicing durationC2
voiced :1059 alveolar voiced :420 Min. : 27.09
voiceless: 734 alveolar voiceless :205 1st Qu.: 78.43
bilabial :426 Median :139.81
pharyngealised voiced :213 Mean :131.23
pharyngealised voiceless:236 3rd Qu.:175.43
velar :293 Max. :315.77
durationVoicedPercC2 durationVOTAll VOTDec
Min. : 0.00 Min. : 0.712 Min. :-299.69
1st Qu.: 23.08 1st Qu.:10.478 1st Qu.:-142.10
Median : 69.12 Median :15.796 Median : -62.73
Mean : 61.60 Mean :18.600 Mean : -61.12
3rd Qu.:100.00 3rd Qu.:23.386 3rd Qu.: 19.00
Max. :100.00 Max. :89.065 Max. : 80.00
durationBurst durationVOT durationVoicedPercVOTAll
Min. : 0.500 Min. : 0.10 Min. : 0.00
1st Qu.: 4.000 1st Qu.: 5.00 1st Qu.: 0.00
Median : 6.000 Median : 9.00 Median : 0.00
Mean : 7.193 Mean :11.84 Mean : 41.02
3rd Qu.: 9.000 3rd Qu.:15.00 3rd Qu.:100.00
Max. :38.000 Max. :97.00 Max. :100.00
intensityOnsetVOTAll intensityOffsetVOTAll durationV2
Min. :41.11 Min. :42.47 Min. : 37.52
1st Qu.:54.91 1st Qu.:62.46 1st Qu.:113.30
Median :58.95 Median :65.72 Median :143.69
Mean :59.42 Mean :65.37 Mean :156.24
3rd Qu.:64.08 3rd Qu.:68.80 3rd Qu.:193.76
Max. :77.19 Max. :82.02 Max. :381.26
f0OnsetV2 intensityOnsetV2 f1OnsetV2
Min. : 80.93 Min. :45.35 Min. : 209.9
1st Qu.:116.99 1st Qu.:64.50 1st Qu.: 431.2
Median :162.49 Median :68.27 Median : 485.1
Mean :162.64 Mean :67.80 Mean : 499.6
3rd Qu.:199.27 3rd Qu.:71.83 3rd Qu.: 555.4
Max. :308.96 Max. :84.14 Max. :1082.2
h1mnh2OnsetNormV2 durationV1 f0OffsetV1
Min. :-23.341 Min. : 4.31 Min. : 82.23
1st Qu.: -7.177 1st Qu.: 57.45 1st Qu.:119.48
Median : -4.097 Median : 78.51 Median :172.18
Mean : -2.673 Mean : 94.63 Mean :171.86
3rd Qu.: 0.769 3rd Qu.:131.74 3rd Qu.:215.80
Max. : 22.130 Max. :264.37 Max. :367.78
intensityOffsetV1 f1OffsetV1 h1mnh2OffsetNorm
Min. :48.25 Min. : 246.6 Min. :-13.019
1st Qu.:65.71 1st Qu.: 470.5 1st Qu.: -5.900
Median :70.19 Median : 534.2 Median : -3.136
Mean :69.53 Mean : 553.6 Mean : -2.378
3rd Qu.:73.82 3rd Qu.: 622.4 3rd Qu.: 0.399
Max. :84.98 Max. :1081.3 Max. : 19.413
write.csv(ResultsFullOriginalCentral,"ResultsFullOriginalCentral.csv")3.3 Z-Scoring the data
Given the differences in scales of each of the acoustic correlates, e.g., durations in milliseconds, intensity in dB, the magnitudes of the values are different. As a way to “normalise” for the data and to put all predictors on the same level, we z-score the data.
# from the package "dplyr"
all_z <- select(ResultsFullOriginalCentral, durationC2:h1mnh2OffsetNorm)
all_z <- cbind(all_z, select(ResultsFullOriginalCentral))
all_z <- apply(all_z, 2, scale)
colnames(all_z) <- paste(colnames(all_z), 'z', sep = '_')
ResultsFullOriginalCentral <- cbind(ResultsFullOriginalCentral, all_z)
summary(ResultsFullOriginalCentral) x file_name speaker sex
Min. : 1 01_F_W_main_1.wav: 55 sp1 : 97 F:940
1st Qu.: 449 06_F_W_main_1.wav: 55 sp10 : 97 M:853
Median : 897 02_F_W_main_1.wav: 53 sp6 : 97
Mean : 897 10_F_W_main_1.wav: 52 sp17 : 96
3rd Qu.:1345 08_F_W_main_1.wav: 51 sp9 : 96
Max. :1793 01_M_W_main_1.wav: 50 sp20 : 95
(Other) :1477 (Other):1215
Number wordTarget word singGem
Min. : 8 2adar : 20 3atab : 20 geminate :1007
1st Qu.: 9584 3aadal : 20 fatal : 20 singleton: 786
Median :19555 3atab : 20 saba2 : 20
Mean :19497 3attaal: 20 2atal : 19
3rd Qu.:29525 fatal : 20 3addad : 19
Max. :39279 Haba2 : 20 badal : 19
(Other):1673 (Other):1676
syllType vowelLength syllStruct c.v
iambic : 356 longV1 : 594 V1C2V2 :433 C2 : 786
trochaic:1437 shortV1:1199 V1CC2V2 :410 CC2:1007
V1CC2VV2:356
VV1C2V2 :353
VV1CC2V2:241
phoneme modeArticulation4 placeArticulation
dd :285 stop:1793 alveolar :625
d :242 bilabial :426
bb :235 pharyngealised:449
b :191 velar :293
kk :160
k :133
(Other):547
voiced.unvoiced placeVoicing durationC2
voiced :1059 alveolar voiced :420 Min. : 27.09
voiceless: 734 alveolar voiceless :205 1st Qu.: 78.43
bilabial :426 Median :139.81
pharyngealised voiced :213 Mean :131.23
pharyngealised voiceless:236 3rd Qu.:175.43
velar :293 Max. :315.77
durationVoicedPercC2 durationVOTAll VOTDec
Min. : 0.00 Min. : 0.712 Min. :-299.69
1st Qu.: 23.08 1st Qu.:10.478 1st Qu.:-142.10
Median : 69.12 Median :15.796 Median : -62.73
Mean : 61.60 Mean :18.600 Mean : -61.12
3rd Qu.:100.00 3rd Qu.:23.386 3rd Qu.: 19.00
Max. :100.00 Max. :89.065 Max. : 80.00
durationBurst durationVOT durationVoicedPercVOTAll
Min. : 0.500 Min. : 0.10 Min. : 0.00
1st Qu.: 4.000 1st Qu.: 5.00 1st Qu.: 0.00
Median : 6.000 Median : 9.00 Median : 0.00
Mean : 7.193 Mean :11.84 Mean : 41.02
3rd Qu.: 9.000 3rd Qu.:15.00 3rd Qu.:100.00
Max. :38.000 Max. :97.00 Max. :100.00
intensityOnsetVOTAll intensityOffsetVOTAll durationV2
Min. :41.11 Min. :42.47 Min. : 37.52
1st Qu.:54.91 1st Qu.:62.46 1st Qu.:113.30
Median :58.95 Median :65.72 Median :143.69
Mean :59.42 Mean :65.37 Mean :156.24
3rd Qu.:64.08 3rd Qu.:68.80 3rd Qu.:193.76
Max. :77.19 Max. :82.02 Max. :381.26
f0OnsetV2 intensityOnsetV2 f1OnsetV2
Min. : 80.93 Min. :45.35 Min. : 209.9
1st Qu.:116.99 1st Qu.:64.50 1st Qu.: 431.2
Median :162.49 Median :68.27 Median : 485.1
Mean :162.64 Mean :67.80 Mean : 499.6
3rd Qu.:199.27 3rd Qu.:71.83 3rd Qu.: 555.4
Max. :308.96 Max. :84.14 Max. :1082.2
h1mnh2OnsetNormV2 durationV1 f0OffsetV1
Min. :-23.341 Min. : 4.31 Min. : 82.23
1st Qu.: -7.177 1st Qu.: 57.45 1st Qu.:119.48
Median : -4.097 Median : 78.51 Median :172.18
Mean : -2.673 Mean : 94.63 Mean :171.86
3rd Qu.: 0.769 3rd Qu.:131.74 3rd Qu.:215.80
Max. : 22.130 Max. :264.37 Max. :367.78
intensityOffsetV1 f1OffsetV1 h1mnh2OffsetNorm
Min. :48.25 Min. : 246.6 Min. :-13.019
1st Qu.:65.71 1st Qu.: 470.5 1st Qu.: -5.900
Median :70.19 Median : 534.2 Median : -3.136
Mean :69.53 Mean : 553.6 Mean : -2.378
3rd Qu.:73.82 3rd Qu.: 622.4 3rd Qu.: 0.399
Max. :84.98 Max. :1081.3 Max. : 19.413
durationC2_z durationVoicedPercC2_z durationVOTAll_z
Min. :-1.8939 Min. :-1.6510 Min. :-1.5024
1st Qu.:-0.9603 1st Qu.:-1.0325 1st Qu.:-0.6822
Median : 0.1561 Median : 0.2014 Median :-0.2355
Mean : 0.0000 Mean : 0.0000 Mean : 0.0000
3rd Qu.: 0.8039 3rd Qu.: 1.0291 3rd Qu.: 0.4020
Max. : 3.3564 Max. : 1.0291 Max. : 5.9185
VOTDec_z durationBurst_z durationVOT_z
Min. :-2.88576 Min. :-1.5989 Min. :-1.1337
1st Qu.:-0.97961 1st Qu.:-0.7628 1st Qu.:-0.6605
Median :-0.01956 Median :-0.2850 Median :-0.2741
Mean : 0.00000 Mean : 0.0000 Mean : 0.0000
3rd Qu.: 0.96913 3rd Qu.: 0.4316 3rd Qu.: 0.3053
Max. : 1.70700 Max. : 7.3590 Max. : 8.2247
durationVoicedPercVOTAll_z intensityOnsetVOTAll_z
Min. :-0.867 Min. :-2.94874
1st Qu.:-0.867 1st Qu.:-0.72620
Median :-0.867 Median :-0.07552
Mean : 0.000 Mean : 0.00000
3rd Qu.: 1.247 3rd Qu.: 0.74937
Max. : 1.247 Max. : 2.86081
intensityOffsetVOTAll_z durationV2_z f0OnsetV2_z
Min. :-4.21154 Min. :-2.0785 Min. :-1.693791
1st Qu.:-0.53597 1st Qu.:-0.7517 1st Qu.:-0.946329
Median : 0.06384 Median :-0.2198 Median :-0.003211
Mean : 0.00000 Mean : 0.0000 Mean : 0.000000
3rd Qu.: 0.63035 3rd Qu.: 0.6571 3rd Qu.: 0.759217
Max. : 3.06307 Max. : 3.9397 Max. : 3.032946
intensityOnsetV2_z f1OnsetV2_z h1mnh2OnsetNormV2_z
Min. :-3.83386 Min. :-2.9352 Min. :-3.2122
1st Qu.:-0.56327 1st Qu.:-0.6926 1st Qu.:-0.7001
Median : 0.08125 Median :-0.1464 Median :-0.2214
Mean : 0.00000 Mean : 0.0000 Mean : 0.0000
3rd Qu.: 0.68836 3rd Qu.: 0.5662 3rd Qu.: 0.5349
Max. : 2.79068 Max. : 5.9039 Max. : 3.8548
durationV1_z f0OffsetV1_z intensityOffsetV1_z
Min. :-1.8270 Min. :-1.642154 Min. :-3.5036
1st Qu.:-0.7520 1st Qu.:-0.959636 1st Qu.:-0.6293
Median :-0.3262 Median : 0.005841 Median : 0.1083
Mean : 0.0000 Mean : 0.000000 Mean : 0.0000
3rd Qu.: 0.7504 3rd Qu.: 0.805067 3rd Qu.: 0.7061
Max. : 3.4333 Max. : 3.589498 Max. : 2.5419
f1OffsetV1_z h1mnh2OffsetNorm_z
Min. :-2.5378 Min. :-2.0611
1st Qu.:-0.6871 1st Qu.:-0.6822
Median :-0.1604 Median :-0.1468
Mean : 0.0000 Mean : 0.0000
3rd Qu.: 0.5685 3rd Qu.: 0.5379
Max. : 4.3619 Max. : 4.2208
str(ResultsFullOriginalCentral)'data.frame': 1793 obs. of 55 variables:
$ x : int 1 2 3 4 5 6 7 8 9 10 ...
$ file_name : Factor w/ 68 levels "01_F_W_main_1.wav",..: 1 1 1 1 1 1 1 1 1 1 ...
$ speaker : Factor w/ 20 levels "sp1","sp10","sp11",..: 1 1 1 1 1 1 1 1 1 1 ...
$ sex : Factor w/ 2 levels "F","M": 1 1 1 1 1 1 1 1 1 1 ...
$ Number : int 8 38 68 98 371 395 419 449 743 767 ...
$ wordTarget : Factor w/ 102 levels "2aabbeh","2aabeD",..: 83 9 28 52 47 38 34 90 6 27 ...
$ word : Factor w/ 331 levels "2AAbaD","2aabbeh",..: 269 31 89 160 151 126 112 281 23 88 ...
$ singGem : Factor w/ 2 levels "geminate","singleton": 2 2 2 2 2 2 2 2 2 2 ...
$ syllType : Factor w/ 2 levels "iambic","trochaic": 2 2 2 2 2 2 2 2 2 2 ...
$ vowelLength : Factor w/ 2 levels "longV1","shortV1": 2 2 2 2 2 2 2 2 2 2 ...
$ syllStruct : Factor w/ 5 levels "V1C2V2","V1CC2V2",..: 1 1 1 1 1 1 1 1 1 1 ...
$ c.v : Factor w/ 2 levels "C2","CC2": 1 1 1 1 1 1 1 1 1 1 ...
$ phoneme : Factor w/ 13 levels "b","bb","d","D",..: 1 3 12 8 1 10 12 8 1 10 ...
$ modeArticulation4 : Factor w/ 1 level "stop": 1 1 1 1 1 1 1 1 1 1 ...
$ placeArticulation : Factor w/ 4 levels "alveolar","bilabial",..: 2 1 3 4 2 1 3 4 2 1 ...
$ voiced.unvoiced : Factor w/ 2 levels "voiced","voiceless": 1 1 2 2 1 2 2 2 1 2 ...
$ placeVoicing : Factor w/ 6 levels "alveolar voiced",..: 3 1 5 6 3 2 5 6 3 2 ...
$ durationC2 : num 60.2 87 106.9 109.7 77 ...
$ durationVoicedPercC2 : num 100 57.5 43.9 32.7 67.5 ...
$ durationVOTAll : num 19.2 11.3 21.4 22 16.4 ...
$ VOTDec : num -60.2 -87 21 22 -77 ...
$ durationBurst : num 6 8 7 11 9 11 7 12 9 7 ...
$ durationVOT : num 13 3 14 11 7 18 14 15 14 14 ...
$ durationVoicedPercVOTAll : num 100 0 0 0 41.2 ...
$ intensityOnsetVOTAll : num 52.6 56.9 52.6 55.3 53.4 ...
$ intensityOffsetVOTAll : num 64.5 63.4 59.7 57.7 64.2 ...
$ durationV2 : num 89 155.2 86.3 113.3 76.4 ...
$ f0OnsetV2 : num 247 241 245 239 244 ...
$ intensityOnsetV2 : num 67.3 66.6 61.1 60.9 64.5 ...
$ f1OnsetV2 : num 699 542 659 444 679 ...
$ h1mnh2OnsetNormV2 : num -2.08 5.31 11.84 17.52 13.02 ...
$ durationV1 : num 90.5 48.4 52 63.2 61.5 ...
$ f0OffsetV1 : num 239 239 241 238 205 ...
$ intensityOffsetV1 : num 63.8 70.4 68.7 66.4 65.9 ...
$ f1OffsetV1 : num 573 644 942 661 637 ...
$ h1mnh2OffsetNorm : num 9.123 -3.136 0.981 3.576 0.861 ...
$ durationC2_z : num -1.292 -0.804 -0.442 -0.391 -0.986 ...
$ durationVoicedPercC2_z : num 1.029 -0.111 -0.474 -0.774 0.159 ...
$ durationVOTAll_z : num 0.0523 -0.6124 0.239 0.2864 -0.1851 ...
$ VOTDec_z : num 0.0112 -0.3133 0.9933 1.0054 -0.1921 ...
$ durationBurst_z : num -0.285 0.1927 -0.0462 0.9093 0.4316 ...
$ durationVOT_z : num 0.112 -0.854 0.209 -0.081 -0.467 ...
$ durationVoicedPercVOTAll_z: num 1.24656 -0.86697 -0.86697 -0.86697 0.00331 ...
$ intensityOnsetVOTAll_z : num -1.107 -0.402 -1.095 -0.669 -0.974 ...
$ intensityOffsetVOTAll_z : num -0.165 -0.363 -1.046 -1.402 -0.219 ...
$ durationV2_z : num -1.1771 -0.0176 -1.2242 -0.7517 -1.3971 ...
$ f0OnsetV2_z : num 1.75 1.63 1.71 1.59 1.69 ...
$ intensityOnsetV2_z : num -0.0892 -0.2073 -1.1401 -1.1719 -0.5585 ...
$ f1OnsetV2_z : num 2.024 0.429 1.617 -0.559 1.819 ...
$ h1mnh2OnsetNormV2_z : num 0.0915 1.2402 2.2558 3.1377 2.4385 ...
$ durationV1_z : num -0.0843 -0.9348 -0.8631 -0.6355 -0.6697 ...
$ f0OffsetV1_z : num 1.223 1.233 1.258 1.22 0.614 ...
$ intensityOffsetV1_z : num -0.947 0.139 -0.145 -0.516 -0.606 ...
$ f1OffsetV1_z : num 0.16 0.75 3.21 0.886 0.693 ...
$ h1mnh2OffsetNorm_z : num 2.228 -0.147 0.651 1.153 0.627 ...3.4 Creating a new outcome
In the following analysis, we are interested in the four-way contrast observed between voicing and gemination, i.e., Voiceless Singleton, Voiced Singleton, Voiceless Geminate and Voiced Geminate.
ResultsFullOriginalCentral["SingGemVoicing"] <- NA
ResultsFullOriginalCentral["SingGemVoicing"] <- paste(ResultsFullOriginalCentral$singGem,
ResultsFullOriginalCentral$voiced.unvoiced)
ResultsFullOriginalCentral$SingGemVoicing <- as.factor(ResultsFullOriginalCentral$SingGemVoicing)
str(ResultsFullOriginalCentral)'data.frame': 1793 obs. of 56 variables:
$ x : int 1 2 3 4 5 6 7 8 9 10 ...
$ file_name : Factor w/ 68 levels "01_F_W_main_1.wav",..: 1 1 1 1 1 1 1 1 1 1 ...
$ speaker : Factor w/ 20 levels "sp1","sp10","sp11",..: 1 1 1 1 1 1 1 1 1 1 ...
$ sex : Factor w/ 2 levels "F","M": 1 1 1 1 1 1 1 1 1 1 ...
$ Number : int 8 38 68 98 371 395 419 449 743 767 ...
$ wordTarget : Factor w/ 102 levels "2aabbeh","2aabeD",..: 83 9 28 52 47 38 34 90 6 27 ...
$ word : Factor w/ 331 levels "2AAbaD","2aabbeh",..: 269 31 89 160 151 126 112 281 23 88 ...
$ singGem : Factor w/ 2 levels "geminate","singleton": 2 2 2 2 2 2 2 2 2 2 ...
$ syllType : Factor w/ 2 levels "iambic","trochaic": 2 2 2 2 2 2 2 2 2 2 ...
$ vowelLength : Factor w/ 2 levels "longV1","shortV1": 2 2 2 2 2 2 2 2 2 2 ...
$ syllStruct : Factor w/ 5 levels "V1C2V2","V1CC2V2",..: 1 1 1 1 1 1 1 1 1 1 ...
$ c.v : Factor w/ 2 levels "C2","CC2": 1 1 1 1 1 1 1 1 1 1 ...
$ phoneme : Factor w/ 13 levels "b","bb","d","D",..: 1 3 12 8 1 10 12 8 1 10 ...
$ modeArticulation4 : Factor w/ 1 level "stop": 1 1 1 1 1 1 1 1 1 1 ...
$ placeArticulation : Factor w/ 4 levels "alveolar","bilabial",..: 2 1 3 4 2 1 3 4 2 1 ...
$ voiced.unvoiced : Factor w/ 2 levels "voiced","voiceless": 1 1 2 2 1 2 2 2 1 2 ...
$ placeVoicing : Factor w/ 6 levels "alveolar voiced",..: 3 1 5 6 3 2 5 6 3 2 ...
$ durationC2 : num 60.2 87 106.9 109.7 77 ...
$ durationVoicedPercC2 : num 100 57.5 43.9 32.7 67.5 ...
$ durationVOTAll : num 19.2 11.3 21.4 22 16.4 ...
$ VOTDec : num -60.2 -87 21 22 -77 ...
$ durationBurst : num 6 8 7 11 9 11 7 12 9 7 ...
$ durationVOT : num 13 3 14 11 7 18 14 15 14 14 ...
$ durationVoicedPercVOTAll : num 100 0 0 0 41.2 ...
$ intensityOnsetVOTAll : num 52.6 56.9 52.6 55.3 53.4 ...
$ intensityOffsetVOTAll : num 64.5 63.4 59.7 57.7 64.2 ...
$ durationV2 : num 89 155.2 86.3 113.3 76.4 ...
$ f0OnsetV2 : num 247 241 245 239 244 ...
$ intensityOnsetV2 : num 67.3 66.6 61.1 60.9 64.5 ...
$ f1OnsetV2 : num 699 542 659 444 679 ...
$ h1mnh2OnsetNormV2 : num -2.08 5.31 11.84 17.52 13.02 ...
$ durationV1 : num 90.5 48.4 52 63.2 61.5 ...
$ f0OffsetV1 : num 239 239 241 238 205 ...
$ intensityOffsetV1 : num 63.8 70.4 68.7 66.4 65.9 ...
$ f1OffsetV1 : num 573 644 942 661 637 ...
$ h1mnh2OffsetNorm : num 9.123 -3.136 0.981 3.576 0.861 ...
$ durationC2_z : num -1.292 -0.804 -0.442 -0.391 -0.986 ...
$ durationVoicedPercC2_z : num 1.029 -0.111 -0.474 -0.774 0.159 ...
$ durationVOTAll_z : num 0.0523 -0.6124 0.239 0.2864 -0.1851 ...
$ VOTDec_z : num 0.0112 -0.3133 0.9933 1.0054 -0.1921 ...
$ durationBurst_z : num -0.285 0.1927 -0.0462 0.9093 0.4316 ...
$ durationVOT_z : num 0.112 -0.854 0.209 -0.081 -0.467 ...
$ durationVoicedPercVOTAll_z: num 1.24656 -0.86697 -0.86697 -0.86697 0.00331 ...
$ intensityOnsetVOTAll_z : num -1.107 -0.402 -1.095 -0.669 -0.974 ...
$ intensityOffsetVOTAll_z : num -0.165 -0.363 -1.046 -1.402 -0.219 ...
$ durationV2_z : num -1.1771 -0.0176 -1.2242 -0.7517 -1.3971 ...
$ f0OnsetV2_z : num 1.75 1.63 1.71 1.59 1.69 ...
$ intensityOnsetV2_z : num -0.0892 -0.2073 -1.1401 -1.1719 -0.5585 ...
$ f1OnsetV2_z : num 2.024 0.429 1.617 -0.559 1.819 ...
$ h1mnh2OnsetNormV2_z : num 0.0915 1.2402 2.2558 3.1377 2.4385 ...
$ durationV1_z : num -0.0843 -0.9348 -0.8631 -0.6355 -0.6697 ...
$ f0OffsetV1_z : num 1.223 1.233 1.258 1.22 0.614 ...
$ intensityOffsetV1_z : num -0.947 0.139 -0.145 -0.516 -0.606 ...
$ f1OffsetV1_z : num 0.16 0.75 3.21 0.886 0.693 ...
$ h1mnh2OffsetNorm_z : num 2.228 -0.147 0.651 1.153 0.627 ...
$ SingGemVoicing : Factor w/ 4 levels "geminate voiced",..: 3 3 4 4 3 4 4 4 3 4 ...After all this pre-processing, we are now ready to start our Random Forest analysis.
4 Tuning our Random Forests
4.1 Creating training and testing sets
We create a training and a testing sets (66% and 33% of the data respectively). The aim is to train three random forests and using the testing set, we are evaluating the prediction power of each.
# creating training and testing sets
# we set the seeds for reproducibility
set.seed(1)
train.idx <- sample(nrow(ResultsFullOriginalCentral), 2/3 * nrow(ResultsFullOriginalCentral))
gemination.train <- ResultsFullOriginalCentral[train.idx, ]
gemination.test <- ResultsFullOriginalCentral[-train.idx, ]4.2 Estimating number of trees required
We use a procedure we developed to obtain the optimal number of trees required to grow a forest with the highest predictive accuracy. This implements the procedure developed by “Oshiro, T. M., Perez, P. S., & Baranauskas, J. A. (2012). How many trees in a random forest? In International Workshop on Machine Learning and Data Mining in Pattern Recognition (pp. 154–168)” (see my github repo for the script).
The code below runs the analysis using parallel computing. We define the number of clusters needed and use these. We are training three Random Forests (see paper for more details):
Model A: A forest with the full 19 predictors (AUCTestWithDur)
Model B: A forest with 18 predictors (Model A minus the closure duration (AUCTestNoDur))
Model C: A forest with 17 predictors (Model B minus the VOT (AUCTestNoDurNoVOT))
## below to prepare using parallel computing
NumberOfCluster <- detectCores()
cl <- makeCluster(NumberOfCluster)
registerDoSNOW(cl)
## below computes the ROC (Receiver Operating Curve) curves for this dataframe
## for the 15 random forests with a 100 trees iteration
## and saves these in a list that will be used later to compare ROC curves
treeNumb <- 15
system.time(AUCTestWithDur <-
foreach (i=1:treeNumb,.packages=c("party","pROC")) %dopar% {
set.seed(1)
cforest.mdl=cforest(SingGemVoicing~durationC2_z+durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z, data = gemination.train,
controls = cforest_unbiased(mtry = round(sqrt(19)), ntree = i*100))
cforest.cond = predict(cforest.mdl,OOB=TRUE)
roc <- roc(gemination.train$SingGemVoicing,as.numeric(cforest.cond))
}) user system elapsed
0.03 0.03 54.53 system.time(AUCTestNoDur <-
foreach (i=1:treeNumb,.packages=c("party","pROC")) %dopar% {
set.seed(1)
cforest.mdl=cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z, data = gemination.train,
controls = cforest_unbiased(mtry = round(sqrt(18)), ntree = i*100))
cforest.cond = predict(cforest.mdl,OOB=TRUE)
roc <- roc(gemination.train$SingGemVoicing,as.numeric(cforest.cond))
}) user system elapsed
0.03 0.01 52.97 system.time(AUCTestNoDurNoVOTDec <-
foreach (i=1:treeNumb,.packages=c("party","pROC")) %dopar% {
set.seed(1)
cforest.mdl=cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z, data = gemination.train,
controls = cforest_unbiased(mtry = round(sqrt(17)), ntree = i*100))
cforest.cond = predict(cforest.mdl,OOB=TRUE)
roc <- roc(gemination.train$SingGemVoicing,as.numeric(cforest.cond))
}) user system elapsed
0.08 0.02 62.23 # stop cluster. If not done R will still use all cores as defined above and the system will become too slow!!
stopCluster(cl)
#4.2.1 Significance testing
4.2.1.1 Model A, 19 predictors
# check significant ROC comparison
# with duration
roc.test(AUCTestWithDur[[1]], AUCTestWithDur[[2]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[1]] and AUCTestWithDur[[2]]
Z = -0.044298, p-value = 0.9647
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9059677 0.9061450 roc.test(AUCTestWithDur[[2]], AUCTestWithDur[[3]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[2]] and AUCTestWithDur[[3]]
Z = -0.75022, p-value = 0.4531
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.906145 0.908296 roc.test(AUCTestWithDur[[3]], AUCTestWithDur[[4]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[3]] and AUCTestWithDur[[4]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.908296 0.908296 roc.test(AUCTestWithDur[[4]], AUCTestWithDur[[5]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[4]] and AUCTestWithDur[[5]]
Z = -0.2101, p-value = 0.8336
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9082960 0.9087579 roc.test(AUCTestWithDur[[5]], AUCTestWithDur[[6]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[5]] and AUCTestWithDur[[6]]
Z = 2.2154, p-value = 0.02673
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9087579 0.9013018 roc.test(AUCTestWithDur[[6]], AUCTestWithDur[[7]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[6]] and AUCTestWithDur[[7]]
Z = -1, p-value = 0.3173
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9013018 0.9030702 roc.test(AUCTestWithDur[[7]], AUCTestWithDur[[8]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[7]] and AUCTestWithDur[[8]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[8]], AUCTestWithDur[[9]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[8]] and AUCTestWithDur[[9]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[9]], AUCTestWithDur[[10]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[9]] and AUCTestWithDur[[10]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[10]], AUCTestWithDur[[11]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[10]] and AUCTestWithDur[[11]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[11]], AUCTestWithDur[[12]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[11]] and AUCTestWithDur[[12]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[12]], AUCTestWithDur[[13]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[12]] and AUCTestWithDur[[13]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[13]], AUCTestWithDur[[14]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[13]] and AUCTestWithDur[[14]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 roc.test(AUCTestWithDur[[14]], AUCTestWithDur[[15]])
DeLong's test for two correlated ROC curves
data: AUCTestWithDur[[14]] and AUCTestWithDur[[15]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.9030702 0.9030702 # 500 trees is the best model4.2.1.2 Model B, 18 predictors (full minus Closure duration)
roc.test(AUCTestNoDur[[1]], AUCTestNoDur[[2]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[1]] and AUCTestNoDur[[2]]
Z = -1.2523, p-value = 0.2105
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8925345 0.8984509 roc.test(AUCTestNoDur[[2]], AUCTestNoDur[[3]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[2]] and AUCTestNoDur[[3]]
Z = 0.49666, p-value = 0.6194
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8984509 0.8974477 roc.test(AUCTestNoDur[[3]], AUCTestNoDur[[4]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[3]] and AUCTestNoDur[[4]]
Z = 1.2903, p-value = 0.1969
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8974477 0.8941443 roc.test(AUCTestNoDur[[4]], AUCTestNoDur[[5]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[4]] and AUCTestNoDur[[5]]
Z = -1.9652, p-value = 0.04939
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8941443 0.8992721 roc.test(AUCTestNoDur[[5]], AUCTestNoDur[[6]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[5]] and AUCTestNoDur[[6]]
Z = 1.3361, p-value = 0.1815
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8992721 0.8977324 roc.test(AUCTestNoDur[[6]], AUCTestNoDur[[7]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[6]] and AUCTestNoDur[[7]]
Z = 0.29669, p-value = 0.7667
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8977324 0.8971258 roc.test(AUCTestNoDur[[7]], AUCTestNoDur[[8]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[7]] and AUCTestNoDur[[8]]
Z = -0.40221, p-value = 0.6875
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8971258 0.8979517 roc.test(AUCTestNoDur[[8]], AUCTestNoDur[[9]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[8]] and AUCTestNoDur[[9]]
Z = 0.98902, p-value = 0.3227
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8979517 0.8977324 roc.test(AUCTestNoDur[[9]], AUCTestNoDur[[10]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[9]] and AUCTestNoDur[[10]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8977324 0.8977324 roc.test(AUCTestNoDur[[10]], AUCTestNoDur[[11]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[10]] and AUCTestNoDur[[11]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8977324 0.8977324 roc.test(AUCTestNoDur[[11]], AUCTestNoDur[[12]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[11]] and AUCTestNoDur[[12]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8977324 0.8977324 roc.test(AUCTestNoDur[[12]], AUCTestNoDur[[13]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[12]] and AUCTestNoDur[[13]]
Z = -0.99967, p-value = 0.3175
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8977324 0.8988429 roc.test(AUCTestNoDur[[13]], AUCTestNoDur[[14]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[13]] and AUCTestNoDur[[14]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8988429 0.8988429 roc.test(AUCTestNoDur[[14]], AUCTestNoDur[[15]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDur[[14]] and AUCTestNoDur[[15]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.8988429 0.8988429 ## 500 trees is the best model.4.2.1.3 Model C, 17 predictors (Full minus Closure duration and VOT)
roc.test(AUCTestNoDurNoVOTDec[[1]], AUCTestNoDurNoVOTDec[[2]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[1]] and AUCTestNoDurNoVOTDec[[2]]
Z = -0.86872, p-value = 0.385
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7544700 0.7623414 roc.test(AUCTestNoDurNoVOTDec[[2]], AUCTestNoDurNoVOTDec[[3]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[2]] and AUCTestNoDurNoVOTDec[[3]]
Z = 0.74265, p-value = 0.4577
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7623414 0.7581980 roc.test(AUCTestNoDurNoVOTDec[[3]], AUCTestNoDurNoVOTDec[[4]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[3]] and AUCTestNoDurNoVOTDec[[4]]
Z = -0.54725, p-value = 0.5842
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7581980 0.7612542 roc.test(AUCTestNoDurNoVOTDec[[4]], AUCTestNoDurNoVOTDec[[5]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[4]] and AUCTestNoDurNoVOTDec[[5]]
Z = 0.26264, p-value = 0.7928
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7612542 0.7599571 roc.test(AUCTestNoDurNoVOTDec[[5]], AUCTestNoDurNoVOTDec[[6]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[5]] and AUCTestNoDurNoVOTDec[[6]]
Z = -0.72598, p-value = 0.4679
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7599571 0.7637318 roc.test(AUCTestNoDurNoVOTDec[[6]], AUCTestNoDurNoVOTDec[[7]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[6]] and AUCTestNoDurNoVOTDec[[7]]
Z = 0.66791, p-value = 0.5042
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7637318 0.7618841 roc.test(AUCTestNoDurNoVOTDec[[7]], AUCTestNoDurNoVOTDec[[8]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[7]] and AUCTestNoDurNoVOTDec[[8]]
Z = 0.10062, p-value = 0.9199
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7618841 0.7615995 roc.test(AUCTestNoDurNoVOTDec[[8]], AUCTestNoDurNoVOTDec[[9]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[8]] and AUCTestNoDurNoVOTDec[[9]]
Z = 0.10783, p-value = 0.9141
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7615995 0.7613195 roc.test(AUCTestNoDurNoVOTDec[[9]], AUCTestNoDurNoVOTDec[[10]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[9]] and AUCTestNoDurNoVOTDec[[10]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7613195 0.7613195 roc.test(AUCTestNoDurNoVOTDec[[10]], AUCTestNoDurNoVOTDec[[11]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[10]] and AUCTestNoDurNoVOTDec[[11]]
Z = 1.406, p-value = 0.1597
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7613195 0.7596865 roc.test(AUCTestNoDurNoVOTDec[[11]], AUCTestNoDurNoVOTDec[[12]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[11]] and AUCTestNoDurNoVOTDec[[12]]
Z = -0.57594, p-value = 0.5647
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.7596865 0.7605030 roc.test(AUCTestNoDurNoVOTDec[[12]], AUCTestNoDurNoVOTDec[[13]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[12]] and AUCTestNoDurNoVOTDec[[13]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.760503 0.760503 roc.test(AUCTestNoDurNoVOTDec[[13]], AUCTestNoDurNoVOTDec[[14]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[13]] and AUCTestNoDurNoVOTDec[[14]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.760503 0.760503 roc.test(AUCTestNoDurNoVOTDec[[14]], AUCTestNoDurNoVOTDec[[15]])
DeLong's test for two correlated ROC curves
data: AUCTestNoDurNoVOTDec[[14]] and AUCTestNoDurNoVOTDec[[15]]
Z = 0, p-value = 1
alternative hypothesis: true difference in AUC is not equal to 0
sample estimates:
AUC of roc1 AUC of roc2
0.760503 0.760503 # 600 trees is the best model. 4.2.2 Models and number of trees
After checking the models above, we have the following number trees that are needed to grow a random forest with the highest predictive accuracy.
Model A: (19 predictors) 500 trees
Model B: (18 predictors) 500 trees
Model C: (17 predictors) 600 trees
4.3 Checking the correlations in the data-frame
Random forests are known to be biased towards correlated data and data with multiple categories and/or cut-points. We start by checking the correlation levels in our data.
# using "psycho"
# we use parts of the data-frame that contains the z-scored variables of interest
cor <- gemination.train[c(37:55)] %>%
correlation()
summary(cor)plot(cor)
As can be seen from the correlation plot, an important number of predictors are correlated with each other. Some with minimal correlations, while other with relatively strong correlations. We will use this information in tuning the computations of variable importance.
4.4 Changing order of outcome
levels(gemination.train$SingGemVoicing)[1] "geminate voiced" "geminate voiceless"
[3] "singleton voiced" "singleton voiceless"gemination.train$SingGemVoicing <- factor(gemination.train$SingGemVoicing,
levels = c("singleton voiceless","singleton voiced","geminate voiceless","geminate voiced"))
levels(gemination.train$SingGemVoicing)[1] "singleton voiceless" "singleton voiced"
[3] "geminate voiceless" "geminate voiced" levels(gemination.test$SingGemVoicing)[1] "geminate voiced" "geminate voiceless"
[3] "singleton voiced" "singleton voiceless"gemination.test$SingGemVoicing <- factor(gemination.test$SingGemVoicing,
levels = c("singleton voiceless","singleton voiced","geminate voiceless","geminate voiced"))
levels(gemination.test$SingGemVoicing)[1] "singleton voiceless" "singleton voiced"
[3] "geminate voiceless" "geminate voiced" 5 Running the three Random Forests via Conditional Inference Trees
5.1 Model A
5.1.1 Training and predicting
set.seed(1)
cforestGemVoicingTrain <- cforest(SingGemVoicing~durationC2_z+durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(19)),ntree=500))
# we use predict to obtain predictions from the training set on the testing set
cforestGemVoicingTest <- predict(cforestGemVoicingTrain, newdata = gemination.test, OOB=TRUE, type = "response")
# Below is the predictions table showing the confusions
# Table predictions (rows) by original data (columns)
round(prop.table(table(cforestGemVoicingTest,gemination.test$SingGemVoicing),2)*100,1)
cforestGemVoicingTest singleton voiceless singleton voiced
singleton voiceless 90.5 8.1
singleton voiced 7.6 91.9
geminate voiceless 0.0 0.0
geminate voiced 1.9 0.0
cforestGemVoicingTest geminate voiceless geminate voiced
singleton voiceless 1.5 0.0
singleton voiced 0.0 0.0
geminate voiceless 92.5 6.1
geminate voiced 6.0 93.9# for percent correct
sum(gemination.test$SingGemVoicing==cforestGemVoicingTest)/nrow(gemination.test)[1] 0.9247492# for correlations between our predictions and original data-frame
cor(as.numeric(cforestGemVoicingTest), as.numeric(gemination.test$SingGemVoicing))[1] 0.95522155.1.2 Variable Importance
We are interested in the importance of our predictors and how they are informative (or not) with respect to our model.
We run two versions. One with no conditional permutations, i.e., with no appropriate corrections for the correlations in our data and one with appropriate corrections
5.1.2.1 No conditional permutations
5.1.2.1.1 Model specification
set.seed(1)
system.time(varImpSingGemVoicingTrainNT <- varimp(cforest(SingGemVoicing~durationC2_z+durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(19)),ntree=500)), conditional = F)) user system elapsed
18.19 0.00 18.24 sort(varImpSingGemVoicingTrainNT) h1mnh2OffsetNorm_z h1mnh2OnsetNormV2_z
-8.200456e-05 -4.555809e-06
intensityOffsetVOTAll_z f1OffsetV1_z
6.879271e-04 7.152620e-04
intensityOnsetV2_z f1OnsetV2_z
7.699317e-04 1.011390e-03
f0OnsetV2_z intensityOffsetV1_z
1.421412e-03 1.662870e-03
f0OffsetV1_z intensityOnsetVOTAll_z
1.922551e-03 5.594533e-03
durationBurst_z durationV2_z
7.216401e-03 7.845103e-03
durationVOT_z durationV1_z
1.505239e-02 1.594077e-02
durationVOTAll_z durationVoicedPercVOTAll_z
2.502961e-02 4.869248e-02
durationVoicedPercC2_z VOTDec_z
1.141959e-01 1.426834e-01
durationC2_z
2.539180e-01 5.1.2.1.2 Plotting
We start by changing names of predictors
# We change names
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationC2_z'] <- 'Closure Dur'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationV2_z'] <- 'V2 Dur'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationV1_z'] <- 'V1 Dur'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'VOTDec_z'] <- 'VOT'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationVOTAll_z'] <- 'Rel Dur'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationBurst_z'] <- 'Burst Dur'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationVOT_z'] <- 'Asp Dur'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationVoicedPercC2_z'] <- 'CD Voicing'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'durationVoicedPercVOTAll_z'] <- 'Rel Voicing'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'intensityOffsetV1_z'] <- 'V1 Off Int'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'intensityOnsetV2_z'] <- 'V2 On Int'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'intensityOffsetVOTAll_z'] <- 'Rel Of Int'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'intensityOnsetVOTAll_z'] <- 'Rel On Int'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'f0OnsetV2_z'] <- 'V2 On f0'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'f0OffsetV1_z'] <- 'V1 Of f0'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'f1OnsetV2_z'] <- 'V2 On F1'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'f1OffsetV1_z'] <- 'V1 Of F1'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'h1mnh2OffsetNorm_z'] <- 'V1 Of H1*-H2*'
names(varImpSingGemVoicingTrainNT)[names(varImpSingGemVoicingTrainNT) == 'h1mnh2OnsetNormV2_z'] <- 'V2 On H1*-H2*'After changing names, we append to a data-frame
varImpSingGemVoicingTrainNT_DF <- varImpSingGemVoicingTrainNT
varImpSingGemVoicingTrainNT_DF <- as.data.frame(varImpSingGemVoicingTrainNT_DF)
varImpSingGemVoicingTrainNT_DF$var <- row.names(varImpSingGemVoicingTrainNT_DF)Then we use ggplot2 to create our Variable Importance figure
ggVarImpSGDurNT_A <- ggplot(varImpSingGemVoicingTrainNT_DF, aes(reorder(var,varImpSingGemVoicingTrainNT_DF),varImpSingGemVoicingTrainNT_DF)) + geom_bar(stat="identity",position="dodge",fill = "gray70") +
coord_flip(ylim = c(0,0.25)) + labs(y = "", x= "", subtitle = "A: Non Conditional 19 predictors") +
scale_y_continuous()+theme_set(theme_bw()) + theme(text=element_text(size=20)) + theme(legend.position="none", axis.title.x=element_blank(),axis.title.y=element_blank())
ggVarImpSGDurNT_A
This plot can already be used to evaluate the importance of predictors. However, it is biased because of the high level of correlations.
5.1.2.2 Conditional permutations
The conditional permutations are used to correct for the correlations in our predictors (conditional = T). This is heavy on resources, and we have already tuned our model to make sure it is not resource intensive. With the update to R 3.5.0, the computations are less resource intensive. One way to reduce resources with previous versions of R is to tune the threshold. Instead of 0.2, one could increase it to 0.5 or 0.8. The threshold is used to increase/decrease number of predictors in the conditional permutations grid and is related to the correlations in the data. With 0.2, any correlation equal or above 0.2 are added to the grid; the same applies to the other thresholds. Here we kept the threshold to 0.2 and we were able to run the computations on a local machine with 4-cores (8 logical) and 32GB RAMs.
5.1.2.2.1 Model specification
set.seed(1)
system.time(varImpSingGemVoicingTrainT <- varimp(cforest(SingGemVoicing~durationC2_z+durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(19)),ntree=500)),
conditional = T, threshold = 0.2)) user system elapsed
725.25 0.21 726.94 sort(varImpSingGemVoicingTrainT) f0OffsetV1_z durationV2_z
-4.555809e-06 -4.555809e-06
h1mnh2OnsetNormV2_z intensityOffsetV1_z
0.000000e+00 0.000000e+00
intensityOnsetV2_z h1mnh2OffsetNorm_z
9.111617e-06 9.111617e-06
f1OffsetV1_z intensityOffsetVOTAll_z
1.366743e-05 1.366743e-05
f0OnsetV2_z f1OnsetV2_z
1.366743e-05 2.277904e-05
durationVoicedPercVOTAll_z durationBurst_z
5.466970e-05 6.833713e-05
intensityOnsetVOTAll_z durationVOT_z
7.289294e-05 8.200456e-05
VOTDec_z durationVOTAll_z
1.047836e-04 1.184510e-04
durationV1_z durationVoicedPercC2_z
2.505695e-04 7.744875e-04
durationC2_z
1.337585e-02 5.1.2.2.2 Plotting
We start by changing names of predictors
# We change names
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationC2_z'] <- 'Closure Dur'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationV2_z'] <- 'V2 Dur'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationV1_z'] <- 'V1 Dur'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'VOTDec_z'] <- 'VOT'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationVOTAll_z'] <- 'Rel Dur'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationBurst_z'] <- 'Burst Dur'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationVOT_z'] <- 'Asp Dur'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationVoicedPercC2_z'] <- 'CD Voicing'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'durationVoicedPercVOTAll_z'] <- 'Rel Voicing'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'intensityOffsetV1_z'] <- 'V1 Off Int'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'intensityOnsetV2_z'] <- 'V2 On Int'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'intensityOffsetVOTAll_z'] <- 'Rel Of Int'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'intensityOnsetVOTAll_z'] <- 'Rel On Int'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'f0OnsetV2_z'] <- 'V2 On f0'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'f0OffsetV1_z'] <- 'V1 Of f0'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'f1OnsetV2_z'] <- 'V2 On F1'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'f1OffsetV1_z'] <- 'V1 Of F1'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'h1mnh2OffsetNorm_z'] <- 'V1 Of H1*-H2*'
names(varImpSingGemVoicingTrainT)[names(varImpSingGemVoicingTrainT) == 'h1mnh2OnsetNormV2_z'] <- 'V2 On H1*-H2*'After changing names, we append to a data-frame
varImpSingGemVoicingTrainT_DF <- varImpSingGemVoicingTrainT
varImpSingGemVoicingTrainT_DF <- as.data.frame(varImpSingGemVoicingTrainT_DF)
varImpSingGemVoicingTrainT_DF$var <- row.names(varImpSingGemVoicingTrainT_DF)Then we use ggplot2 to create our Variable Importance figure
ggVarImpSGDurT_A <- ggplot(varImpSingGemVoicingTrainT_DF, aes(reorder(var,varImpSingGemVoicingTrainT_DF),varImpSingGemVoicingTrainT_DF)) + geom_bar(stat="identity",position="dodge",fill = "gray70") +
coord_flip(ylim = c(0,0.015)) + labs(y = "", x= "", subtitle = "A: Conditional 19 predictors")+
scale_y_continuous()+theme_set(theme_bw()) + theme(text=element_text(size=20)) + theme(legend.position="none", axis.title.x=element_blank(),axis.title.y=element_blank())
ggVarImpSGDurT_A
As can be seen from the results, the closure duration is the most important predictor, followed by voicing in the closure, duration of V1 and voicing in the release. Most of the remaining predictors are contributing to the contrast to a lesser extent with the last six not contributing to it.
5.2 Model B
5.2.1 Training and predicting
set.seed(1)
cforestGemVoicingTrainNoDur <- cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(18)),ntree=500))
cforestGemVoicingTestNoDur <- predict(cforestGemVoicingTrainNoDur, newdata = gemination.test, OOB=TRUE, type = "response")
## Table predictions (rows) by original data (columns)
round(prop.table(table(cforestGemVoicingTestNoDur,gemination.test$SingGemVoicing),2)*100,1)
cforestGemVoicingTestNoDur singleton voiceless singleton voiced
singleton voiceless 57.1 6.8
singleton voiced 8.6 91.9
geminate voiceless 32.4 0.7
geminate voiced 1.9 0.7
cforestGemVoicingTestNoDur geminate voiceless geminate voiced
singleton voiceless 15.0 3.3
singleton voiced 0.0 1.4
geminate voiceless 78.9 5.2
geminate voiced 6.0 90.1# percent correct
sum(gemination.test$SingGemVoicing==cforestGemVoicingTestNoDur)/nrow(gemination.test)[1] 0.8227425#correlations
cor(as.numeric(cforestGemVoicingTestNoDur), as.numeric(gemination.test$SingGemVoicing))[1] 0.7574884The percent classification accuracy of Model B is around 10% lower than that of Model A. This is a clear indication that the closure duration has an important role in discriminating the four-way contrast.
5.2.2 Variable Importance
As before we ran two versions of variable importance: non conditional and conditional
5.2.2.1 No conditional permutations
5.2.2.1.1 Model specification
set.seed(1)
system.time(varImpSingGemVoicingTrainNoDurNT <- varimp(cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(18)),ntree=500)), conditional = F)) user system elapsed
20.59 0.02 20.64 sort(varImpSingGemVoicingTrainNoDurNT) h1mnh2OffsetNorm_z h1mnh2OnsetNormV2_z
6.833713e-05 4.145786e-04
f0OnsetV2_z f1OffsetV1_z
9.430524e-04 1.097950e-03
intensityOffsetVOTAll_z f0OffsetV1_z
1.102506e-03 1.389522e-03
intensityOnsetV2_z f1OnsetV2_z
2.059226e-03 2.118451e-03
intensityOffsetV1_z durationBurst_z
3.334852e-03 6.232346e-03
intensityOnsetVOTAll_z durationVOT_z
7.981777e-03 1.848747e-02
durationV2_z durationV1_z
2.284282e-02 2.370843e-02
durationVOTAll_z durationVoicedPercVOTAll_z
2.516629e-02 5.084282e-02
durationVoicedPercC2_z VOTDec_z
1.260091e-01 2.236492e-01 5.2.2.1.2 Plotting
We start by changing names of predictors
# We change names
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationV2_z'] <- 'V2 Dur'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationV1_z'] <- 'V1 Dur'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'VOTDec_z'] <- 'VOT'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationVOTAll_z'] <- 'Rel Dur'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationBurst_z'] <- 'Burst Dur'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationVOT_z'] <- 'Asp Dur'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationVoicedPercC2_z'] <- 'CD Voicing'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'durationVoicedPercVOTAll_z'] <- 'Rel Voicing'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'intensityOffsetV1_z'] <- 'V1 Off Int'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'intensityOnsetV2_z'] <- 'V2 On Int'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'intensityOffsetVOTAll_z'] <- 'Rel Of Int'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'intensityOnsetVOTAll_z'] <- 'Rel On Int'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'f0OnsetV2_z'] <- 'V2 On f0'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'f0OffsetV1_z'] <- 'V1 Of f0'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'f1OnsetV2_z'] <- 'V2 On F1'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'f1OffsetV1_z'] <- 'V1 Of F1'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'h1mnh2OffsetNorm_z'] <- 'V1 Of H1*-H2*'
names(varImpSingGemVoicingTrainNoDurNT)[names(varImpSingGemVoicingTrainNoDurNT) == 'h1mnh2OnsetNormV2_z'] <- 'V2 On H1*-H2*'After changing names, we append to a data-frame
varImpSingGemVoicingTrainNoDurNT_DF <- varImpSingGemVoicingTrainNoDurNT
varImpSingGemVoicingTrainNoDurNT_DF <- as.data.frame(varImpSingGemVoicingTrainNoDurNT_DF)
varImpSingGemVoicingTrainNoDurNT_DF$var <- row.names(varImpSingGemVoicingTrainNoDurNT_DF)Then we use ggplot2 to create our Variable Importance figure
ggVarImpSGDurNT_B <- ggplot(varImpSingGemVoicingTrainNoDurNT_DF, aes(reorder(var,varImpSingGemVoicingTrainNoDurNT_DF),varImpSingGemVoicingTrainNoDurNT_DF)) + geom_bar(stat="identity",position="dodge",fill = "gray70") +
coord_flip(ylim = c(0,0.25)) + labs(subtitle = "B: Non Conditional 18 predictors") +
scale_y_continuous()+theme_set(theme_bw()) + theme(text=element_text(size=20)) + theme(legend.position="none", axis.title.x=element_blank(),axis.title.y=element_blank())
ggVarImpSGDurNT_B
5.2.2.2 Conditional permutations
As above, we use the conditional permutations and adjust the threshold.
5.2.2.2.1 Model specification
set.seed(1)
system.time(varImpSingGemVoicingTrainNoDurT <- varimp(cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
VOTDec_z+durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(18)),ntree=500)),
conditional = T, threshold = 0.2)) user system elapsed
873.86 0.09 874.93 sort(varImpSingGemVoicingTrainNoDurT) intensityOffsetVOTAll_z f1OnsetV2_z
-1.366743e-05 -9.111617e-06
f1OffsetV1_z f0OnsetV2_z
-4.555809e-06 9.111617e-06
intensityOnsetV2_z f0OffsetV1_z
9.111617e-06 1.366743e-05
durationVOT_z h1mnh2OffsetNorm_z
1.822323e-05 2.277904e-05
durationV2_z h1mnh2OnsetNormV2_z
2.733485e-05 3.189066e-05
durationVoicedPercVOTAll_z durationVOTAll_z
3.644647e-05 4.555809e-05
durationBurst_z intensityOffsetV1_z
5.922551e-05 7.289294e-05
intensityOnsetVOTAll_z durationVoicedPercC2_z
1.093394e-04 1.776765e-04
durationV1_z VOTDec_z
2.551253e-04 7.667426e-03 5.2.2.2.2 Plotting
We start by changing names of predictors
# We change names
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationV2_z'] <- 'V2 Dur'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationV1_z'] <- 'V1 Dur'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'VOTDec_z'] <- 'VOT'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationVOTAll_z'] <- 'Rel Dur'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationBurst_z'] <- 'Burst Dur'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationVOT_z'] <- 'Asp Dur'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationVoicedPercC2_z'] <- 'CD Voicing'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'durationVoicedPercVOTAll_z'] <- 'Rel Voicing'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'intensityOffsetV1_z'] <- 'V1 Off Int'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'intensityOnsetV2_z'] <- 'V2 On Int'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'intensityOffsetVOTAll_z'] <- 'Rel Of Int'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'intensityOnsetVOTAll_z'] <- 'Rel On Int'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'f0OnsetV2_z'] <- 'V2 On f0'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'f0OffsetV1_z'] <- 'V1 Of f0'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'f1OnsetV2_z'] <- 'V2 On F1'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'f1OffsetV1_z'] <- 'V1 Of F1'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'h1mnh2OffsetNorm_z'] <- 'V1 Of H1*-H2*'
names(varImpSingGemVoicingTrainNoDurT)[names(varImpSingGemVoicingTrainNoDurT) == 'h1mnh2OnsetNormV2_z'] <- 'V2 On H1*-H2*'After changing names, we append to a data-frame
varImpSingGemVoicingTrainNoDurT_DF <- varImpSingGemVoicingTrainNoDurT
varImpSingGemVoicingTrainNoDurT_DF <- as.data.frame(varImpSingGemVoicingTrainNoDurT_DF)
varImpSingGemVoicingTrainNoDurT_DF$var <- row.names(varImpSingGemVoicingTrainNoDurT_DF)Then we use ggplot2 to create our Variable Importance figure
ggVarImpSGDurT_B <- ggplot(varImpSingGemVoicingTrainNoDurT_DF, aes(reorder(var,varImpSingGemVoicingTrainNoDurT_DF),varImpSingGemVoicingTrainNoDurT_DF)) + geom_bar(stat="identity",position="dodge",fill = "gray70") +
coord_flip(ylim = c(0,0.01)) + labs(y = "", x= "", subtitle = "B: Conditional 18 predictors") +
scale_y_continuous()+theme_set(theme_bw()) + theme(text=element_text(size=20)) + theme(legend.position="none", axis.title.x=element_blank(),axis.title.y=element_blank())
ggVarImpSGDurT_B
As can be seen from the results, the VOT is the most important predictor, followed by the duration of V1, the voicing in the closure, the intensity at the onset of the release. Most of the remaining predictors are contributing to the contrast to a lesser degree with the last three not contributing to it.
5.3 Model C
5.3.1 Training and predicting
set.seed(1)
cforestGemVoicingTrainNoDurNoVOT <- cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(17)),ntree=600))
cforestGemVoicingTestNoDurNoVOT <- predict(cforestGemVoicingTrainNoDurNoVOT, newdata = gemination.test, OOB=TRUE, type = "response")
## Table predictions (rows) by original data (columns)
round(prop.table(table(cforestGemVoicingTestNoDurNoVOT,gemination.test$SingGemVoicing),2)*100,1)
cforestGemVoicingTestNoDurNoVOT singleton voiceless
singleton voiceless 51.4
singleton voiced 6.7
geminate voiceless 32.4
geminate voiced 9.5
cforestGemVoicingTestNoDurNoVOT singleton voiced geminate voiceless
singleton voiceless 4.7 16.5
singleton voiced 67.6 0.0
geminate voiceless 1.4 78.2
geminate voiced 26.4 5.3
cforestGemVoicingTestNoDurNoVOT geminate voiced
singleton voiceless 4.7
singleton voiced 23.1
geminate voiceless 4.2
geminate voiced 67.9# percent correct
sum(gemination.test$SingGemVoicing==cforestGemVoicingTestNoDurNoVOT)/nrow(gemination.test)[1] 0.6722408# correlation
cor(as.numeric(cforestGemVoicingTestNoDurNoVOT), as.numeric(gemination.test$SingGemVoicing))[1] 0.4525515As can be seen from the above, the percent correct is much lower that the two other models and the levels of correlations as well. This indicates that these secondary correlates are not providing a clear discrimination of the data.
5.3.2 Variable Importance
As before we ran two versions of variable importance: non conditional and conditional
5.3.2.1 No conditional permutations
5.3.2.1.1 Model specification
set.seed(1)
system.time(varImpSingGemVoicingTrainNoDurNoVOTNT <- varimp(cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(17)),ntree=600)), conditional = F)) user system elapsed
25.41 0.00 25.45 sort(varImpSingGemVoicingTrainNoDurNoVOTNT) h1mnh2OnsetNormV2_z h1mnh2OffsetNorm_z
0.0001746393 0.0003302961
f0OnsetV2_z f1OffsetV1_z
0.0015261959 0.0021753986
intensityOffsetVOTAll_z f0OffsetV1_z
0.0027372817 0.0033902809
intensityOnsetV2_z intensityOffsetV1_z
0.0036142749 0.0049544419
f1OnsetV2_z durationBurst_z
0.0062604404 0.0078170084
intensityOnsetVOTAll_z durationVOT_z
0.0167615793 0.0186560364
durationVOTAll_z durationV1_z
0.0326689446 0.0430523918
durationV2_z durationVoicedPercVOTAll_z
0.0486446469 0.0643811693
durationVoicedPercC2_z
0.1278739560 5.3.2.1.2 Plotting
We start by changing names of predictors
# We change names
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationV2_z'] <- 'V2 Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationV1_z'] <- 'V1 Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationVOTAll_z'] <- 'Rel Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationBurst_z'] <- 'Burst Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationVOT_z'] <- 'Asp Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationVoicedPercC2_z'] <- 'CD Voicing'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'durationVoicedPercVOTAll_z'] <- 'Rel Voicing'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'intensityOffsetV1_z'] <- 'V1 Off Int'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'intensityOnsetV2_z'] <- 'V2 On Int'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'intensityOffsetVOTAll_z'] <- 'Rel Of Int'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'intensityOnsetVOTAll_z'] <- 'Rel On Int'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'f0OnsetV2_z'] <- 'V2 On f0'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'f0OffsetV1_z'] <- 'V1 Of f0'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'f1OnsetV2_z'] <- 'V2 On F1'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'f1OffsetV1_z'] <- 'V1 Of F1'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'h1mnh2OffsetNorm_z'] <- 'V1 Of H1*-H2*'
names(varImpSingGemVoicingTrainNoDurNoVOTNT)[names(varImpSingGemVoicingTrainNoDurNoVOTNT) == 'h1mnh2OnsetNormV2_z'] <- 'V2 On H1*-H2*'After changing names, we append to a data-frame
varImpSingGemVoicingTrainNoDurNoVOTNT_DF <- varImpSingGemVoicingTrainNoDurNoVOTNT
varImpSingGemVoicingTrainNoDurNoVOTNT_DF <- as.data.frame(varImpSingGemVoicingTrainNoDurNoVOTNT_DF)
varImpSingGemVoicingTrainNoDurNoVOTNT_DF$var <- row.names(varImpSingGemVoicingTrainNoDurNoVOTNT_DF)Then we use ggplot2 to create our Variable Importance figure
ggVarImpSGDurNT_C <- ggplot(varImpSingGemVoicingTrainNoDurNoVOTNT_DF, aes(reorder(var,varImpSingGemVoicingTrainNoDurNoVOTNT_DF),varImpSingGemVoicingTrainNoDurNoVOTNT_DF)) + geom_bar(stat="identity",position="dodge",fill = "gray70") +
coord_flip(ylim = c(0,0.25)) + labs(subtitle = "C: Non Conditional 17 predictors") +
scale_y_continuous()+theme_set(theme_bw()) + theme(text=element_text(size=20)) + theme(legend.position="none", axis.title.x=element_blank(),axis.title.y=element_blank())
ggVarImpSGDurNT_C
5.3.2.2 Conditional permutations
As above, we use the conditional permutations and adjust the threshold.
5.3.2.2.1 Model specification
set.seed(1)
system.time(varImpSingGemVoicingTrainNoDurNoVOTT <- varimp(cforest(SingGemVoicing~durationVoicedPercC2_z+durationVOTAll_z+
durationBurst_z+durationVOT_z+durationVoicedPercVOTAll_z+
intensityOnsetVOTAll_z+intensityOffsetVOTAll_z+durationV2_z+
f0OnsetV2_z+intensityOnsetV2_z+f1OnsetV2_z+
h1mnh2OnsetNormV2_z+durationV1_z+f0OffsetV1_z+intensityOffsetV1_z+
f1OffsetV1_z+h1mnh2OffsetNorm_z,
data = gemination.train,
control=cforest_unbiased(mtry=round(sqrt(17)),ntree=600)),
conditional = T, threshold = 0.2)) user system elapsed
1202.19 0.19 1203.72 sort(varImpSingGemVoicingTrainNoDurNoVOTT) intensityOffsetVOTAll_z durationBurst_z
-1.518603e-05 -7.593014e-06
h1mnh2OffsetNorm_z f0OffsetV1_z
-7.593014e-06 -3.796507e-06
f0OnsetV2_z durationVOT_z
3.796507e-06 7.593014e-06
f1OffsetV1_z intensityOnsetV2_z
7.593014e-06 1.518603e-05
h1mnh2OnsetNormV2_z intensityOffsetV1_z
1.518603e-05 1.518603e-05
durationVoicedPercVOTAll_z f1OnsetV2_z
2.277904e-05 2.277904e-05
durationVOTAll_z durationVoicedPercC2_z
4.176158e-05 6.074412e-05
intensityOnsetVOTAll_z durationV2_z
1.366743e-04 2.467730e-04
durationV1_z
3.834472e-04 5.3.2.2.2 Plotting
We start by changing names of predictors
# We change names
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationV2_z'] <- 'V2 Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationV1_z'] <- 'V1 Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationVOTAll_z'] <- 'Rel Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationBurst_z'] <- 'Burst Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationVOT_z'] <- 'Asp Dur'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationVoicedPercC2_z'] <- 'CD Voicing'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'durationVoicedPercVOTAll_z'] <- 'Rel Voicing'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'intensityOffsetV1_z'] <- 'V1 Off Int'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'intensityOnsetV2_z'] <- 'V2 On Int'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'intensityOffsetVOTAll_z'] <- 'Rel Of Int'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'intensityOnsetVOTAll_z'] <- 'Rel On Int'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'f0OnsetV2_z'] <- 'V2 On f0'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'f0OffsetV1_z'] <- 'V1 Of f0'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'f1OnsetV2_z'] <- 'V2 On F1'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'f1OffsetV1_z'] <- 'V1 Of F1'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'h1mnh2OffsetNorm_z'] <- 'V1 Of H1*-H2*'
names(varImpSingGemVoicingTrainNoDurNoVOTT)[names(varImpSingGemVoicingTrainNoDurNoVOTT) == 'h1mnh2OnsetNormV2_z'] <- 'V2 On H1*-H2*'After changing names, we append to a data-frame
varImpSingGemVoicingTrainNoDurNoVOTT_DF <- varImpSingGemVoicingTrainNoDurNoVOTT
varImpSingGemVoicingTrainNoDurNoVOTT_DF <- as.data.frame(varImpSingGemVoicingTrainNoDurNoVOTT_DF)
varImpSingGemVoicingTrainNoDurNoVOTT_DF$var <- row.names(varImpSingGemVoicingTrainNoDurNoVOTT_DF)Then we use ggplot2 to create our Variable Importance figure
ggVarImpSGDurT_C <- ggplot(varImpSingGemVoicingTrainNoDurNoVOTT_DF, aes(reorder(var,varImpSingGemVoicingTrainNoDurNoVOTT_DF),varImpSingGemVoicingTrainNoDurNoVOTT_DF)) + geom_bar(stat="identity",position="dodge",fill = "gray70") +
coord_flip(ylim = c(0,0.001)) + labs(y = "", x= "", subtitle = "C: Conditional 17 predictors") +
scale_y_continuous()+theme_set(theme_bw()) + theme(text=element_text(size=20)) + theme(legend.position="none", axis.title.x=element_blank(),axis.title.y=element_blank())
ggVarImpSGDurT_C
As can be seen from the results, the the durations of surrounding vowels followed by intensity at the onset of the release, voicing in the closure and duration of the release are the most important predictors. Their scores are much lower than in the other two models indicating these are secondary. Most of the remaining predictors are contributing to the contrast to a lesser degree with the last four not contributing to it.
6 Conclusion
The results of the Random Forests are a clear indication that the closure duration is the main correlate to the four-way voicing by gemination contrast. Model A had an accuracy of 92.5% which is an extremely high classification rate. The second model that excludes the closure duration is relatively high in classification accuracy yielding an 82.3%. A difference of 10.2% is observed between the two models. This is a clear indication of the role of closure duration in both voicing and gemination. Model C aimed at evaluating the role of all secondary predictors without using the closure duration or the VOT. A relatively low classification rate at 67.2% is indicative of a relatively successful model. It is statistically significant (rate at 25% and above), but the predictive accuracy is not high enough. This is clearly indicating that these correlates act as secondary one and have an enhancement role to the primary correlate; closure duration.
LS0tDQp0aXRsZTogIlZvaWNpbmcgYW5kIEdlbWluYXRpb24gLSBSYW5kb20gRm9yZXN0cyINCmF1dGhvcjogDQogIG5hbWU6ICJKYWxhbCBBbC1UYW1pbWkiDQogIGFmZmlsaWF0aW9uOiAiTmV3Y2FzdGxlIFVuaXZlcnNpdHkiDQpkYXRlOiAiMTcgSnVseSAyMDE4Ig0Kb3V0cHV0OiANCiAgaHRtbF9ub3RlYm9vazoNCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUNCiAgICB0b2M6IHRydWUNCiAgICB0b2NfZGVwdGg6IDYNCiAgICB0b2NfZmxvYXQ6DQogICAgICBjb2xsYXBzZWQ6IHRydWUNCi0tLQ0KDQojIEludHJvZHVjdGlvbg0KVGhpcyBhbmFseXNpcyBhY2NvbXBhbmllcyB0aGUgYXJ0aWNsZSAiQWwtVGFtaW1pLCBKLiBhbmQgS2hhdHRhYiwgRy4sICgyMDE4KS4gKkFjb3VzdGljIGNvcnJlbGF0ZXMgb2YgdGhlIHZvaWNpbmcgY29udHJhc3QgaW4gTGViYW5lc2UgQXJhYmljIHNpbmdsZXRvbiBhbmQgZ2VtaW5hdGUgcGxvc2l2ZXMqLiBJbnZpdGVkIG1hbnVzY3JpcHQgZm9yIHRoZSBzcGVjaWFsIGlzc3VlIG9mIEpvdXJuYWwgb2YgUGhvbmV0aWNzLCAiTWFya2luZyA1MCBZZWFycyBvZiBSZXNlYXJjaCBvbiBWb2ljZSBPbnNldCBUaW1lIGFuZCB0aGUgVm9pY2luZyBDb250cmFzdCBpbiB0aGUgV29ybGTigJlzIExhbmd1YWdlcyIgKGVkcy4sIFQuIENobywgRy4gRG9jaGVydHkgJiBELiBXaGFsZW4pLiBET0k6IGh0dHBzOi8vZG9pLm9yZy8xMC4xMDE2L2oud29jbi4yMDE4LjA5LjAxMA0KDQpUaGlzIG5vdGVib29rIHByZXNlbnRzIHRoZSBhbmFseXNlcyBhbmQgcmVzdWx0cyBvZiB0aGUgUmFuZG9tIEZvcmVzdHMgY2xhc3NpZmljYXRpb24uIFRoZSBhaW0gd2FzIHRvIGV2YWx1YXRlIHRoZSByb2J1c3RuZXNzIG9mIHRoZSByZXN1bHRzIG9idGFpbmVkIGZvciBlYWNoIG9mIHRoZSAxOSBhY291c3RpYyBjb3JyZWxhdGVzIHVzZWQsICg2IGR1cmF0aW9uYWwsIHRocmVlIHZvaWNpbmcsIGFuZCAxMCBub24tdGVtcG9yYWwgaW5jbHVkaW5nIGYwLCBGMSBhbmQgaGFybW9uaWMgZGlmZmVyZW5jZXMpLg0KVGhlc2UgYWNvdXN0aWMgY29ycmVsYXRlcyB3ZXJlIHVzZWQgdG8gZXZhbHVhdGUgdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIHRoZW0gYW5kIHRoZSBmb3VyLXdheSAiY29udHJhc3QiIG9ic2VydmVkIGJldHdlZW4gdm9pY2luZyBhbmQgZ2VtaW5hdGlvbiwgaS5lLiwgVm9pY2VkIFNpbmdsZXRvbiwgVm9pY2VsZXNzIFNpbmdsZXRvbiwgVm9pY2VkIEdlbWluYXRlIGFuZCBWb2ljZWxlc3MgR2VtaW5hdGUuIFRoZSBwcmV2aW91cyBub3RlYm9vayBsb29rZWQgc3BlY2lmaWNhbGx5IGF0IGhvdyBlYWNoICppbmRpdmlkdWFsKiAgYWNvdXN0aWMgY29ycmVsYXRlIHdhcyBhc3NvY2lhdGVkIHdpdGggdGhlIHByZWRpY3RvcnMuIFRoaXMgd2FzIGRvbmUgb24gYW4gaW5kaXZpZHVhbCBiYXNpcyBbc2VlIG5vdGVib29rXShodHRwczovL2phbGFsYWwtdGFtaW1pLmdpdGh1Yi5pby9SLVZvaWNpbmctR2VtaW5hdGlvbi1WT1QvVm9pY2luZyUyMGFuZCUyMEdlbWluYXRpb24lMjAtJTIwTWl4ZWQlMjBFZmZlY3RzJTIwTW9kZWxsaW5nLm5iLmh0bWwpLiBUaGlzIHNlY3Rpb24gZXh0ZW5kcyB0aGUgYW5hbHlzaXMgYnkgbG9va2luZyBhdCB0aGUgKmNvbWJpbmF0aW9uKiBvZiB0aGVzZSBhY291c3RpYyBjb3JyZWxhdGVzIGFuZCB0aGVpciBwcmVkaWN0aXZlIHBvd2VyIGluIGRpc2NyaW1pbmF0aW5nIGJldHdlZW4gdGhlIGZvdXIgY2F0ZWdvcmllcyBhYm92ZS4gDQoNCiMgTG9hZGluZyByZXF1aXJlZCBwYWNrYWdlcw0KDQpXZSBzdGFydCBieSBsb2FkaW5nIHRoZSByZXF1aXJlZCBwYWNrYWdlcy4NCg0KYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgZXJyb3I9RkFMU0V9DQpyZXF1aXJlZFBhY2thZ2VzID0gYygnZHBseXInLCdETXdSJywncGFydHknLCdwUk9DJywncHN5Y2hvJywnZ2dwbG90MicsJ3Jlc2hhcGUnLCdmb3JlYWNoJywnZG9TTk9XJywncGFyYWxsZWwnKQ0KZm9yKHAgaW4gcmVxdWlyZWRQYWNrYWdlcyl7DQogIGlmKCFyZXF1aXJlKHAsY2hhcmFjdGVyLm9ubHkgPSBUUlVFKSkgaW5zdGFsbC5wYWNrYWdlcyhwKQ0KICBsaWJyYXJ5KHAsY2hhcmFjdGVyLm9ubHkgPSBUUlVFKQ0KfQ0KYGBgDQoNCg0KIyBQcmVwcm9jZXNzaW5nIHRoZSBkYXRhDQoNCiMjIFJlYWRpbmcgaW4gdGhlIGRhdGENCg0KV2Ugc3RhcnQgcmVhZGluZyBpbiB0aGUgZGF0YSBhbmQgY2hlY2tpbmcgdGhlIHN0cnVjdHVyZS4gVGhlIGRhdGEgY29udGFpbnMgc29tZSBtaXNzaW5nIGRhdGEuIEZvciB0aGUgUmFuZG9tIEZvcmVzdHMgdG8gd29yayBhcHByb3ByaWF0ZWx5LCB3ZSByZXBsYWNlIHRoZSBtaXNzaW5nIHZhbHVlcyBieSBjZW50cmFsbHkgaW1wdXRlZCB2YWx1ZXMuIA0KDQpgYGB7cn0NClJlc3VsdHNGdWxsT3JpZ2luYWwgPC0gcmVhZC5jc3YoIlJlc3VsdHNGdWxsT3JpZ2luYWxEYXRhLmNzdiIpDQoNCnN0cihSZXN1bHRzRnVsbE9yaWdpbmFsKQ0Kc3VtbWFyeShSZXN1bHRzRnVsbE9yaWdpbmFsKQ0KDQpgYGANCg0KIyMgQ2VudHJhbCBpbXB1dGF0aW9ucw0KDQpBcyBjYW4gYmUgc2VlbiBmcm9tIHRoZSBzdW1tYXJ5IGFib3ZlLCB0aGUgZGF0YSBjb250YWlucyBzb21lIG1pc3NpbmcgZGF0YS4gRm9yIHRoZSBSYW5kb20gRm9yZXN0cyB0byB3b3JrIGFwcHJvcHJpYXRlbHksIHdlIHJlcGxhY2UgdGhlIG1pc3NpbmcgdmFsdWVzIGJ5IGNlbnRyYWxseSBpbXB1dGVkIG9uZXMgDQoNCmBgYHtyfQ0KIyBDZW50cmFsIGltcHV0YXRpb24gdXNpbmcgdGhlIHBhY2thZ2UgIkRNd1IiLg0KDQpSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCA8LSBSZXN1bHRzRnVsbE9yaWdpbmFsCSMgY3JlYXRlIGNvcHkNClJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsWywgMTg6MzZdIDwtIGNlbnRyYWxJbXB1dGF0aW9uKFJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsWywgMTg6MzZdKQ0Kc3RyKFJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsKQ0Kc3VtbWFyeShSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCkNCndyaXRlLmNzdihSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCwiUmVzdWx0c0Z1bGxPcmlnaW5hbENlbnRyYWwuY3N2IikNCmBgYA0KDQojIyBaLVNjb3JpbmcgdGhlIGRhdGENCg0KR2l2ZW4gdGhlIGRpZmZlcmVuY2VzIGluIHNjYWxlcyBvZiBlYWNoIG9mIHRoZSBhY291c3RpYyBjb3JyZWxhdGVzLCBlLmcuLCBkdXJhdGlvbnMgaW4gbWlsbGlzZWNvbmRzLCBpbnRlbnNpdHkgaW4gZEIsIHRoZSBtYWduaXR1ZGVzIG9mIHRoZSB2YWx1ZXMgYXJlIGRpZmZlcmVudC4gQXMgYSB3YXkgdG8gIm5vcm1hbGlzZSIgZm9yIHRoZSBkYXRhIGFuZCB0byBwdXQgYWxsIHByZWRpY3RvcnMgb24gdGhlIHNhbWUgbGV2ZWwsIHdlIHotc2NvcmUgdGhlIGRhdGEuDQoNCmBgYHtyfQ0KIyBmcm9tIHRoZSBwYWNrYWdlICJkcGx5ciINCmFsbF96IDwtIHNlbGVjdChSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCwgZHVyYXRpb25DMjpoMW1uaDJPZmZzZXROb3JtKQ0KYWxsX3ogPC0gY2JpbmQoYWxsX3osIHNlbGVjdChSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCkpDQphbGxfeiA8LSBhcHBseShhbGxfeiwgMiwgc2NhbGUpDQpjb2xuYW1lcyhhbGxfeikgPC0gcGFzdGUoY29sbmFtZXMoYWxsX3opLCAneicsIHNlcCA9ICdfJykNClJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsIDwtIGNiaW5kKFJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsLCBhbGxfeikNCg0Kc3VtbWFyeShSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCkNCnN0cihSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCkNCmBgYA0KDQojIyBDcmVhdGluZyBhIG5ldyBvdXRjb21lDQoNCkluIHRoZSBmb2xsb3dpbmcgYW5hbHlzaXMsIHdlIGFyZSBpbnRlcmVzdGVkIGluIHRoZSBmb3VyLXdheSBjb250cmFzdCBvYnNlcnZlZCBiZXR3ZWVuIHZvaWNpbmcgYW5kIGdlbWluYXRpb24sIGkuZS4sIFZvaWNlbGVzcyBTaW5nbGV0b24sIFZvaWNlZCBTaW5nbGV0b24sIFZvaWNlbGVzcyBHZW1pbmF0ZSBhbmQgVm9pY2VkIEdlbWluYXRlLiANCg0KYGBge3J9DQpSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbFsiU2luZ0dlbVZvaWNpbmciXSA8LSBOQQ0KUmVzdWx0c0Z1bGxPcmlnaW5hbENlbnRyYWxbIlNpbmdHZW1Wb2ljaW5nIl0gPC0gcGFzdGUoUmVzdWx0c0Z1bGxPcmlnaW5hbENlbnRyYWwkc2luZ0dlbSwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsJHZvaWNlZC51bnZvaWNlZCkNClJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsJFNpbmdHZW1Wb2ljaW5nIDwtIGFzLmZhY3RvcihSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCRTaW5nR2VtVm9pY2luZykNCg0Kc3RyKFJlc3VsdHNGdWxsT3JpZ2luYWxDZW50cmFsKQ0KDQpgYGANCg0KQWZ0ZXIgYWxsIHRoaXMgcHJlLXByb2Nlc3NpbmcsIHdlIGFyZSBub3cgcmVhZHkgdG8gc3RhcnQgb3VyIFJhbmRvbSBGb3Jlc3QgYW5hbHlzaXMuDQoNCiMgVHVuaW5nIG91ciBSYW5kb20gRm9yZXN0cw0KDQojIyBDcmVhdGluZyB0cmFpbmluZyBhbmQgdGVzdGluZyBzZXRzDQoNCldlIGNyZWF0ZSBhIHRyYWluaW5nIGFuZCBhIHRlc3Rpbmcgc2V0cyAoNjYlIGFuZCAzMyUgb2YgdGhlIGRhdGEgcmVzcGVjdGl2ZWx5KS4gVGhlIGFpbSBpcyB0byB0cmFpbiB0aHJlZSByYW5kb20gZm9yZXN0cyBhbmQgdXNpbmcgdGhlIHRlc3Rpbmcgc2V0LCB3ZSBhcmUgZXZhbHVhdGluZyB0aGUgcHJlZGljdGlvbiBwb3dlciBvZiBlYWNoLiAgDQoNCmBgYHtyfQ0KIyBjcmVhdGluZyB0cmFpbmluZyBhbmQgdGVzdGluZyBzZXRzDQojIHdlIHNldCB0aGUgc2VlZHMgZm9yIHJlcHJvZHVjaWJpbGl0eQ0Kc2V0LnNlZWQoMSkNCnRyYWluLmlkeCA8LSBzYW1wbGUobnJvdyhSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbCksIDIvMyAqIG5yb3coUmVzdWx0c0Z1bGxPcmlnaW5hbENlbnRyYWwpKQ0KZ2VtaW5hdGlvbi50cmFpbiA8LSBSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbFt0cmFpbi5pZHgsIF0NCmdlbWluYXRpb24udGVzdCA8LSBSZXN1bHRzRnVsbE9yaWdpbmFsQ2VudHJhbFstdHJhaW4uaWR4LCBdDQpgYGANCg0KIyMgRXN0aW1hdGluZyBudW1iZXIgb2YgdHJlZXMgcmVxdWlyZWQNCg0KV2UgdXNlIGEgcHJvY2VkdXJlIHdlIGRldmVsb3BlZCB0byBvYnRhaW4gdGhlIG9wdGltYWwgbnVtYmVyIG9mIHRyZWVzIHJlcXVpcmVkIHRvIGdyb3cgYSBmb3Jlc3Qgd2l0aCB0aGUgaGlnaGVzdCBwcmVkaWN0aXZlIGFjY3VyYWN5LiBUaGlzIGltcGxlbWVudHMgdGhlIHByb2NlZHVyZSBkZXZlbG9wZWQgYnkgIk9zaGlybywgVC4gTS4sIFBlcmV6LCBQLiBTLiwgJiBCYXJhbmF1c2thcywgSi4gQS4gKDIwMTIpLiBIb3cgbWFueSB0cmVlcyBpbiBhIHJhbmRvbSBmb3Jlc3Q/IA0KSW4gSW50ZXJuYXRpb25hbCBXb3Jrc2hvcCBvbiBNYWNoaW5lIExlYXJuaW5nIGFuZCBEYXRhIE1pbmluZyBpbiBQYXR0ZXJuIFJlY29nbml0aW9uIChwcC4gMTU04oCTMTY4KSIgKHNlZSBteSBnaXRodWIgcmVwbyBmb3IgdGhlIHNjcmlwdCkuDQoNClRoZSBjb2RlIGJlbG93IHJ1bnMgdGhlIGFuYWx5c2lzIHVzaW5nIHBhcmFsbGVsIGNvbXB1dGluZy4gV2UgZGVmaW5lIHRoZSBudW1iZXIgb2YgY2x1c3RlcnMgbmVlZGVkIGFuZCB1c2UgdGhlc2UuIFdlIGFyZSB0cmFpbmluZyB0aHJlZSBSYW5kb20gRm9yZXN0cyAoc2VlIHBhcGVyIGZvciBtb3JlIGRldGFpbHMpOg0KDQoxLiBNb2RlbCBBOiBBIGZvcmVzdCB3aXRoIHRoZSBmdWxsIDE5IHByZWRpY3RvcnMgKEFVQ1Rlc3RXaXRoRHVyKSANCg0KMi4gTW9kZWwgQjogQSBmb3Jlc3Qgd2l0aCAxOCBwcmVkaWN0b3JzIChNb2RlbCBBIG1pbnVzIHRoZSBjbG9zdXJlIGR1cmF0aW9uIChBVUNUZXN0Tm9EdXIpKQ0KDQozLiBNb2RlbCBDOiBBIGZvcmVzdCB3aXRoIDE3IHByZWRpY3RvcnMgKE1vZGVsIEIgbWludXMgdGhlIFZPVCAoQVVDVGVzdE5vRHVyTm9WT1QpKQ0KDQpgYGB7cn0NCiMjIGJlbG93IHRvIHByZXBhcmUgdXNpbmcgcGFyYWxsZWwgY29tcHV0aW5nDQpOdW1iZXJPZkNsdXN0ZXIgPC0gZGV0ZWN0Q29yZXMoKQ0KY2wgPC0gbWFrZUNsdXN0ZXIoTnVtYmVyT2ZDbHVzdGVyKQ0KcmVnaXN0ZXJEb1NOT1coY2wpDQoNCiMjIGJlbG93IGNvbXB1dGVzIHRoZSBST0MgKFJlY2VpdmVyIE9wZXJhdGluZyBDdXJ2ZSkgY3VydmVzIGZvciB0aGlzIGRhdGFmcmFtZSANCiMjIGZvciB0aGUgMTUgcmFuZG9tIGZvcmVzdHMgd2l0aCBhIDEwMCB0cmVlcyBpdGVyYXRpb24NCiMjIGFuZCBzYXZlcyB0aGVzZSBpbiBhIGxpc3QgdGhhdCB3aWxsIGJlIHVzZWQgbGF0ZXIgdG8gY29tcGFyZSBST0MgY3VydmVzDQp0cmVlTnVtYiA8LSAxNQ0KDQpzeXN0ZW0udGltZShBVUNUZXN0V2l0aER1ciA8LQ0KICAgICAgICAgICAgICBmb3JlYWNoIChpPTE6dHJlZU51bWIsLnBhY2thZ2VzPWMoInBhcnR5IiwicFJPQyIpKSAlZG9wYXIlIHsNCiAgICAgICAgICAgICAgICBzZXQuc2VlZCgxKQ0KICAgICAgICAgICAgICAgIGNmb3Jlc3QubWRsPWNmb3Jlc3QoU2luZ0dlbVZvaWNpbmd+ZHVyYXRpb25DMl96K2R1cmF0aW9uVm9pY2VkUGVyY0MyX3orZHVyYXRpb25WT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVk9URGVjX3orZHVyYXRpb25CdXJzdF96K2R1cmF0aW9uVk9UX3orZHVyYXRpb25Wb2ljZWRQZXJjVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGludGVuc2l0eU9uc2V0Vk9UQWxsX3oraW50ZW5zaXR5T2Zmc2V0Vk9UQWxsX3orZHVyYXRpb25WMl96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmME9uc2V0VjJfeitpbnRlbnNpdHlPbnNldFYyX3orZjFPbnNldFYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGgxbW5oMk9uc2V0Tm9ybVYyX3orZHVyYXRpb25WMV96K2YwT2Zmc2V0VjFfeitpbnRlbnNpdHlPZmZzZXRWMV96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmMU9mZnNldFYxX3oraDFtbmgyT2Zmc2V0Tm9ybV96LCBkYXRhID0gZ2VtaW5hdGlvbi50cmFpbiwNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnRyb2xzID0gY2ZvcmVzdF91bmJpYXNlZChtdHJ5ID0gcm91bmQoc3FydCgxOSkpLCBudHJlZSA9IGkqMTAwKSkNCiAgICAgICAgICAgICAgICBjZm9yZXN0LmNvbmQgPSBwcmVkaWN0KGNmb3Jlc3QubWRsLE9PQj1UUlVFKQ0KICAgICAgICAgICAgICAgIHJvYyA8LSByb2MoZ2VtaW5hdGlvbi50cmFpbiRTaW5nR2VtVm9pY2luZyxhcy5udW1lcmljKGNmb3Jlc3QuY29uZCkpDQogICAgICAgICAgICAgIH0pDQoNCg0Kc3lzdGVtLnRpbWUoQVVDVGVzdE5vRHVyIDwtDQogICAgICAgICAgICAgIGZvcmVhY2ggKGk9MTp0cmVlTnVtYiwucGFja2FnZXM9YygicGFydHkiLCJwUk9DIikpICVkb3BhciUgew0KICAgICAgICAgICAgICAgIHNldC5zZWVkKDEpDQogICAgICAgICAgICAgICAgY2ZvcmVzdC5tZGw9Y2ZvcmVzdChTaW5nR2VtVm9pY2luZ35kdXJhdGlvblZvaWNlZFBlcmNDMl96K2R1cmF0aW9uVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZPVERlY196K2R1cmF0aW9uQnVyc3RfeitkdXJhdGlvblZPVF96K2R1cmF0aW9uVm9pY2VkUGVyY1ZPVEFsbF96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnRlbnNpdHlPbnNldFZPVEFsbF96K2ludGVuc2l0eU9mZnNldFZPVEFsbF96K2R1cmF0aW9uVjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZjBPbnNldFYyX3oraW50ZW5zaXR5T25zZXRWMl96K2YxT25zZXRWMl96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoMW1uaDJPbnNldE5vcm1WMl96K2R1cmF0aW9uVjFfeitmME9mZnNldFYxX3oraW50ZW5zaXR5T2Zmc2V0VjFfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZjFPZmZzZXRWMV96K2gxbW5oMk9mZnNldE5vcm1feiwgZGF0YSA9IGdlbWluYXRpb24udHJhaW4sDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjb250cm9scyA9IGNmb3Jlc3RfdW5iaWFzZWQobXRyeSA9IHJvdW5kKHNxcnQoMTgpKSwgbnRyZWUgPSBpKjEwMCkpDQogICAgICAgICAgICAgICAgY2ZvcmVzdC5jb25kID0gcHJlZGljdChjZm9yZXN0Lm1kbCxPT0I9VFJVRSkNCiAgICAgICAgICAgICAgICByb2MgPC0gcm9jKGdlbWluYXRpb24udHJhaW4kU2luZ0dlbVZvaWNpbmcsYXMubnVtZXJpYyhjZm9yZXN0LmNvbmQpKQ0KICAgICAgICAgICAgICB9KQ0KDQpzeXN0ZW0udGltZShBVUNUZXN0Tm9EdXJOb1ZPVERlYyA8LQ0KICAgICAgICAgICAgICBmb3JlYWNoIChpPTE6dHJlZU51bWIsLnBhY2thZ2VzPWMoInBhcnR5IiwicFJPQyIpKSAlZG9wYXIlIHsNCiAgICAgICAgICAgICAgICBzZXQuc2VlZCgxKQ0KICAgICAgICAgICAgICAgIGNmb3Jlc3QubWRsPWNmb3Jlc3QoU2luZ0dlbVZvaWNpbmd+ZHVyYXRpb25Wb2ljZWRQZXJjQzJfeitkdXJhdGlvblZPVEFsbF96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkdXJhdGlvbkJ1cnN0X3orZHVyYXRpb25WT1RfeitkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ZW5zaXR5T25zZXRWT1RBbGxfeitpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeitkdXJhdGlvblYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYwT25zZXRWMl96K2ludGVuc2l0eU9uc2V0VjJfeitmMU9uc2V0VjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaDFtbmgyT25zZXROb3JtVjJfeitkdXJhdGlvblYxX3orZjBPZmZzZXRWMV96K2ludGVuc2l0eU9mZnNldFYxX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYxT2Zmc2V0VjFfeitoMW1uaDJPZmZzZXROb3JtX3osIGRhdGEgPSBnZW1pbmF0aW9uLnRyYWluLA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbHMgPSBjZm9yZXN0X3VuYmlhc2VkKG10cnkgPSByb3VuZChzcXJ0KDE3KSksIG50cmVlID0gaSoxMDApKQ0KICAgICAgICAgICAgICAgIGNmb3Jlc3QuY29uZCA9IHByZWRpY3QoY2ZvcmVzdC5tZGwsT09CPVRSVUUpDQogICAgICAgICAgICAgICAgcm9jIDwtIHJvYyhnZW1pbmF0aW9uLnRyYWluJFNpbmdHZW1Wb2ljaW5nLGFzLm51bWVyaWMoY2ZvcmVzdC5jb25kKSkNCiAgICAgICAgICAgICAgfSkNCg0KDQojIHN0b3AgY2x1c3Rlci4gSWYgbm90IGRvbmUgUiB3aWxsIHN0aWxsIHVzZSBhbGwgY29yZXMgYXMgZGVmaW5lZCBhYm92ZSBhbmQgdGhlIHN5c3RlbSB3aWxsIGJlY29tZSB0b28gc2xvdyEhDQpzdG9wQ2x1c3RlcihjbCkNCiMNCg0KYGBgDQoNCiMjIyBTaWduaWZpY2FuY2UgdGVzdGluZw0KDQojIyMjIE1vZGVsIEEsIDE5IHByZWRpY3RvcnMNCg0KYGBge3J9DQojIGNoZWNrIHNpZ25pZmljYW50IFJPQyBjb21wYXJpc29uDQojIHdpdGggZHVyYXRpb24NCg0Kcm9jLnRlc3QoQVVDVGVzdFdpdGhEdXJbWzFdXSwgQVVDVGVzdFdpdGhEdXJbWzJdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1syXV0sIEFVQ1Rlc3RXaXRoRHVyW1szXV0pDQpyb2MudGVzdChBVUNUZXN0V2l0aER1cltbM11dLCBBVUNUZXN0V2l0aER1cltbNF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdFdpdGhEdXJbWzRdXSwgQVVDVGVzdFdpdGhEdXJbWzVdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1s1XV0sIEFVQ1Rlc3RXaXRoRHVyW1s2XV0pDQpyb2MudGVzdChBVUNUZXN0V2l0aER1cltbNl1dLCBBVUNUZXN0V2l0aER1cltbN11dKQ0Kcm9jLnRlc3QoQVVDVGVzdFdpdGhEdXJbWzddXSwgQVVDVGVzdFdpdGhEdXJbWzhdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1s4XV0sIEFVQ1Rlc3RXaXRoRHVyW1s5XV0pDQpyb2MudGVzdChBVUNUZXN0V2l0aER1cltbOV1dLCBBVUNUZXN0V2l0aER1cltbMTBdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1sxMF1dLCBBVUNUZXN0V2l0aER1cltbMTFdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1sxMV1dLCBBVUNUZXN0V2l0aER1cltbMTJdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1sxMl1dLCBBVUNUZXN0V2l0aER1cltbMTNdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1sxM11dLCBBVUNUZXN0V2l0aER1cltbMTRdXSkNCnJvYy50ZXN0KEFVQ1Rlc3RXaXRoRHVyW1sxNF1dLCBBVUNUZXN0V2l0aER1cltbMTVdXSkNCg0KIyA1MDAgdHJlZXMgaXMgdGhlIGJlc3QgbW9kZWwNCg0KYGBgDQoNCiMjIyMgTW9kZWwgQiwgMTggcHJlZGljdG9ycyAoZnVsbCBtaW51cyBDbG9zdXJlIGR1cmF0aW9uKQ0KDQpgYGB7cn0NCg0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1sxXV0sIEFVQ1Rlc3ROb0R1cltbMl1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1syXV0sIEFVQ1Rlc3ROb0R1cltbM11dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1szXV0sIEFVQ1Rlc3ROb0R1cltbNF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1s0XV0sIEFVQ1Rlc3ROb0R1cltbNV1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1s1XV0sIEFVQ1Rlc3ROb0R1cltbNl1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1s2XV0sIEFVQ1Rlc3ROb0R1cltbN11dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1s3XV0sIEFVQ1Rlc3ROb0R1cltbOF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1s4XV0sIEFVQ1Rlc3ROb0R1cltbOV1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1s5XV0sIEFVQ1Rlc3ROb0R1cltbMTBdXSkNCnJvYy50ZXN0KEFVQ1Rlc3ROb0R1cltbMTBdXSwgQVVDVGVzdE5vRHVyW1sxMV1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1sxMV1dLCBBVUNUZXN0Tm9EdXJbWzEyXV0pDQpyb2MudGVzdChBVUNUZXN0Tm9EdXJbWzEyXV0sIEFVQ1Rlc3ROb0R1cltbMTNdXSkNCnJvYy50ZXN0KEFVQ1Rlc3ROb0R1cltbMTNdXSwgQVVDVGVzdE5vRHVyW1sxNF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyW1sxNF1dLCBBVUNUZXN0Tm9EdXJbWzE1XV0pDQoNCg0KIyMgNTAwIHRyZWVzIGlzIHRoZSBiZXN0IG1vZGVsLg0KDQpgYGANCg0KIyMjIyBNb2RlbCBDLCAxNyBwcmVkaWN0b3JzIChGdWxsIG1pbnVzIENsb3N1cmUgZHVyYXRpb24gYW5kIFZPVCkNCg0KYGBge3J9DQpyb2MudGVzdChBVUNUZXN0Tm9EdXJOb1ZPVERlY1tbMV1dLCBBVUNUZXN0Tm9EdXJOb1ZPVERlY1tbMl1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzJdXSwgQVVDVGVzdE5vRHVyTm9WT1REZWNbWzNdXSkNCnJvYy50ZXN0KEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1szXV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1s0XV0pDQpyb2MudGVzdChBVUNUZXN0Tm9EdXJOb1ZPVERlY1tbNF1dLCBBVUNUZXN0Tm9EdXJOb1ZPVERlY1tbNV1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzVdXSwgQVVDVGVzdE5vRHVyTm9WT1REZWNbWzZdXSkNCnJvYy50ZXN0KEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1s2XV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1s3XV0pDQpyb2MudGVzdChBVUNUZXN0Tm9EdXJOb1ZPVERlY1tbN11dLCBBVUNUZXN0Tm9EdXJOb1ZPVERlY1tbOF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzhdXSwgQVVDVGVzdE5vRHVyTm9WT1REZWNbWzldXSkNCnJvYy50ZXN0KEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1s5XV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1sxMF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzEwXV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1sxMV1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzExXV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1sxMl1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzEyXV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1sxM11dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzEzXV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1sxNF1dKQ0Kcm9jLnRlc3QoQVVDVGVzdE5vRHVyTm9WT1REZWNbWzE0XV0sIEFVQ1Rlc3ROb0R1ck5vVk9URGVjW1sxNV1dKQ0KDQojIDYwMCB0cmVlcyBpcyB0aGUgYmVzdCBtb2RlbC4gDQpgYGANCg0KIyMjIE1vZGVscyBhbmQgbnVtYmVyIG9mIHRyZWVzDQoNCkFmdGVyIGNoZWNraW5nIHRoZSBtb2RlbHMgYWJvdmUsIHdlIGhhdmUgdGhlIGZvbGxvd2luZyBudW1iZXIgdHJlZXMgdGhhdCBhcmUgbmVlZGVkIHRvIGdyb3cgYSByYW5kb20gZm9yZXN0IHdpdGggdGhlIGhpZ2hlc3QgcHJlZGljdGl2ZSBhY2N1cmFjeS4NCg0KMS4gTW9kZWwgQTogKDE5IHByZWRpY3RvcnMpIDUwMCB0cmVlcw0KDQoyLiBNb2RlbCBCOiAoMTggcHJlZGljdG9ycykgNTAwIHRyZWVzDQoNCjMuIE1vZGVsIEM6ICgxNyBwcmVkaWN0b3JzKSA2MDAgdHJlZXMNCg0KIyMgQ2hlY2tpbmcgdGhlIGNvcnJlbGF0aW9ucyBpbiB0aGUgZGF0YS1mcmFtZQ0KDQpSYW5kb20gZm9yZXN0cyBhcmUga25vd24gdG8gYmUgYmlhc2VkIHRvd2FyZHMgY29ycmVsYXRlZCBkYXRhIGFuZCBkYXRhIHdpdGggbXVsdGlwbGUgY2F0ZWdvcmllcyBhbmQvb3IgY3V0LXBvaW50cy4gV2Ugc3RhcnQgYnkgY2hlY2tpbmcgdGhlIGNvcnJlbGF0aW9uIGxldmVscyBpbiBvdXIgZGF0YS4NCg0KYGBge3IgZmlnLndpZHRoPTksIGZpZy5oZWlnaHQ9OX0NCiMgdXNpbmcgInBzeWNobyINCiMgd2UgdXNlIHBhcnRzIG9mIHRoZSBkYXRhLWZyYW1lIHRoYXQgY29udGFpbnMgdGhlIHotc2NvcmVkIHZhcmlhYmxlcyBvZiBpbnRlcmVzdA0KY29yIDwtIGdlbWluYXRpb24udHJhaW5bYygzNzo1NSldICU+JSANCiAgY29ycmVsYXRpb24oKQ0Kc3VtbWFyeShjb3IpDQpwbG90KGNvcikNCmBgYA0KDQpBcyBjYW4gYmUgc2VlbiBmcm9tIHRoZSBjb3JyZWxhdGlvbiBwbG90LCBhbiBpbXBvcnRhbnQgbnVtYmVyIG9mIHByZWRpY3RvcnMgYXJlIGNvcnJlbGF0ZWQgd2l0aCBlYWNoIG90aGVyLiBTb21lIHdpdGggbWluaW1hbCBjb3JyZWxhdGlvbnMsIHdoaWxlIG90aGVyIHdpdGggcmVsYXRpdmVseSBzdHJvbmcgY29ycmVsYXRpb25zLiBXZSB3aWxsIHVzZSB0aGlzIGluZm9ybWF0aW9uIGluIHR1bmluZyB0aGUgY29tcHV0YXRpb25zIG9mIHZhcmlhYmxlIGltcG9ydGFuY2UuDQoNCiMjIENoYW5naW5nIG9yZGVyIG9mIG91dGNvbWUNCg0KYGBge3J9DQpsZXZlbHMoZ2VtaW5hdGlvbi50cmFpbiRTaW5nR2VtVm9pY2luZykNCmdlbWluYXRpb24udHJhaW4kU2luZ0dlbVZvaWNpbmcgPC0gZmFjdG9yKGdlbWluYXRpb24udHJhaW4kU2luZ0dlbVZvaWNpbmcsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsZXZlbHMgPSBjKCJzaW5nbGV0b24gdm9pY2VsZXNzIiwic2luZ2xldG9uIHZvaWNlZCIsImdlbWluYXRlIHZvaWNlbGVzcyIsImdlbWluYXRlIHZvaWNlZCIpKQ0KbGV2ZWxzKGdlbWluYXRpb24udHJhaW4kU2luZ0dlbVZvaWNpbmcpDQpsZXZlbHMoZ2VtaW5hdGlvbi50ZXN0JFNpbmdHZW1Wb2ljaW5nKQ0KZ2VtaW5hdGlvbi50ZXN0JFNpbmdHZW1Wb2ljaW5nIDwtIGZhY3RvcihnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmcsDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsZXZlbHMgPSBjKCJzaW5nbGV0b24gdm9pY2VsZXNzIiwic2luZ2xldG9uIHZvaWNlZCIsImdlbWluYXRlIHZvaWNlbGVzcyIsImdlbWluYXRlIHZvaWNlZCIpKQ0KbGV2ZWxzKGdlbWluYXRpb24udGVzdCRTaW5nR2VtVm9pY2luZykNCg0KYGBgDQoNCiMgUnVubmluZyB0aGUgdGhyZWUgUmFuZG9tIEZvcmVzdHMgdmlhIENvbmRpdGlvbmFsIEluZmVyZW5jZSBUcmVlcw0KDQojIyBNb2RlbCBBDQoNCiMjIyBUcmFpbmluZyBhbmQgcHJlZGljdGluZw0KDQpgYGB7cn0NCnNldC5zZWVkKDEpDQpjZm9yZXN0R2VtVm9pY2luZ1RyYWluIDwtIGNmb3Jlc3QoU2luZ0dlbVZvaWNpbmd+ZHVyYXRpb25DMl96K2R1cmF0aW9uVm9pY2VkUGVyY0MyX3orZHVyYXRpb25WT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVk9URGVjX3orZHVyYXRpb25CdXJzdF96K2R1cmF0aW9uVk9UX3orZHVyYXRpb25Wb2ljZWRQZXJjVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGludGVuc2l0eU9uc2V0Vk9UQWxsX3oraW50ZW5zaXR5T2Zmc2V0Vk9UQWxsX3orZHVyYXRpb25WMl96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmME9uc2V0VjJfeitpbnRlbnNpdHlPbnNldFYyX3orZjFPbnNldFYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGgxbW5oMk9uc2V0Tm9ybVYyX3orZHVyYXRpb25WMV96K2YwT2Zmc2V0VjFfeitpbnRlbnNpdHlPZmZzZXRWMV96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmMU9mZnNldFYxX3oraDFtbmgyT2Zmc2V0Tm9ybV96LCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBnZW1pbmF0aW9uLnRyYWluLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnRyb2w9Y2ZvcmVzdF91bmJpYXNlZChtdHJ5PXJvdW5kKHNxcnQoMTkpKSxudHJlZT01MDApKQ0KDQojIHdlIHVzZSBwcmVkaWN0IHRvIG9idGFpbiBwcmVkaWN0aW9ucyBmcm9tIHRoZSB0cmFpbmluZyBzZXQgb24gdGhlIHRlc3Rpbmcgc2V0DQpjZm9yZXN0R2VtVm9pY2luZ1Rlc3QgPC0gcHJlZGljdChjZm9yZXN0R2VtVm9pY2luZ1RyYWluLCBuZXdkYXRhID0gZ2VtaW5hdGlvbi50ZXN0LCBPT0I9VFJVRSwgdHlwZSA9ICJyZXNwb25zZSIpDQoNCiMgQmVsb3cgaXMgdGhlIHByZWRpY3Rpb25zIHRhYmxlIHNob3dpbmcgdGhlIGNvbmZ1c2lvbnMNCg0KIyBUYWJsZSBwcmVkaWN0aW9ucyAocm93cykgYnkgb3JpZ2luYWwgZGF0YSAoY29sdW1ucykgDQpyb3VuZChwcm9wLnRhYmxlKHRhYmxlKGNmb3Jlc3RHZW1Wb2ljaW5nVGVzdCxnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmcpLDIpKjEwMCwxKQ0KDQojIGZvciBwZXJjZW50IGNvcnJlY3QNCnN1bShnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmc9PWNmb3Jlc3RHZW1Wb2ljaW5nVGVzdCkvbnJvdyhnZW1pbmF0aW9uLnRlc3QpDQoNCiMgZm9yIGNvcnJlbGF0aW9ucyBiZXR3ZWVuIG91ciBwcmVkaWN0aW9ucyBhbmQgb3JpZ2luYWwgZGF0YS1mcmFtZQ0KY29yKGFzLm51bWVyaWMoY2ZvcmVzdEdlbVZvaWNpbmdUZXN0KSwgYXMubnVtZXJpYyhnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmcpKQ0KDQpgYGANCg0KIyMjIFZhcmlhYmxlIEltcG9ydGFuY2UNCg0KV2UgYXJlIGludGVyZXN0ZWQgaW4gdGhlIGltcG9ydGFuY2Ugb2Ygb3VyIHByZWRpY3RvcnMgYW5kIGhvdyB0aGV5IGFyZSBpbmZvcm1hdGl2ZSAob3Igbm90KSB3aXRoIHJlc3BlY3QgdG8gb3VyIG1vZGVsLg0KDQpXZSBydW4gdHdvIHZlcnNpb25zLiBPbmUgd2l0aCBubyBjb25kaXRpb25hbCBwZXJtdXRhdGlvbnMsIGkuZS4sIHdpdGggbm8gYXBwcm9wcmlhdGUgY29ycmVjdGlvbnMgZm9yIHRoZSBjb3JyZWxhdGlvbnMgaW4gb3VyIGRhdGEgYW5kIG9uZSB3aXRoIGFwcHJvcHJpYXRlIGNvcnJlY3Rpb25zDQoNCiMjIyMgTm8gY29uZGl0aW9uYWwgcGVybXV0YXRpb25zDQoNCiMjIyMjIE1vZGVsIHNwZWNpZmljYXRpb24NCg0KYGBge3J9DQpzZXQuc2VlZCgxKQ0Kc3lzdGVtLnRpbWUodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UIDwtIHZhcmltcChjZm9yZXN0KFNpbmdHZW1Wb2ljaW5nfmR1cmF0aW9uQzJfeitkdXJhdGlvblZvaWNlZFBlcmNDMl96K2R1cmF0aW9uVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFZPVERlY196K2R1cmF0aW9uQnVyc3RfeitkdXJhdGlvblZPVF96K2R1cmF0aW9uVm9pY2VkUGVyY1ZPVEFsbF96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGludGVuc2l0eU9uc2V0Vk9UQWxsX3oraW50ZW5zaXR5T2Zmc2V0Vk9UQWxsX3orZHVyYXRpb25WMl96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYwT25zZXRWMl96K2ludGVuc2l0eU9uc2V0VjJfeitmMU9uc2V0VjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoMW1uaDJPbnNldE5vcm1WMl96K2R1cmF0aW9uVjFfeitmME9mZnNldFYxX3oraW50ZW5zaXR5T2Zmc2V0VjFfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmMU9mZnNldFYxX3oraDFtbmgyT2Zmc2V0Tm9ybV96LCAgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBnZW1pbmF0aW9uLnRyYWluLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxOSkpLG50cmVlPTUwMCkpLCBjb25kaXRpb25hbCA9IEYpKQ0KDQpzb3J0KHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkNCg0KYGBgDQoNCiMjIyMjIFBsb3R0aW5nDQoNCldlIHN0YXJ0IGJ5IGNoYW5naW5nIG5hbWVzIG9mIHByZWRpY3RvcnMgDQoNCmBgYHtyfQ0KIyBXZSBjaGFuZ2UgbmFtZXMNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKSA9PSAnZHVyYXRpb25DMl96J10gPC0gJ0Nsb3N1cmUgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpID09ICdkdXJhdGlvblYyX3onXSA8LSAnVjIgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpID09ICdkdXJhdGlvblYxX3onXSA8LSAnVjEgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpID09ICdWT1REZWNfeiddIDwtICdWT1QnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2R1cmF0aW9uVk9UQWxsX3onXSA8LSAnUmVsIER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKSA9PSAnZHVyYXRpb25CdXJzdF96J10gPC0gJ0J1cnN0IER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKSA9PSAnZHVyYXRpb25WT1RfeiddIDwtICdBc3AgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpID09ICdkdXJhdGlvblZvaWNlZFBlcmNDMl96J10gPC0gJ0NEIFZvaWNpbmcnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2R1cmF0aW9uVm9pY2VkUGVyY1ZPVEFsbF96J10gPC0gJ1JlbCBWb2ljaW5nJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpID09ICdpbnRlbnNpdHlPZmZzZXRWMV96J10gPC0gJ1YxIE9mZiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2ludGVuc2l0eU9uc2V0VjJfeiddIDwtICdWMiBPbiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2ludGVuc2l0eU9mZnNldFZPVEFsbF96J10gPC0gJ1JlbCBPZiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2ludGVuc2l0eU9uc2V0Vk9UQWxsX3onXSA8LSAnUmVsIE9uIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKSA9PSAnZjBPbnNldFYyX3onXSA8LSAnVjIgT24gZjAnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2YwT2Zmc2V0VjFfeiddIDwtICdWMSBPZiBmMCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKSA9PSAnZjFPbnNldFYyX3onXSA8LSAnVjIgT24gRjEnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2YxT2Zmc2V0VjFfeiddIDwtICdWMSBPZiBGMScNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UKSA9PSAnaDFtbmgyT2Zmc2V0Tm9ybV96J10gPC0gJ1YxIE9mIEgxKi1IMionDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5OVCkgPT0gJ2gxbW5oMk9uc2V0Tm9ybVYyX3onXSA8LSAnVjIgT24gSDEqLUgyKicNCg0KYGBgDQoNCkFmdGVyIGNoYW5naW5nIG5hbWVzLCB3ZSBhcHBlbmQgdG8gYSBkYXRhLWZyYW1lDQoNCmBgYHtyfQ0KDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlRfREYgPC0gdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlRfREYgPC0gYXMuZGF0YS5mcmFtZSh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlRfREYpDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlRfREYkdmFyIDwtIHJvdy5uYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlRfREYpDQpgYGANCg0KVGhlbiB3ZSB1c2UgZ2dwbG90MiB0byBjcmVhdGUgb3VyIFZhcmlhYmxlIEltcG9ydGFuY2UgZmlndXJlDQoNCmBgYHtyIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTZ9DQpnZ1ZhckltcFNHRHVyTlRfQSA8LSBnZ3Bsb3QodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UX0RGLCBhZXMocmVvcmRlcih2YXIsdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5UX0RGKSx2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTlRfREYpKSArIGdlb21fYmFyKHN0YXQ9ImlkZW50aXR5Iixwb3NpdGlvbj0iZG9kZ2UiLGZpbGwgPSAiZ3JheTcwIikgKyANCiAgY29vcmRfZmxpcCh5bGltID0gYygwLDAuMjUpKSArIGxhYnMoeSA9ICIiLCB4PSAiIiwgc3VidGl0bGUgPSAiQTogTm9uIENvbmRpdGlvbmFsIDE5IHByZWRpY3RvcnMiKSArDQogIHNjYWxlX3lfY29udGludW91cygpK3RoZW1lX3NldCh0aGVtZV9idygpKSArIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKSArIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj0ibm9uZSIsIGF4aXMudGl0bGUueD1lbGVtZW50X2JsYW5rKCksYXhpcy50aXRsZS55PWVsZW1lbnRfYmxhbmsoKSkNCg0KZ2dWYXJJbXBTR0R1ck5UX0ENCmBgYA0KDQpUaGlzIHBsb3QgY2FuIGFscmVhZHkgYmUgdXNlZCB0byBldmFsdWF0ZSB0aGUgaW1wb3J0YW5jZSBvZiBwcmVkaWN0b3JzLiBIb3dldmVyLCBpdCBpcyBiaWFzZWQgYmVjYXVzZSBvZiB0aGUgaGlnaCBsZXZlbCBvZiBjb3JyZWxhdGlvbnMuIA0KDQojIyMjIENvbmRpdGlvbmFsIHBlcm11dGF0aW9ucw0KDQpUaGUgY29uZGl0aW9uYWwgcGVybXV0YXRpb25zIGFyZSB1c2VkIHRvIGNvcnJlY3QgZm9yIHRoZSBjb3JyZWxhdGlvbnMgaW4gb3VyIHByZWRpY3RvcnMgKGNvbmRpdGlvbmFsID0gVCkuIFRoaXMgaXMgaGVhdnkgb24gcmVzb3VyY2VzLCBhbmQgd2UgaGF2ZSBhbHJlYWR5IHR1bmVkIG91ciBtb2RlbCB0byBtYWtlIHN1cmUgaXQgaXMgbm90IHJlc291cmNlIGludGVuc2l2ZS4gV2l0aCB0aGUgdXBkYXRlIHRvIFIgMy41LjAsIHRoZSBjb21wdXRhdGlvbnMgYXJlIGxlc3MgcmVzb3VyY2UgaW50ZW5zaXZlLiBPbmUgd2F5IHRvIHJlZHVjZSByZXNvdXJjZXMgd2l0aCBwcmV2aW91cyB2ZXJzaW9ucyBvZiBSIGlzIHRvIHR1bmUgdGhlIHRocmVzaG9sZC4gSW5zdGVhZCBvZiAwLjIsIG9uZSBjb3VsZCBpbmNyZWFzZSBpdCB0byAwLjUgb3IgMC44LiBUaGUgdGhyZXNob2xkIGlzIHVzZWQgdG8gaW5jcmVhc2UvZGVjcmVhc2UgbnVtYmVyIG9mIHByZWRpY3RvcnMgaW4gdGhlIGNvbmRpdGlvbmFsIHBlcm11dGF0aW9ucyBncmlkIGFuZCBpcyByZWxhdGVkIHRvIHRoZSBjb3JyZWxhdGlvbnMgaW4gdGhlIGRhdGEuIFdpdGggMC4yLCBhbnkgY29ycmVsYXRpb24gZXF1YWwgb3IgYWJvdmUgMC4yIGFyZSBhZGRlZCB0byB0aGUgZ3JpZDsgdGhlIHNhbWUgYXBwbGllcyB0byB0aGUgb3RoZXIgdGhyZXNob2xkcy4gSGVyZSB3ZSBrZXB0IHRoZSB0aHJlc2hvbGQgdG8gMC4yIGFuZCB3ZSB3ZXJlIGFibGUgdG8gcnVuIHRoZSBjb21wdXRhdGlvbnMgb24gYSBsb2NhbCBtYWNoaW5lIHdpdGggNC1jb3JlcyAoOCBsb2dpY2FsKSBhbmQgMzJHQiBSQU1zLg0KDQojIyMjIyBNb2RlbCBzcGVjaWZpY2F0aW9uDQoNCmBgYHtyfQ0Kc2V0LnNlZWQoMSkNCnN5c3RlbS50aW1lKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UIDwtIHZhcmltcChjZm9yZXN0KFNpbmdHZW1Wb2ljaW5nfmR1cmF0aW9uQzJfeitkdXJhdGlvblZvaWNlZFBlcmNDMl96K2R1cmF0aW9uVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVk9URGVjX3orZHVyYXRpb25CdXJzdF96K2R1cmF0aW9uVk9UX3orZHVyYXRpb25Wb2ljZWRQZXJjVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnRlbnNpdHlPbnNldFZPVEFsbF96K2ludGVuc2l0eU9mZnNldFZPVEFsbF96K2R1cmF0aW9uVjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYwT25zZXRWMl96K2ludGVuc2l0eU9uc2V0VjJfeitmMU9uc2V0VjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGgxbW5oMk9uc2V0Tm9ybVYyX3orZHVyYXRpb25WMV96K2YwT2Zmc2V0VjFfeitpbnRlbnNpdHlPZmZzZXRWMV96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZjFPZmZzZXRWMV96K2gxbW5oMk9mZnNldE5vcm1feiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGF0YSA9IGdlbWluYXRpb24udHJhaW4sIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbnRyb2w9Y2ZvcmVzdF91bmJpYXNlZChtdHJ5PXJvdW5kKHNxcnQoMTkpKSxudHJlZT01MDApKSwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNvbmRpdGlvbmFsID0gVCwgdGhyZXNob2xkID0gMC4yKSkNCnNvcnQodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpDQoNCmBgYA0KDQojIyMjIyBQbG90dGluZw0KDQpXZSBzdGFydCBieSBjaGFuZ2luZyBuYW1lcyBvZiBwcmVkaWN0b3JzIA0KDQpgYGB7cn0NCiMgV2UgY2hhbmdlIG5hbWVzDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdkdXJhdGlvbkMyX3onXSA8LSAnQ2xvc3VyZSBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdkdXJhdGlvblYyX3onXSA8LSAnVjIgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKSA9PSAnZHVyYXRpb25WMV96J10gPC0gJ1YxIER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ1ZPVERlY196J10gPC0gJ1ZPVCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ2R1cmF0aW9uVk9UQWxsX3onXSA8LSAnUmVsIER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ2R1cmF0aW9uQnVyc3RfeiddIDwtICdCdXJzdCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdkdXJhdGlvblZPVF96J10gPC0gJ0FzcCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdkdXJhdGlvblZvaWNlZFBlcmNDMl96J10gPC0gJ0NEIFZvaWNpbmcnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeiddIDwtICdSZWwgVm9pY2luZycNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ2ludGVuc2l0eU9mZnNldFYxX3onXSA8LSAnVjEgT2ZmIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ2ludGVuc2l0eU9uc2V0VjJfeiddIDwtICdWMiBPbiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeiddIDwtICdSZWwgT2YgSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKSA9PSAnaW50ZW5zaXR5T25zZXRWT1RBbGxfeiddIDwtICdSZWwgT24gSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKSA9PSAnZjBPbnNldFYyX3onXSA8LSAnVjIgT24gZjAnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdmME9mZnNldFYxX3onXSA8LSAnVjEgT2YgZjAnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpID09ICdmMU9uc2V0VjJfeiddIDwtICdWMiBPbiBGMScNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ2YxT2Zmc2V0VjFfeiddIDwtICdWMSBPZiBGMScNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVCkgPT0gJ2gxbW5oMk9mZnNldE5vcm1feiddIDwtICdWMSBPZiBIMSotSDIqJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UKSA9PSAnaDFtbmgyT25zZXROb3JtVjJfeiddIDwtICdWMiBPbiBIMSotSDIqJw0KDQpgYGANCg0KQWZ0ZXIgY2hhbmdpbmcgbmFtZXMsIHdlIGFwcGVuZCB0byBhIGRhdGEtZnJhbWUNCg0KYGBge3J9DQoNCnZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UX0RGIDwtIHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVF9ERiA8LSBhcy5kYXRhLmZyYW1lKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5UX0RGKQ0KdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblRfREYkdmFyIDwtIHJvdy5uYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVF9ERikNCg0KYGBgDQoNClRoZW4gd2UgdXNlIGdncGxvdDIgdG8gY3JlYXRlIG91ciBWYXJpYWJsZSBJbXBvcnRhbmNlIGZpZ3VyZQ0KDQpgYGB7ciBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD02fQ0KZ2dWYXJJbXBTR0R1clRfQSA8LSBnZ3Bsb3QodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblRfREYsIGFlcyhyZW9yZGVyKHZhcix2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluVF9ERiksdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpblRfREYpKSArIGdlb21fYmFyKHN0YXQ9ImlkZW50aXR5Iixwb3NpdGlvbj0iZG9kZ2UiLGZpbGwgPSAiZ3JheTcwIikgKyANCiAgY29vcmRfZmxpcCh5bGltID0gYygwLDAuMDE1KSkgKyAgbGFicyh5ID0gIiIsIHg9ICIiLCBzdWJ0aXRsZSA9ICJBOiBDb25kaXRpb25hbCAxOSBwcmVkaWN0b3JzIikrDQogIHNjYWxlX3lfY29udGludW91cygpK3RoZW1lX3NldCh0aGVtZV9idygpKSArIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKSArIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj0ibm9uZSIsIGF4aXMudGl0bGUueD1lbGVtZW50X2JsYW5rKCksYXhpcy50aXRsZS55PWVsZW1lbnRfYmxhbmsoKSkNCmdnVmFySW1wU0dEdXJUX0ENCmBgYA0KDQoNCkFzIGNhbiBiZSBzZWVuIGZyb20gdGhlIHJlc3VsdHMsIHRoZSBjbG9zdXJlIGR1cmF0aW9uIGlzIHRoZSBtb3N0IGltcG9ydGFudCBwcmVkaWN0b3IsIGZvbGxvd2VkIGJ5IHZvaWNpbmcgaW4gdGhlIGNsb3N1cmUsIGR1cmF0aW9uIG9mIFYxIGFuZCB2b2ljaW5nIGluIHRoZSByZWxlYXNlLiBNb3N0IG9mIHRoZSByZW1haW5pbmcgcHJlZGljdG9ycyBhcmUgY29udHJpYnV0aW5nIHRvIHRoZSBjb250cmFzdCB0byBhIGxlc3NlciBleHRlbnQgd2l0aCB0aGUgbGFzdCBzaXggbm90IGNvbnRyaWJ1dGluZyB0byBpdC4gDQoNCiMjIE1vZGVsIEINCg0KIyMjIFRyYWluaW5nIGFuZCBwcmVkaWN0aW5nDQoNCmBgYHtyfQ0Kc2V0LnNlZWQoMSkNCmNmb3Jlc3RHZW1Wb2ljaW5nVHJhaW5Ob0R1ciA8LSBjZm9yZXN0KFNpbmdHZW1Wb2ljaW5nfmR1cmF0aW9uVm9pY2VkUGVyY0MyX3orZHVyYXRpb25WT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVk9URGVjX3orZHVyYXRpb25CdXJzdF96K2R1cmF0aW9uVk9UX3orZHVyYXRpb25Wb2ljZWRQZXJjVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ZW5zaXR5T25zZXRWT1RBbGxfeitpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeitkdXJhdGlvblYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZjBPbnNldFYyX3oraW50ZW5zaXR5T25zZXRWMl96K2YxT25zZXRWMl96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGgxbW5oMk9uc2V0Tm9ybVYyX3orZHVyYXRpb25WMV96K2YwT2Zmc2V0VjFfeitpbnRlbnNpdHlPZmZzZXRWMV96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYxT2Zmc2V0VjFfeitoMW1uaDJPZmZzZXROb3JtX3osIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgZGF0YSA9IGdlbWluYXRpb24udHJhaW4sIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxOCkpLG50cmVlPTUwMCkpDQoNCg0KY2ZvcmVzdEdlbVZvaWNpbmdUZXN0Tm9EdXIgPC0gcHJlZGljdChjZm9yZXN0R2VtVm9pY2luZ1RyYWluTm9EdXIsIG5ld2RhdGEgPSBnZW1pbmF0aW9uLnRlc3QsIE9PQj1UUlVFLCB0eXBlID0gInJlc3BvbnNlIikNCg0KIyMgVGFibGUgcHJlZGljdGlvbnMgKHJvd3MpIGJ5IG9yaWdpbmFsIGRhdGEgKGNvbHVtbnMpIA0Kcm91bmQocHJvcC50YWJsZSh0YWJsZShjZm9yZXN0R2VtVm9pY2luZ1Rlc3ROb0R1cixnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmcpLDIpKjEwMCwxKQ0KDQojIHBlcmNlbnQgY29ycmVjdA0Kc3VtKGdlbWluYXRpb24udGVzdCRTaW5nR2VtVm9pY2luZz09Y2ZvcmVzdEdlbVZvaWNpbmdUZXN0Tm9EdXIpL25yb3coZ2VtaW5hdGlvbi50ZXN0KQ0KI2NvcnJlbGF0aW9ucw0KY29yKGFzLm51bWVyaWMoY2ZvcmVzdEdlbVZvaWNpbmdUZXN0Tm9EdXIpLCBhcy5udW1lcmljKGdlbWluYXRpb24udGVzdCRTaW5nR2VtVm9pY2luZykpDQoNCmBgYA0KDQpUaGUgcGVyY2VudCBjbGFzc2lmaWNhdGlvbiBhY2N1cmFjeSBvZiBNb2RlbCBCIGlzIGFyb3VuZCAxMCUgbG93ZXIgdGhhbiB0aGF0IG9mIE1vZGVsIEEuIFRoaXMgaXMgYSBjbGVhciBpbmRpY2F0aW9uIHRoYXQgdGhlIGNsb3N1cmUgZHVyYXRpb24gaGFzIGFuIGltcG9ydGFudCByb2xlIGluIGRpc2NyaW1pbmF0aW5nIHRoZSBmb3VyLXdheSBjb250cmFzdC4gDQoNCiMjIyBWYXJpYWJsZSBJbXBvcnRhbmNlDQoNCkFzIGJlZm9yZSB3ZSByYW4gdHdvIHZlcnNpb25zIG9mIHZhcmlhYmxlIGltcG9ydGFuY2U6IG5vbiBjb25kaXRpb25hbCBhbmQgY29uZGl0aW9uYWwNCg0KIyMjIyBObyBjb25kaXRpb25hbCBwZXJtdXRhdGlvbnMNCg0KIyMjIyMgTW9kZWwgc3BlY2lmaWNhdGlvbg0KDQpgYGB7cn0NCnNldC5zZWVkKDEpDQpzeXN0ZW0udGltZSh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVCA8LSB2YXJpbXAoY2ZvcmVzdChTaW5nR2VtVm9pY2luZ35kdXJhdGlvblZvaWNlZFBlcmNDMl96K2R1cmF0aW9uVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgVk9URGVjX3orZHVyYXRpb25CdXJzdF96K2R1cmF0aW9uVk9UX3orZHVyYXRpb25Wb2ljZWRQZXJjVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnRlbnNpdHlPbnNldFZPVEFsbF96K2ludGVuc2l0eU9mZnNldFZPVEFsbF96K2R1cmF0aW9uVjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYwT25zZXRWMl96K2ludGVuc2l0eU9uc2V0VjJfeitmMU9uc2V0VjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGgxbW5oMk9uc2V0Tm9ybVYyX3orZHVyYXRpb25WMV96K2YwT2Zmc2V0VjFfeitpbnRlbnNpdHlPZmZzZXRWMV96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZjFPZmZzZXRWMV96K2gxbW5oMk9mZnNldE5vcm1feiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBnZW1pbmF0aW9uLnRyYWluLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxOCkpLG50cmVlPTUwMCkpLCBjb25kaXRpb25hbCA9IEYpKQ0KDQpzb3J0KHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKQ0KDQpgYGANCg0KIyMjIyMgUGxvdHRpbmcNCg0KV2Ugc3RhcnQgYnkgY2hhbmdpbmcgbmFtZXMgb2YgcHJlZGljdG9ycyANCg0KYGBge3J9DQojIFdlIGNoYW5nZSBuYW1lcw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnZHVyYXRpb25WMl96J10gPC0gJ1YyIER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVCkgPT0gJ2R1cmF0aW9uVjFfeiddIDwtICdWMSBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdWT1REZWNfeiddIDwtICdWT1QnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdkdXJhdGlvblZPVEFsbF96J10gPC0gJ1JlbCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdkdXJhdGlvbkJ1cnN0X3onXSA8LSAnQnVyc3QgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnZHVyYXRpb25WT1RfeiddIDwtICdBc3AgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnZHVyYXRpb25Wb2ljZWRQZXJjQzJfeiddIDwtICdDRCBWb2ljaW5nJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnZHVyYXRpb25Wb2ljZWRQZXJjVk9UQWxsX3onXSA8LSAnUmVsIFZvaWNpbmcnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdpbnRlbnNpdHlPZmZzZXRWMV96J10gPC0gJ1YxIE9mZiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdpbnRlbnNpdHlPbnNldFYyX3onXSA8LSAnVjIgT24gSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnaW50ZW5zaXR5T2Zmc2V0Vk9UQWxsX3onXSA8LSAnUmVsIE9mIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVCkgPT0gJ2ludGVuc2l0eU9uc2V0Vk9UQWxsX3onXSA8LSAnUmVsIE9uIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVCkgPT0gJ2YwT25zZXRWMl96J10gPC0gJ1YyIE9uIGYwJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnZjBPZmZzZXRWMV96J10gPC0gJ1YxIE9mIGYwJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKSA9PSAnZjFPbnNldFYyX3onXSA8LSAnVjIgT24gRjEnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdmMU9mZnNldFYxX3onXSA8LSAnVjEgT2YgRjEnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlQpID09ICdoMW1uaDJPZmZzZXROb3JtX3onXSA8LSAnVjEgT2YgSDEqLUgyKicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVCkgPT0gJ2gxbW5oMk9uc2V0Tm9ybVYyX3onXSA8LSAnVjIgT24gSDEqLUgyKicNCg0KDQpgYGANCg0KQWZ0ZXIgY2hhbmdpbmcgbmFtZXMsIHdlIGFwcGVuZCB0byBhIGRhdGEtZnJhbWUNCg0KYGBge3J9DQoNCnZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UX0RGIDwtIHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVF9ERiA8LSBhcy5kYXRhLmZyYW1lKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UX0RGKQ0KdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlRfREYkdmFyIDwtIHJvdy5uYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOVF9ERikNCg0KYGBgDQoNClRoZW4gd2UgdXNlIGdncGxvdDIgdG8gY3JlYXRlIG91ciBWYXJpYWJsZSBJbXBvcnRhbmNlIGZpZ3VyZQ0KDQpgYGB7ciBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD02fQ0KZ2dWYXJJbXBTR0R1ck5UX0IgPC0gZ2dwbG90KHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UX0RGLCBhZXMocmVvcmRlcih2YXIsdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTlRfREYpLHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5UX0RGKSkgKyBnZW9tX2JhcihzdGF0PSJpZGVudGl0eSIscG9zaXRpb249ImRvZGdlIixmaWxsID0gImdyYXk3MCIpICsgDQogICAgICAgICAgICAgICAgICBjb29yZF9mbGlwKHlsaW0gPSBjKDAsMC4yNSkpICsgbGFicyhzdWJ0aXRsZSA9ICJCOiBOb24gQ29uZGl0aW9uYWwgMTggcHJlZGljdG9ycyIpICsgDQogICAgICAgICAgICAgICAgICBzY2FsZV95X2NvbnRpbnVvdXMoKSt0aGVtZV9zZXQodGhlbWVfYncoKSkgKyB0aGVtZSh0ZXh0PWVsZW1lbnRfdGV4dChzaXplPTIwKSkgKyB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiLCBheGlzLnRpdGxlLng9ZWxlbWVudF9ibGFuaygpLGF4aXMudGl0bGUueT1lbGVtZW50X2JsYW5rKCkpDQpnZ1ZhckltcFNHRHVyTlRfQg0KYGBgDQoNCg0KIyMjIyBDb25kaXRpb25hbCBwZXJtdXRhdGlvbnMNCg0KQXMgYWJvdmUsIHdlIHVzZSB0aGUgY29uZGl0aW9uYWwgcGVybXV0YXRpb25zIGFuZCBhZGp1c3QgdGhlIHRocmVzaG9sZC4gDQoNCiMjIyMjIE1vZGVsIHNwZWNpZmljYXRpb24NCg0KYGBge3J9DQpzZXQuc2VlZCgxKQ0Kc3lzdGVtLnRpbWUodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCA8LSB2YXJpbXAoY2ZvcmVzdChTaW5nR2VtVm9pY2luZ35kdXJhdGlvblZvaWNlZFBlcmNDMl96K2R1cmF0aW9uVk9UQWxsX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWT1REZWNfeitkdXJhdGlvbkJ1cnN0X3orZHVyYXRpb25WT1RfeitkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ZW5zaXR5T25zZXRWT1RBbGxfeitpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeitkdXJhdGlvblYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYwT25zZXRWMl96K2ludGVuc2l0eU9uc2V0VjJfeitmMU9uc2V0VjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaDFtbmgyT25zZXROb3JtVjJfeitkdXJhdGlvblYxX3orZjBPZmZzZXRWMV96K2ludGVuc2l0eU9mZnNldFYxX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYxT2Zmc2V0VjFfeitoMW1uaDJPZmZzZXROb3JtX3osICANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gZ2VtaW5hdGlvbi50cmFpbiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxOCkpLG50cmVlPTUwMCkpLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uZGl0aW9uYWwgPSBULCB0aHJlc2hvbGQgPSAwLjIpKQ0Kc29ydCh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKQ0KDQoNCmBgYA0KDQojIyMjIyBQbG90dGluZw0KDQpXZSBzdGFydCBieSBjaGFuZ2luZyBuYW1lcyBvZiBwcmVkaWN0b3JzIA0KDQpgYGB7cn0NCiMgV2UgY2hhbmdlIG5hbWVzDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKSA9PSAnZHVyYXRpb25WMl96J10gPC0gJ1YyIER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpID09ICdkdXJhdGlvblYxX3onXSA8LSAnVjEgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCkgPT0gJ1ZPVERlY196J10gPC0gJ1ZPVCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpID09ICdkdXJhdGlvblZPVEFsbF96J10gPC0gJ1JlbCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKSA9PSAnZHVyYXRpb25CdXJzdF96J10gPC0gJ0J1cnN0IER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpID09ICdkdXJhdGlvblZPVF96J10gPC0gJ0FzcCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKSA9PSAnZHVyYXRpb25Wb2ljZWRQZXJjQzJfeiddIDwtICdDRCBWb2ljaW5nJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCkgPT0gJ2R1cmF0aW9uVm9pY2VkUGVyY1ZPVEFsbF96J10gPC0gJ1JlbCBWb2ljaW5nJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCkgPT0gJ2ludGVuc2l0eU9mZnNldFYxX3onXSA8LSAnVjEgT2ZmIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpID09ICdpbnRlbnNpdHlPbnNldFYyX3onXSA8LSAnVjIgT24gSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCkgPT0gJ2ludGVuc2l0eU9mZnNldFZPVEFsbF96J10gPC0gJ1JlbCBPZiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKSA9PSAnaW50ZW5zaXR5T25zZXRWT1RBbGxfeiddIDwtICdSZWwgT24gSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCkgPT0gJ2YwT25zZXRWMl96J10gPC0gJ1YyIE9uIGYwJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVCkgPT0gJ2YwT2Zmc2V0VjFfeiddIDwtICdWMSBPZiBmMCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpID09ICdmMU9uc2V0VjJfeiddIDwtICdWMiBPbiBGMScNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clQpID09ICdmMU9mZnNldFYxX3onXSA8LSAnVjEgT2YgRjEnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKSA9PSAnaDFtbmgyT2Zmc2V0Tm9ybV96J10gPC0gJ1YxIE9mIEgxKi1IMionDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUKSA9PSAnaDFtbmgyT25zZXROb3JtVjJfeiddIDwtICdWMiBPbiBIMSotSDIqJw0KDQoNCmBgYA0KDQpBZnRlciBjaGFuZ2luZyBuYW1lcywgd2UgYXBwZW5kIHRvIGEgZGF0YS1mcmFtZQ0KDQpgYGB7cn0NCg0KdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVF9ERiA8LSB2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUX0RGIDwtIGFzLmRhdGEuZnJhbWUodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyVF9ERikNCnZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clRfREYkdmFyIDwtIHJvdy5uYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUX0RGKQ0KDQoNCmBgYA0KDQpUaGVuIHdlIHVzZSBnZ3Bsb3QyIHRvIGNyZWF0ZSBvdXIgVmFyaWFibGUgSW1wb3J0YW5jZSBmaWd1cmUNCg0KYGBge3IgZmlnLndpZHRoPTYsIGZpZy5oZWlnaHQ9Nn0NCmdnVmFySW1wU0dEdXJUX0IgPC0gZ2dwbG90KHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1clRfREYsIGFlcyhyZW9yZGVyKHZhcix2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUX0RGKSx2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJUX0RGKSkgKyBnZW9tX2JhcihzdGF0PSJpZGVudGl0eSIscG9zaXRpb249ImRvZGdlIixmaWxsID0gImdyYXk3MCIpICsgDQogIGNvb3JkX2ZsaXAoeWxpbSA9IGMoMCwwLjAxKSkgKyBsYWJzKHkgPSAiIiwgeD0gIiIsIHN1YnRpdGxlID0gIkI6IENvbmRpdGlvbmFsIDE4IHByZWRpY3RvcnMiKSArDQogIHNjYWxlX3lfY29udGludW91cygpK3RoZW1lX3NldCh0aGVtZV9idygpKSArIHRoZW1lKHRleHQ9ZWxlbWVudF90ZXh0KHNpemU9MjApKSArIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj0ibm9uZSIsIGF4aXMudGl0bGUueD1lbGVtZW50X2JsYW5rKCksYXhpcy50aXRsZS55PWVsZW1lbnRfYmxhbmsoKSkNCmdnVmFySW1wU0dEdXJUX0INCmBgYA0KDQoNCkFzIGNhbiBiZSBzZWVuIGZyb20gdGhlIHJlc3VsdHMsIHRoZSBWT1QgaXMgdGhlIG1vc3QgaW1wb3J0YW50IHByZWRpY3RvciwgZm9sbG93ZWQgYnkgdGhlIGR1cmF0aW9uIG9mIFYxLCB0aGUgdm9pY2luZyBpbiB0aGUgY2xvc3VyZSwgdGhlIGludGVuc2l0eSBhdCB0aGUgb25zZXQgb2YgdGhlIHJlbGVhc2UuIE1vc3Qgb2YgdGhlIHJlbWFpbmluZyBwcmVkaWN0b3JzIGFyZSBjb250cmlidXRpbmcgdG8gdGhlIGNvbnRyYXN0IHRvIGEgbGVzc2VyIGRlZ3JlZSB3aXRoIHRoZSBsYXN0IHRocmVlIG5vdCBjb250cmlidXRpbmcgdG8gaXQuDQoNCiMjIE1vZGVsIEMNCg0KIyMjIFRyYWluaW5nIGFuZCBwcmVkaWN0aW5nDQoNCmBgYHtyfQ0Kc2V0LnNlZWQoMSkNCmNmb3Jlc3RHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UIDwtIGNmb3Jlc3QoU2luZ0dlbVZvaWNpbmd+ZHVyYXRpb25Wb2ljZWRQZXJjQzJfeitkdXJhdGlvblZPVEFsbF96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkdXJhdGlvbkJ1cnN0X3orZHVyYXRpb25WT1RfeitkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ZW5zaXR5T25zZXRWT1RBbGxfeitpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeitkdXJhdGlvblYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYwT25zZXRWMl96K2ludGVuc2l0eU9uc2V0VjJfeitmMU9uc2V0VjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaDFtbmgyT25zZXROb3JtVjJfeitkdXJhdGlvblYxX3orZjBPZmZzZXRWMV96K2ludGVuc2l0eU9mZnNldFYxX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYxT2Zmc2V0VjFfeitoMW1uaDJPZmZzZXROb3JtX3osIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZGF0YSA9IGdlbWluYXRpb24udHJhaW4sIA0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxNykpLG50cmVlPTYwMCkpDQoNCg0KY2ZvcmVzdEdlbVZvaWNpbmdUZXN0Tm9EdXJOb1ZPVCA8LSBwcmVkaWN0KGNmb3Jlc3RHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9ULCBuZXdkYXRhID0gZ2VtaW5hdGlvbi50ZXN0LCBPT0I9VFJVRSwgdHlwZSA9ICJyZXNwb25zZSIpDQoNCiMjIFRhYmxlIHByZWRpY3Rpb25zIChyb3dzKSBieSBvcmlnaW5hbCBkYXRhIChjb2x1bW5zKSANCnJvdW5kKHByb3AudGFibGUodGFibGUoY2ZvcmVzdEdlbVZvaWNpbmdUZXN0Tm9EdXJOb1ZPVCxnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmcpLDIpKjEwMCwxKQ0KDQojIHBlcmNlbnQgY29ycmVjdA0Kc3VtKGdlbWluYXRpb24udGVzdCRTaW5nR2VtVm9pY2luZz09Y2ZvcmVzdEdlbVZvaWNpbmdUZXN0Tm9EdXJOb1ZPVCkvbnJvdyhnZW1pbmF0aW9uLnRlc3QpDQojIGNvcnJlbGF0aW9uDQpjb3IoYXMubnVtZXJpYyhjZm9yZXN0R2VtVm9pY2luZ1Rlc3ROb0R1ck5vVk9UKSwgYXMubnVtZXJpYyhnZW1pbmF0aW9uLnRlc3QkU2luZ0dlbVZvaWNpbmcpKQ0KDQoNCmBgYA0KDQpBcyBjYW4gYmUgc2VlbiBmcm9tIHRoZSBhYm92ZSwgdGhlIHBlcmNlbnQgY29ycmVjdCBpcyBtdWNoIGxvd2VyIHRoYXQgdGhlIHR3byBvdGhlciBtb2RlbHMgYW5kIHRoZSBsZXZlbHMgb2YgY29ycmVsYXRpb25zIGFzIHdlbGwuIFRoaXMgaW5kaWNhdGVzIHRoYXQgdGhlc2Ugc2Vjb25kYXJ5IGNvcnJlbGF0ZXMgYXJlIG5vdCBwcm92aWRpbmcgYSBjbGVhciBkaXNjcmltaW5hdGlvbiBvZiB0aGUgZGF0YS4gDQoNCiMjIyBWYXJpYWJsZSBJbXBvcnRhbmNlDQoNCkFzIGJlZm9yZSB3ZSByYW4gdHdvIHZlcnNpb25zIG9mIHZhcmlhYmxlIGltcG9ydGFuY2U6IG5vbiBjb25kaXRpb25hbCBhbmQgY29uZGl0aW9uYWwNCg0KIyMjIyBObyBjb25kaXRpb25hbCBwZXJtdXRhdGlvbnMNCg0KIyMjIyMgTW9kZWwgc3BlY2lmaWNhdGlvbg0KDQpgYGB7cn0NCnNldC5zZWVkKDEpDQpzeXN0ZW0udGltZSh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UIDwtIHZhcmltcChjZm9yZXN0KFNpbmdHZW1Wb2ljaW5nfmR1cmF0aW9uVm9pY2VkUGVyY0MyX3orZHVyYXRpb25WT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkdXJhdGlvbkJ1cnN0X3orZHVyYXRpb25WT1RfeitkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbnRlbnNpdHlPbnNldFZPVEFsbF96K2ludGVuc2l0eU9mZnNldFZPVEFsbF96K2R1cmF0aW9uVjJfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmME9uc2V0VjJfeitpbnRlbnNpdHlPbnNldFYyX3orZjFPbnNldFYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaDFtbmgyT25zZXROb3JtVjJfeitkdXJhdGlvblYxX3orZjBPZmZzZXRWMV96K2ludGVuc2l0eU9mZnNldFYxX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZjFPZmZzZXRWMV96K2gxbW5oMk9mZnNldE5vcm1feiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGEgPSBnZW1pbmF0aW9uLnRyYWluLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxNykpLG50cmVlPTYwMCkpLCBjb25kaXRpb25hbCA9IEYpKQ0KDQoNCnNvcnQodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkNCg0KYGBgDQoNCiMjIyMjIFBsb3R0aW5nDQoNCldlIHN0YXJ0IGJ5IGNoYW5naW5nIG5hbWVzIG9mIHByZWRpY3RvcnMgDQoNCmBgYHtyfQ0KIyBXZSBjaGFuZ2UgbmFtZXMNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpID09ICdkdXJhdGlvblYyX3onXSA8LSAnVjIgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkgPT0gJ2R1cmF0aW9uVjFfeiddIDwtICdWMSBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKSA9PSAnZHVyYXRpb25WT1RBbGxfeiddIDwtICdSZWwgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkgPT0gJ2R1cmF0aW9uQnVyc3RfeiddIDwtICdCdXJzdCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKSA9PSAnZHVyYXRpb25WT1RfeiddIDwtICdBc3AgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkgPT0gJ2R1cmF0aW9uVm9pY2VkUGVyY0MyX3onXSA8LSAnQ0QgVm9pY2luZycNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpID09ICdkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeiddIDwtICdSZWwgVm9pY2luZycNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpID09ICdpbnRlbnNpdHlPZmZzZXRWMV96J10gPC0gJ1YxIE9mZiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKSA9PSAnaW50ZW5zaXR5T25zZXRWMl96J10gPC0gJ1YyIE9uIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpID09ICdpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeiddIDwtICdSZWwgT2YgSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkgPT0gJ2ludGVuc2l0eU9uc2V0Vk9UQWxsX3onXSA8LSAnUmVsIE9uIEludCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpID09ICdmME9uc2V0VjJfeiddIDwtICdWMiBPbiBmMCcNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlQpID09ICdmME9mZnNldFYxX3onXSA8LSAnVjEgT2YgZjAnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKSA9PSAnZjFPbnNldFYyX3onXSA8LSAnVjIgT24gRjEnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UKSA9PSAnZjFPZmZzZXRWMV96J10gPC0gJ1YxIE9mIEYxJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkgPT0gJ2gxbW5oMk9mZnNldE5vcm1feiddIDwtICdWMSBPZiBIMSotSDIqJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVCkgPT0gJ2gxbW5oMk9uc2V0Tm9ybVYyX3onXSA8LSAnVjIgT24gSDEqLUgyKicNCg0KDQpgYGANCg0KQWZ0ZXIgY2hhbmdpbmcgbmFtZXMsIHdlIGFwcGVuZCB0byBhIGRhdGEtZnJhbWUNCg0KYGBge3J9DQoNCnZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlRfREYgPC0gdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVA0KdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVF9ERiA8LSBhcy5kYXRhLmZyYW1lKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlRfREYpDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVE5UX0RGJHZhciA8LSByb3cubmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVF9ERikNCg0KDQpgYGANCg0KVGhlbiB3ZSB1c2UgZ2dwbG90MiB0byBjcmVhdGUgb3VyIFZhcmlhYmxlIEltcG9ydGFuY2UgZmlndXJlDQoNCmBgYHtyIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTZ9DQpnZ1ZhckltcFNHRHVyTlRfQyA8LSBnZ3Bsb3QodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1ROVF9ERiwgYWVzKHJlb3JkZXIodmFyLHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlRfREYpLHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UTlRfREYpKSArIGdlb21fYmFyKHN0YXQ9ImlkZW50aXR5Iixwb3NpdGlvbj0iZG9kZ2UiLGZpbGwgPSAiZ3JheTcwIikgKyANCiAgY29vcmRfZmxpcCh5bGltID0gYygwLDAuMjUpKSArIGxhYnMoc3VidGl0bGUgPSAiQzogTm9uIENvbmRpdGlvbmFsIDE3IHByZWRpY3RvcnMiKSArIA0KICBzY2FsZV95X2NvbnRpbnVvdXMoKSt0aGVtZV9zZXQodGhlbWVfYncoKSkgKyB0aGVtZSh0ZXh0PWVsZW1lbnRfdGV4dChzaXplPTIwKSkgKyB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiLCBheGlzLnRpdGxlLng9ZWxlbWVudF9ibGFuaygpLGF4aXMudGl0bGUueT1lbGVtZW50X2JsYW5rKCkpDQpnZ1ZhckltcFNHRHVyTlRfQw0KYGBgDQoNCg0KIyMjIyBDb25kaXRpb25hbCBwZXJtdXRhdGlvbnMNCg0KQXMgYWJvdmUsIHdlIHVzZSB0aGUgY29uZGl0aW9uYWwgcGVybXV0YXRpb25zIGFuZCBhZGp1c3QgdGhlIHRocmVzaG9sZC4gDQoNCiMjIyMjIE1vZGVsIHNwZWNpZmljYXRpb24NCg0KYGBge3J9DQpzZXQuc2VlZCgxKQ0Kc3lzdGVtLnRpbWUodmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUIDwtIHZhcmltcChjZm9yZXN0KFNpbmdHZW1Wb2ljaW5nfmR1cmF0aW9uVm9pY2VkUGVyY0MyX3orZHVyYXRpb25WT1RBbGxfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGR1cmF0aW9uQnVyc3RfeitkdXJhdGlvblZPVF96K2R1cmF0aW9uVm9pY2VkUGVyY1ZPVEFsbF96Kw0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaW50ZW5zaXR5T25zZXRWT1RBbGxfeitpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeitkdXJhdGlvblYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmME9uc2V0VjJfeitpbnRlbnNpdHlPbnNldFYyX3orZjFPbnNldFYyX3orDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBoMW1uaDJPbnNldE5vcm1WMl96K2R1cmF0aW9uVjFfeitmME9mZnNldFYxX3oraW50ZW5zaXR5T2Zmc2V0VjFfeisNCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGYxT2Zmc2V0VjFfeitoMW1uaDJPZmZzZXROb3JtX3osICANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkYXRhID0gZ2VtaW5hdGlvbi50cmFpbiwgDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29udHJvbD1jZm9yZXN0X3VuYmlhc2VkKG10cnk9cm91bmQoc3FydCgxNykpLG50cmVlPTYwMCkpLCANCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29uZGl0aW9uYWwgPSBULCB0aHJlc2hvbGQgPSAwLjIpKQ0KDQpzb3J0KHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVCkNCg0KDQpgYGANCg0KIyMjIyMgUGxvdHRpbmcNCg0KV2Ugc3RhcnQgYnkgY2hhbmdpbmcgbmFtZXMgb2YgcHJlZGljdG9ycyANCg0KYGBge3J9DQojIFdlIGNoYW5nZSBuYW1lcw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdkdXJhdGlvblYyX3onXSA8LSAnVjIgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdkdXJhdGlvblYxX3onXSA8LSAnVjEgRHVyJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdkdXJhdGlvblZPVEFsbF96J10gPC0gJ1JlbCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVCkgPT0gJ2R1cmF0aW9uQnVyc3RfeiddIDwtICdCdXJzdCBEdXInDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVCkgPT0gJ2R1cmF0aW9uVk9UX3onXSA8LSAnQXNwIER1cicNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKSA9PSAnZHVyYXRpb25Wb2ljZWRQZXJjQzJfeiddIDwtICdDRCBWb2ljaW5nJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdkdXJhdGlvblZvaWNlZFBlcmNWT1RBbGxfeiddIDwtICdSZWwgVm9pY2luZycNCm5hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVClbbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKSA9PSAnaW50ZW5zaXR5T2Zmc2V0VjFfeiddIDwtICdWMSBPZmYgSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdpbnRlbnNpdHlPbnNldFYyX3onXSA8LSAnVjIgT24gSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdpbnRlbnNpdHlPZmZzZXRWT1RBbGxfeiddIDwtICdSZWwgT2YgSW50Jw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdpbnRlbnNpdHlPbnNldFZPVEFsbF96J10gPC0gJ1JlbCBPbiBJbnQnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVCkgPT0gJ2YwT25zZXRWMl96J10gPC0gJ1YyIE9uIGYwJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdmME9mZnNldFYxX3onXSA8LSAnVjEgT2YgZjAnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVCkgPT0gJ2YxT25zZXRWMl96J10gPC0gJ1YyIE9uIEYxJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdmMU9mZnNldFYxX3onXSA8LSAnVjEgT2YgRjEnDQpuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpW25hbWVzKHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVCkgPT0gJ2gxbW5oMk9mZnNldE5vcm1feiddIDwtICdWMSBPZiBIMSotSDIqJw0KbmFtZXModmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUKVtuYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFQpID09ICdoMW1uaDJPbnNldE5vcm1WMl96J10gPC0gJ1YyIE9uIEgxKi1IMionDQoNCg0KYGBgDQoNCkFmdGVyIGNoYW5naW5nIG5hbWVzLCB3ZSBhcHBlbmQgdG8gYSBkYXRhLWZyYW1lDQoNCmBgYHtyfQ0KDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYgPC0gdmFySW1wU2luZ0dlbVZvaWNpbmdUcmFpbk5vRHVyTm9WT1RUDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYgPC0gYXMuZGF0YS5mcmFtZSh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYpDQp2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYkdmFyIDwtIHJvdy5uYW1lcyh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYpDQoNCg0KDQpgYGANCg0KVGhlbiB3ZSB1c2UgZ2dwbG90MiB0byBjcmVhdGUgb3VyIFZhcmlhYmxlIEltcG9ydGFuY2UgZmlndXJlDQoNCmBgYHtyIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTZ9DQpnZ1ZhckltcFNHRHVyVF9DIDwtIGdncGxvdCh2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYsIGFlcyhyZW9yZGVyKHZhcix2YXJJbXBTaW5nR2VtVm9pY2luZ1RyYWluTm9EdXJOb1ZPVFRfREYpLHZhckltcFNpbmdHZW1Wb2ljaW5nVHJhaW5Ob0R1ck5vVk9UVF9ERikpICsgZ2VvbV9iYXIoc3RhdD0iaWRlbnRpdHkiLHBvc2l0aW9uPSJkb2RnZSIsZmlsbCA9ICJncmF5NzAiKSArIA0KICBjb29yZF9mbGlwKHlsaW0gPSBjKDAsMC4wMDEpKSArIGxhYnMoeSA9ICIiLCB4PSAiIiwgc3VidGl0bGUgPSAiQzogQ29uZGl0aW9uYWwgMTcgcHJlZGljdG9ycyIpICsNCiAgc2NhbGVfeV9jb250aW51b3VzKCkrdGhlbWVfc2V0KHRoZW1lX2J3KCkpICsgdGhlbWUodGV4dD1lbGVtZW50X3RleHQoc2l6ZT0yMCkpICsgdGhlbWUobGVnZW5kLnBvc2l0aW9uPSJub25lIiwgYXhpcy50aXRsZS54PWVsZW1lbnRfYmxhbmsoKSxheGlzLnRpdGxlLnk9ZWxlbWVudF9ibGFuaygpKQ0KZ2dWYXJJbXBTR0R1clRfQw0KYGBgDQoNCg0KQXMgY2FuIGJlIHNlZW4gZnJvbSB0aGUgcmVzdWx0cywgdGhlIHRoZSBkdXJhdGlvbnMgb2Ygc3Vycm91bmRpbmcgdm93ZWxzIGZvbGxvd2VkIGJ5IGludGVuc2l0eSBhdCB0aGUgb25zZXQgb2YgdGhlIHJlbGVhc2UsIHZvaWNpbmcgaW4gdGhlIGNsb3N1cmUgYW5kIGR1cmF0aW9uIG9mIHRoZSByZWxlYXNlIGFyZSB0aGUgbW9zdCBpbXBvcnRhbnQgcHJlZGljdG9ycy4gVGhlaXIgc2NvcmVzIGFyZSBtdWNoIGxvd2VyIHRoYW4gaW4gdGhlIG90aGVyIHR3byBtb2RlbHMgaW5kaWNhdGluZyB0aGVzZSBhcmUgc2Vjb25kYXJ5LiBNb3N0IG9mIHRoZSByZW1haW5pbmcgcHJlZGljdG9ycyBhcmUgY29udHJpYnV0aW5nIHRvIHRoZSBjb250cmFzdCB0byBhIGxlc3NlciBkZWdyZWUgd2l0aCB0aGUgbGFzdCBmb3VyIG5vdCBjb250cmlidXRpbmcgdG8gaXQuDQoNCiMgQ29uY2x1c2lvbg0KDQpUaGUgcmVzdWx0cyBvZiB0aGUgUmFuZG9tIEZvcmVzdHMgYXJlIGEgY2xlYXIgaW5kaWNhdGlvbiB0aGF0IHRoZSBjbG9zdXJlIGR1cmF0aW9uIGlzIHRoZSBtYWluIGNvcnJlbGF0ZSB0byB0aGUgZm91ci13YXkgdm9pY2luZyBieSBnZW1pbmF0aW9uIGNvbnRyYXN0LiBNb2RlbCBBIGhhZCBhbiBhY2N1cmFjeSBvZiA5Mi41JSB3aGljaCBpcyBhbiBleHRyZW1lbHkgaGlnaCBjbGFzc2lmaWNhdGlvbiByYXRlLiBUaGUgc2Vjb25kIG1vZGVsIHRoYXQgZXhjbHVkZXMgdGhlIGNsb3N1cmUgZHVyYXRpb24gaXMgcmVsYXRpdmVseSBoaWdoIGluIGNsYXNzaWZpY2F0aW9uIGFjY3VyYWN5IHlpZWxkaW5nIGFuIDgyLjMlLiBBIGRpZmZlcmVuY2Ugb2YgMTAuMiUgaXMgb2JzZXJ2ZWQgYmV0d2VlbiB0aGUgdHdvIG1vZGVscy4gVGhpcyBpcyBhIGNsZWFyIGluZGljYXRpb24gb2YgdGhlIHJvbGUgb2YgY2xvc3VyZSBkdXJhdGlvbiBpbiBib3RoIHZvaWNpbmcgYW5kIGdlbWluYXRpb24uIE1vZGVsIEMgYWltZWQgYXQgZXZhbHVhdGluZyB0aGUgcm9sZSBvZiBhbGwgKnNlY29uZGFyeSogcHJlZGljdG9ycyB3aXRob3V0IHVzaW5nIHRoZSBjbG9zdXJlIGR1cmF0aW9uIG9yIHRoZSBWT1QuIEEgcmVsYXRpdmVseSBsb3cgY2xhc3NpZmljYXRpb24gcmF0ZSBhdCA2Ny4yJSBpcyBpbmRpY2F0aXZlIG9mIGEgcmVsYXRpdmVseSBzdWNjZXNzZnVsIG1vZGVsLiBJdCBpcyBzdGF0aXN0aWNhbGx5IHNpZ25pZmljYW50IChyYXRlIGF0IDI1JSBhbmQgYWJvdmUpLCBidXQgdGhlIHByZWRpY3RpdmUgYWNjdXJhY3kgaXMgbm90IGhpZ2ggZW5vdWdoLiBUaGlzIGlzIGNsZWFybHkgaW5kaWNhdGluZyB0aGF0IHRoZXNlIGNvcnJlbGF0ZXMgYWN0IGFzIHNlY29uZGFyeSBvbmUgYW5kIGhhdmUgYW4gZW5oYW5jZW1lbnQgcm9sZSB0byB0aGUgcHJpbWFyeSBjb3JyZWxhdGU7IGNsb3N1cmUgZHVyYXRpb24uIA==