🕷️ Crawler Inspector

[HDDA21](https://statomics.github.io/HDDA21/index.html) - [About](https://statomics.github.io/HDDA21/about.html) - [Lectures](https://statomics.github.io/HDDA21/lsGeometricInterpretation.html) - [1\. Introduction](https://statomics.github.io/HDDA21/intro.html) - [2\. Singular Value Decomposition](https://statomics.github.io/HDDA21/svd.html) - [2\.3. Geometric Interpretation SVD](https://statomics.github.io/HDDA21/svdGeometricInterpretation.html) - [2\.7. Link MDS and Gram Distance Matrix](https://statomics.github.io/HDDA21/MDS_linkGramDistanceMatrix.html) - [3\. Prediction with High Dimensional Predictors](https://statomics.github.io/HDDA21/prediction.html) - [4\. Sparse Singular Value Decomposition](https://statomics.github.io/HDDA21/sparseSvd.html) - [5\. Linear Discriminant Analysis](https://statomics.github.io/HDDA21/lda.html) - [6\. Large Scale Inference](https://statomics.github.io/HDDA21/lsi.html) - [Paper 1: Model-based Clustering](https://sites.stat.washington.edu/people/raftery/Research/PDF/fraley1998.pdf) - [Paper 1: Intro Hierarchical Clustering](https://statomics.github.io/HDDA21/hclust.html) - [Paper 1: EM algorithm](https://statomics.github.io/HDDA21/em.html) - [Labs](https://statomics.github.io/HDDA21/lsGeometricInterpretation.html) - [Lab 1](https://statomics.github.io/HDDA21/Lab1-Intro-SVD.html) - [Lab 2](https://statomics.github.io/HDDA21/Lab2-PCA.html) - [Lab 3](https://statomics.github.io/HDDA21/Lab3-Penalized-Regression.html) - [Lab 4](https://statomics.github.io/HDDA21/Lab4-Sparse-PCA-LDA.html) - [Lab 5](https://statomics.github.io/HDDA21/Lab5-Large-Scale-Inference.html) - [statOmics](http://statomics.github.io/) Code - [Show All Code](https://statomics.github.io/HDDA21/lsGeometricInterpretation.html) - [Hide All Code](https://statomics.github.io/HDDA21/lsGeometricInterpretation.html) - [Download Rmd](https://statomics.github.io/HDDA21/lsGeometricInterpretation.html) # 1\. Introduction: Linear Regression - Geometric interpretation #### Lieven Clement #### statOmics, Ghent University (<https://statomics.github.io>) # 1 Intro ## 1\.1 Data Dataset with 3 observation (X,Y): ``` library(tidyverse) ``` ``` ## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ── ``` ``` ## ✔ ggplot2 3.3.5 ✔ purrr 0.3.4 ## ✔ tibble 3.1.5 ✔ dplyr 1.0.7 ## ✔ tidyr 1.1.4 ✔ stringr 1.4.0 ## ✔ readr 2.0.1 ✔ forcats 0.5.1 ``` ``` ## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ── ## ✖ dplyr::filter() masks stats::filter() ## ✖ dplyr::lag() masks stats::lag() ``` ``` data <- data.frame(x=1:3,y=c(1,2,2)) data ``` ## 1\.2 Model \\\[ y\_i=\\beta\_0+\\beta\_1x + \\epsilon\_i \\\] If we write the model for each observation: \\\[ \\begin{array} {lcl} 1 &=& \\beta\_0+\\beta\_1 1 + \\epsilon\_1 \\\\ 2 &=& \\beta\_0+\\beta\_1 2 + \\epsilon\_2 \\\\ 2 &=& \\beta\_0+\\beta\_1 2 + \\epsilon\_3 \\\\ \\end{array} \\\] We can also write this in matrix form \\\[ \\mathbf{Y} = \\mathbf{X}\\boldsymbol{\\beta}+\\boldsymbol{\\epsilon} \\\] with \\\[ \\mathbf{Y}=\\left\[ \\begin{array}{c} 1\\\\ 2\\\\ 2\\\\ \\end{array}\\right\], \\quad \\mathbf{X}= \\left\[ \\begin{array}{cc} 1&1\\\\ 1&2\\\\ 1&3\\\\ \\end{array} \\right\], \\quad \\boldsymbol{\\beta} = \\left\[ \\begin{array}{c} \\beta\_0\\\\ \\beta\_1\\\\ \\end{array} \\right\] \\quad \\text{and} \\quad \\boldsymbol{\\epsilon}= \\left\[ \\begin{array}{c} \\epsilon\_1\\\\ \\epsilon\_2\\\\ \\epsilon\_3 \\end{array} \\right\] \\\] ``` lm1 <- lm(y~x,data) data$yhat <- lm1$fitted data %>% ggplot(aes(x,y)) + geom_point() + ylim(0,4) + xlim(0,4) + stat_smooth(method = "lm", color = "red", fullrange = TRUE) + geom_point(aes(x=x, y =yhat), pch = 2, size = 3, color = "red") + geom_segment(data = data, aes(x = x, xend = x, y = y, yend = yhat), lty = 2 ) ``` ``` ## `geom_smooth()` using formula 'y ~ x' ``` ``` ## Warning in max(ids, na.rm = TRUE): no non-missing arguments to max; returning ## -Inf ```  # 2 Least Squares (LS) - Minimize the residual sum of squares \\\[\\begin{eqnarray\*} RSS(\\boldsymbol{\\beta})&=&\\sum\\limits\_{i=1}^n e^2\_i\\\\ &=&\\sum\\limits\_{i=1}^n \\left(y\_i-\\beta\_0-\\sum\\limits\_{j=1}^p x\_{ij}\\beta\_j\\right)^2 \\end{eqnarray\*}\\\] - or in matrix notation \\\[\\begin{eqnarray\*} RSS(\\boldsymbol{\\beta})&=&(\\mathbf{Y}-\\mathbf{X\\beta})^T(\\mathbf{Y}-\\mathbf{X\\beta})\\\\ &=&\\Vert \\mathbf{Y}-\\mathbf{X\\beta}\\Vert^2\_2 \\end{eqnarray\*}\\\] with the \\(L\_2\\)\-norm of a \\(p\\)\-dim. vector \\(v\\) \\(\\Vert \\mathbf{v} \\Vert\_2=\\sqrt{v\_1^2+\\ldots+v\_p^2}\\) \\(\\rightarrow\\) \\(\\hat{\\boldsymbol{\\beta}}=\\text{argmin}\_\\beta \\Vert \\mathbf{Y}-\\mathbf{X\\beta}\\Vert^2\\) ## 2\.1 Minimize RSS \\\[ \\begin{array}{ccc} \\frac{\\partial RSS}{\\partial \\boldsymbol{\\beta}}&=&\\mathbf{0}\\\\\\\\ \\frac{(\\mathbf{Y}-\\mathbf{X\\beta})^T(\\mathbf{Y}-\\mathbf{X}\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}}&=&\\mathbf{0}\\\\\\\\ -2\\mathbf{X}^T(\\mathbf{Y}-\\mathbf{X}\\boldsymbol{\\beta})&=&\\mathbf{0}\\\\\\\\ \\mathbf{X}^T\\mathbf{X\\beta}&=&\\mathbf{X}^T\\mathbf{Y}\\\\\\\\ \\hat{\\boldsymbol{\\beta}}&=&(\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{Y} \\end{array} \\\] \\\[ \\hat{\\boldsymbol{\\beta}}=(\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{Y} \\\] ## 2\.2 Fitted values: \\\[ \\begin{array}{lcl} \\hat{\\mathbf{Y}} &=& \\mathbf{X}\\hat{\\boldsymbol{\\beta}}\\\\ &=& \\mathbf{X} (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{Y}\\\\ \\end{array} \\\] # 3 Geometric interpretation of Linear regression There is also another picture to interpret linear regression\! Linear regression can also be seen as a projection\! Fitted values: \\\[ \\begin{array}{lcl} \\hat{\\mathbf{Y}} &=& \\mathbf{X}\\hat{\\boldsymbol{\\beta}}\\\\ &=& \\mathbf{X} (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{Y}\\\\ &=& \\mathbf{HY} \\end{array} \\\] with \\(\\mathbf{H}\\) the projection matrix also referred to as the hat matrix. ``` X <- model.matrix(~x,data) X ``` ``` ## (Intercept) x ## 1 1 1 ## 2 1 2 ## 3 1 3 ## attr(,"assign") ## [1] 0 1 ``` ``` XtX <- t(X)%*%X XtX ``` ``` ## (Intercept) x ## (Intercept) 3 6 ## x 6 14 ``` ``` XtXinv <- solve(t(X)%*%X) XtXinv ``` ``` ## (Intercept) x ## (Intercept) 2.333333 -1.0 ## x -1.000000 0.5 ``` ``` H <- X %*% XtXinv %*% t(X) H ``` ``` ## 1 2 3 ## 1 0.8333333 0.3333333 -0.1666667 ## 2 0.3333333 0.3333333 0.3333333 ## 3 -0.1666667 0.3333333 0.8333333 ``` ``` Y <- data$y Yhat <- H%*%Y Yhat ``` ``` ## [,1] ## 1 1.166667 ## 2 1.666667 ## 3 2.166667 ``` ## 3\.1 What do these projections mean geometrically? The other picture to linear regression is to consider \\(X\_0\\), \\(X\_1\\) and \\(Y\\) as vectors in the space of the data \\(\\mathbb{R}^n\\), here \\(\\mathbb{R}^3\\) because we have three data points. ``` originRn <- data.frame(X1=0,X2=0,X3=0) data$x0 <- 1 dataRn <- data.frame(t(data)) library(plotly) ``` ``` ## ## Attaching package: 'plotly' ``` ``` ## The following object is masked from 'package:ggplot2': ## ## last_plot ``` ``` ## The following object is masked from 'package:stats': ## ## filter ``` ``` ## The following object is masked from 'package:graphics': ## ## layout ``` ``` p1 <- plot_ly( originRn, x = ~ X1, y = ~ X2, z= ~ X3) %>% add_markers(type="scatter3d") %>% layout( scene = list( aspectmode="cube", xaxis = list(range=c(-4,4)), yaxis = list(range=c(-4,4)), zaxis = list(range=c(-4,4)) ) ) p1 <- p1 %>% add_trace( x = c(0,1), y = c(0,0), z = c(0,0), mode = "lines", line = list(width = 5, color = "grey"), type="scatter3d") %>% add_trace( x = c(0,0), y = c(0,1), z = c(0,0), mode = "lines", line = list(width = 5, color = "grey"), type="scatter3d") %>% add_trace( x = c(0,0), y = c(0,0), z = c(0,1), mode = "lines", line = list(width = 5, color = "grey"), type="scatter3d") %>% add_trace( x = c(0,1), y = c(0,1), z = c(0,1), mode = "lines", line = list(width = 5, color = "black"), type="scatter3d") %>% add_trace( x = c(0,1), y = c(0,2), z = c(0,3), mode = "lines", line = list(width = 5, color = "black"), type="scatter3d") ``` ``` p2 <- p1 %>% add_trace( x = c(0,Y[1]), y = c(0,Y[2]), z = c(0,Y[3]), mode = "lines", line = list(width = 5, color = "red"), type="scatter3d") %>% add_trace( x = c(0,Yhat[1]), y = c(0,Yhat[2]), z = c(0,Yhat[3]), mode = "lines", line = list(width = 5, color = "red"), type="scatter3d") %>% add_trace( x = c(Y[1],Yhat[1]), y = c(Y[2],Yhat[2]), z = c(Y[3],Yhat[3]), mode = "lines", line = list(width = 5, color = "red", dash="dash"), type="scatter3d") p2 ``` ### 3\.1.1 How does this projection work? \\\[ \\begin{array}{lcl} \\hat{\\mathbf{Y}} &=& \\mathbf{X} (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^T\\mathbf{Y}\\\\ &=& \\mathbf{X}(\\mathbf{X}^T\\mathbf{X})^{-1/2}(\\mathbf{X}^T\\mathbf{X})^{-1/2}\\mathbf{X}^T\\mathbf{Y}\\\\ &=& \\mathbf{U}\\mathbf{U}^T\\mathbf{Y} \\end{array} \\\] - \\(\\mathbf{U}\\) is a new orthonormal basis in \\(\\mathbb{R}^2\\), a subspace of \\(\\mathbb{R}^3\\) - The space spanned by U and X is the column space of X, e.g. it contains all possible linear combinantions of X. \\(\\mathbf{U}^t\\mathbf{Y}\\) is the projection of Y on this new orthonormal basis ``` eigenXtX <- eigen(XtX) XtXinvSqrt <- eigenXtX$vectors %*%diag(1/eigenXtX$values^.5)%*%t(eigenXtX$vectors) U <- X %*% XtXinvSqrt p3 <- p1 %>% add_trace( x = c(0,U[1,1]), y = c(0,U[2,1]), z = c(0,U[3,1]), mode = "lines", line = list(width = 5, color = "blue"), type="scatter3d") %>% add_trace( x = c(0,U[1,2]), y = c(0,U[2,2]), z = c(0,U[3,2]), mode = "lines", line = list(width = 5, color = "blue"), type="scatter3d") p3 ``` - \\(\\mathbf{U}^T\\mathbf{Y}\\) is the projection of \\(\\mathbf{Y}\\) in the space spanned by \\(\\mathbf{U}\\). - Indeed \\(\\mathbf{U}\_1^T\\mathbf{Y}\\) ``` p4 <- p3 %>% add_trace( x = c(0,Y[1]), y = c(0,Y[2]), z = c(0,Y[3]), mode = "lines", line = list(width = 5, color = "red"), type="scatter3d") %>% add_trace( x = c(0,U[1,1]*(U[,1]%*%Y)), y = c(0,U[2,1]*(U[,1]%*%Y)), z = c(0,U[3,1]*(U[,1]%*%Y)), mode = "lines", line = list(width = 5, color = "red",dash="dash"), type="scatter3d") %>% add_trace( x = c(Y[1],U[1,1]*(U[,1]%*%Y)), y = c(Y[2],U[2,1]*(U[,1]%*%Y)), z = c(Y[3],U[3,1]*(U[,1]%*%Y)), mode = "lines", line = list(width = 5, color = "red", dash="dash"), type="scatter3d") p4 ``` - and \\(\\mathbf{U}\_2^T\\mathbf{Y}\\) ``` p5 <- p4 %>% add_trace( x = c(0,Y[1]), y = c(0,Y[2]), z = c(0,Y[3]), mode = "lines", line = list(width = 5, color = "red"), type="scatter3d") %>% add_trace( x = c(0,U[1,2]*(U[,2]%*%Y)), y = c(0,U[2,2]*(U[,2]%*%Y)), z = c(0,U[3,2]*(U[,2]%*%Y)), mode = "lines", line = list(width = 5, color = "red",dash="dash"), type="scatter3d") %>% add_trace( x = c(Y[1],U[1,2]*(U[,2]%*%Y)), y = c(Y[2],U[2,2]*(U[,2]%*%Y)), z = c(Y[3],U[3,2]*(U[,2]%*%Y)), mode = "lines", line = list(width = 5, color = "red", dash="dash"), type="scatter3d") p5 ``` ``` p6 <- p5 %>% add_trace( x = c(0,Yhat[1]), y = c(0,Yhat[2]), z = c(0,Yhat[3]), mode = "lines", line = list(width = 5, color = "red"), type="scatter3d") %>% add_trace( x = c(Y[1],Yhat[1]), y = c(Y[2],Yhat[2]), z = c(Y[3],Yhat[3]), mode = "lines", line = list(width = 5, color = "red", dash="dash"), type="scatter3d") %>% add_trace( x = c(U[1,1]*(U[,1]%*%Y),Yhat[1]), y = c(U[2,1]*(U[,1]%*%Y),Yhat[2]), z = c(U[3,1]*(U[,1]%*%Y),Yhat[3]), mode = "lines", line = list(width = 5, color = "red", dash="dash"), type="scatter3d") %>% add_trace( x = c(U[1,2]*(U[,2]%*%Y),Yhat[1]), y = c(U[2,2]*(U[,2]%*%Y),Yhat[2]), z = c(U[3,2]*(U[,2]%*%Y),Yhat[3]), mode = "lines", line = list(width = 5, color = "red", dash="dash"), type="scatter3d") p6 ``` ## 3\.2 The Error vector Note, that it is also clear from the equation in the derivation of the least squares solution that the residual is orthogonal on the column space: \\\[ -2 \\mathbf{X}^T(\\mathbf{Y}-\\mathbf{X}\\boldsymbol{\\beta}) = 0 \\\] # 4 Curse of dimensionality? - Imagine what happens when p approaches n \\(p=n\\) or becomes much larger than p \>\> n!!\! - Suppose that we add a predictor \\(\\mathbf{X}\_2 = \[2,0,1\]^T\\)? \\\[ \\mathbf{Y}=\\left\[ \\begin{array}{c} 1\\\\ 2\\\\ 2\\\\ \\end{array}\\right\], \\quad \\mathbf{X}= \\left\[ \\begin{array}{ccc} 1&1&2\\\\ 1&2&0\\\\ 1&3&1\\\\ \\end{array} \\right\], \\quad \\boldsymbol{\\beta} = \\left\[ \\begin{array}{c} \\beta\_0\\\\ \\beta\_1\\\\ \\beta\_2 \\end{array} \\right\] \\quad \\text{and} \\quad \\boldsymbol{\\epsilon}= \\left\[ \\begin{array}{c} \\epsilon\_1\\\\ \\epsilon\_2\\\\ \\epsilon\_3 \\end{array} \\right\] \\\] ``` data$x2 <- c(2,0,1) fit <- lm(y~x+x2,data) # predict values on regular xy grid x1pred <- seq(-1, 4, length.out = 10) x2pred <- seq(-1, 4, length.out = 10) xy <- expand.grid(x = x1pred, x2 = x2pred) ypred <- matrix (nrow = 30, ncol = 30, data = predict(fit, newdata = data.frame(xy))) library(plot3D) ``` ``` ## Warning: no DISPLAY variable so Tk is not available ``` ``` # fitted points for droplines to surface th=20 ph=5 scatter3D(data$x, data$x2, Y, pch = 16, col="darkblue", cex = 1, theta = th, ticktype = "detailed", xlab = "x1", ylab = "x2", zlab = "y", colvar=FALSE, bty = "g", xlim=c(-1,3), ylim=c(-1,3), zlim=c(-2,4)) ```  ``` z.pred3D <- outer( x1pred, x2pred, function(x1,x2) { fit$coef[1] + fit$coef[2]*x1+fit$coef[2]*x2 }) x.pred3D <- outer( x1pred, x2pred, function(x,y) x) y.pred3D <- outer( x1pred, x2pred, function(x,y) y) scatter3D(data$x, data$x2, data$y, pch = 16, col="darkblue", cex = 1, theta = th, ticktype = "detailed", xlab = "x1", ylab = "x2", zlab = "y", colvar=FALSE, bty = "g", xlim=c(-1,4), ylim=c(-1,4), zlim=c(-2,4)) surf3D( x.pred3D, y.pred3D, z.pred3D, col="blue", facets=NA, add=TRUE) ```  Note, that the linear regression is now a plane. However, we obtain a perfect fit and all the data points are falling in the plane! 😱 This is obvious if we look at the column space of X\! ``` X <- cbind(X,c(2,0,1)) XtX <- t(X)%*%X eigenXtX <- eigen(XtX) XtXinvSqrt <- eigenXtX$vectors %*%diag(1/eigenXtX$values^.5)%*%t(eigenXtX$vectors) U <- X %*% XtXinvSqrt p7 <- p1 %>% add_trace( x = c(0,2), y = c(0,0), z = c(0,1), mode = "lines", line = list(width = 5, color = "darkgreen"), type="scatter3d") p7 ``` ``` p8 <- p7 %>% add_trace( x = c(0,U[1,1]), y = c(0,U[2,1]), z = c(0,U[3,1]), mode = "lines", line = list(width = 5, color = "blue"), type="scatter3d") %>% add_trace( x = c(0,U[1,2]), y = c(0,U[2,2]), z = c(0,U[3,2]), mode = "lines", line = list(width = 5, color = "blue"), type="scatter3d") %>% add_trace( x = c(0,U[1,3]), y = c(0,U[2,3]), z = c(0,U[3,3]), mode = "lines", line = list(width = 5, color = "blue"), type="scatter3d") p8 ``` - The column space now spans the entire \\(\\mathbb{R}^3\\)\! - With the intercept and the two predictors we can thus fit every dataset that only has 3 observations for the predictors and the response. - So the model can no longer be used to generalise the patterns seen in the data towards the population (new observations). - Problem of overfitting!!\! - If \\(p \>\> n\\) then the problem gets even worse! Then there is even no longer a unique solution to the least squares problem… LS0tCnRpdGxlOiAnMS4gSW50cm9kdWN0aW9uOiBMaW5lYXIgUmVncmVzc2lvbiAtIEdlb21ldHJpYyBpbnRlcnByZXRhdGlvbicKYXV0aG9yOiAiTGlldmVuIENsZW1lbnQiCmRhdGU6ICJzdGF0T21pY3MsIEdoZW50IFVuaXZlcnNpdHkgKGh0dHBzOi8vc3RhdG9taWNzLmdpdGh1Yi5pbykiCmFsd2F5c19hbGxvd19odG1sOiB5ZXMKLS0tCgojIEludHJvCiMjIERhdGEKCkRhdGFzZXQgd2l0aCAzIG9ic2VydmF0aW9uIChYLFkpOgoKYGBge3J9CmxpYnJhcnkodGlkeXZlcnNlKQpkYXRhIDwtIGRhdGEuZnJhbWUoeD0xOjMseT1jKDEsMiwyKSkKZGF0YQpgYGAKCiMjIE1vZGVsCgokJAp5X2k9XGJldGFfMCtcYmV0YV8xeCArIFxlcHNpbG9uX2kKJCQKCklmIHdlIHdyaXRlIHRoZSBtb2RlbCBmb3IgZWFjaCBvYnNlcnZhdGlvbjoKCiQkClxiZWdpbnthcnJheX0ge2xjbH0KMSAmPSYgXGJldGFfMCtcYmV0YV8xIDEgKyBcZXBzaWxvbl8xIFxcCjIgJj0mIFxiZXRhXzArXGJldGFfMSAyICsgXGVwc2lsb25fMiBcXAoyICY9JiBcYmV0YV8wK1xiZXRhXzEgMiArIFxlcHNpbG9uXzMgXFwKXGVuZHthcnJheX0KJCQKCldlIGNhbiBhbHNvIHdyaXRlIHRoaXMgaW4gbWF0cml4IGZvcm0KCiQkClxtYXRoYmZ7WX0gPSBcbWF0aGJme1h9XGJvbGRzeW1ib2x7XGJldGF9K1xib2xkc3ltYm9se1xlcHNpbG9ufQokJAoKd2l0aAoKJCQKXG1hdGhiZntZfT1cbGVmdFsKXGJlZ2lue2FycmF5fXtjfQoxXFwKMlxcCjJcXApcZW5ke2FycmF5fVxyaWdodF0sClxxdWFkClxtYXRoYmZ7WH09IFxsZWZ0WwpcYmVnaW57YXJyYXl9e2NjfQoxJjFcXAoxJjJcXAoxJjNcXApcZW5ke2FycmF5fQpccmlnaHRdLApccXVhZCBcYm9sZHN5bWJvbHtcYmV0YX0gPSBcbGVmdFsKXGJlZ2lue2FycmF5fXtjfQpcYmV0YV8wXFwKXGJldGFfMVxcClxlbmR7YXJyYXl9ClxyaWdodF0KXHF1YWQKXHRleHR7YW5kfQpccXVhZApcYm9sZHN5bWJvbHtcZXBzaWxvbn09ClxsZWZ0WwpcYmVnaW57YXJyYXl9e2N9ClxlcHNpbG9uXzFcXApcZXBzaWxvbl8yXFwKXGVwc2lsb25fMwpcZW5ke2FycmF5fQpccmlnaHRdCiQkCgpgYGB7cn0KbG0xIDwtIGxtKHl+eCxkYXRhKQpkYXRhJHloYXQgPC0gbG0xJGZpdHRlZAoKZGF0YSAlPiUKICBnZ3Bsb3QoYWVzKHgseSkpICsKICBnZW9tX3BvaW50KCkgKwogIHlsaW0oMCw0KSArCiAgeGxpbSgwLDQpICsKICBzdGF0X3Ntb290aChtZXRob2QgPSAibG0iLCBjb2xvciA9ICJyZWQiLCBmdWxscmFuZ2UgPSBUUlVFKSArCiAgZ2VvbV9wb2ludChhZXMoeD14LCB5ID15aGF0KSwgcGNoID0gMiwgc2l6ZSA9IDMsIGNvbG9yID0gInJlZCIpICsKICBnZW9tX3NlZ21lbnQoZGF0YSA9IGRhdGEsIGFlcyh4ID0geCwgeGVuZCA9IHgsIHkgPSB5LCB5ZW5kID0geWhhdCksIGx0eSA9IDIgKQpgYGAKCiMgTGVhc3QgU3F1YXJlcyAoTFMpCgotIE1pbmltaXplIHRoZSByZXNpZHVhbCBzdW0gb2Ygc3F1YXJlcwpcYmVnaW57ZXFuYXJyYXkqfQpSU1MoXGJvbGRzeW1ib2x7XGJldGF9KSY9JlxzdW1cbGltaXRzX3tpPTF9Xm4gZV4yX2lcXAomPSZcc3VtXGxpbWl0c197aT0xfV5uIFxsZWZ0KHlfaS1cYmV0YV8wLVxzdW1cbGltaXRzX3tqPTF9XnAgeF97aWp9XGJldGFfalxyaWdodCleMgpcZW5ke2VxbmFycmF5Kn0KCi0gb3IgaW4gbWF0cml4IG5vdGF0aW9uClxiZWdpbntlcW5hcnJheSp9ClJTUyhcYm9sZHN5bWJvbHtcYmV0YX0pJj0mKFxtYXRoYmZ7WX0tXG1hdGhiZntYXGJldGF9KV5UKFxtYXRoYmZ7WX0tXG1hdGhiZntYXGJldGF9KVxcCiY9JlxWZXJ0IFxtYXRoYmZ7WX0tXG1hdGhiZntYXGJldGF9XFZlcnReMl8yClxlbmR7ZXFuYXJyYXkqfQp3aXRoIHRoZSAkTF8yJC1ub3JtIG9mIGEgJHAkLWRpbS4gdmVjdG9yICR2JCAkXFZlcnQgXG1hdGhiZnt2fSBcVmVydF8yPVxzcXJ0e3ZfMV4yK1xsZG90cyt2X3BeMn0kCiRccmlnaHRhcnJvdyQgJFxoYXR7XGJvbGRzeW1ib2x7XGJldGF9fT1cdGV4dHthcmdtaW59X1xiZXRhIFxWZXJ0IFxtYXRoYmZ7WX0tXG1hdGhiZntYXGJldGF9XFZlcnReMiQKCgojIyBNaW5pbWl6ZSBSU1MKXFsKXGJlZ2lue2FycmF5fXtjY2N9ClxmcmFje1xwYXJ0aWFsIFJTU317XHBhcnRpYWwgXGJvbGRzeW1ib2x7XGJldGF9fSY9JlxtYXRoYmZ7MH1cXFxcClxmcmFjeyhcbWF0aGJme1l9LVxtYXRoYmZ7WFxiZXRhfSleVChcbWF0aGJme1l9LVxtYXRoYmZ7WH1cYm9sZHN5bWJvbHtcYmV0YX0pfXtccGFydGlhbCBcYm9sZHN5bWJvbHtcYmV0YX19Jj0mXG1hdGhiZnswfVxcXFwKLTJcbWF0aGJme1h9XlQoXG1hdGhiZntZfS1cbWF0aGJme1h9XGJvbGRzeW1ib2x7XGJldGF9KSY9JlxtYXRoYmZ7MH1cXFxcClxtYXRoYmZ7WH1eVFxtYXRoYmZ7WFxiZXRhfSY9JlxtYXRoYmZ7WH1eVFxtYXRoYmZ7WX1cXFxcClxoYXR7XGJvbGRzeW1ib2x7XGJldGF9fSY9JihcbWF0aGJme1h9XlRcbWF0aGJme1h9KV57LTF9XG1hdGhiZntYfV5UXG1hdGhiZntZfQpcZW5ke2FycmF5fQpcXQoKJCQKXGhhdHtcYm9sZHN5bWJvbHtcYmV0YX19PShcbWF0aGJme1h9XlRcbWF0aGJme1h9KV57LTF9XG1hdGhiZntYfV5UXG1hdGhiZntZfQokJAoKIyMgRml0dGVkIHZhbHVlczoKCiQkClxiZWdpbnthcnJheX17bGNsfQpcaGF0e1xtYXRoYmZ7WX19ICY9JiBcbWF0aGJme1h9XGhhdHtcYm9sZHN5bWJvbHtcYmV0YX19XFwKJj0mIFxtYXRoYmZ7WH0gKFxtYXRoYmZ7WH1eVFxtYXRoYmZ7WH0pXnstMX1cbWF0aGJme1h9XlRcbWF0aGJme1l9XFwKXGVuZHthcnJheX0KJCQKCiMgR2VvbWV0cmljIGludGVycHJldGF0aW9uIG9mIExpbmVhciByZWdyZXNzaW9uCgpUaGVyZSBpcyBhbHNvIGFub3RoZXIgcGljdHVyZSB0byBpbnRlcnByZXQgbGluZWFyIHJlZ3Jlc3Npb24hCgpMaW5lYXIgcmVncmVzc2lvbiBjYW4gYWxzbyBiZSBzZWVuIGFzIGEgcHJvamVjdGlvbiEKCkZpdHRlZCB2YWx1ZXM6CgokJApcYmVnaW57YXJyYXl9e2xjbH0KXGhhdHtcbWF0aGJme1l9fSAmPSYgXG1hdGhiZntYfVxoYXR7XGJvbGRzeW1ib2x7XGJldGF9fVxcCiY9JiBcbWF0aGJme1h9IChcbWF0aGJme1h9XlRcbWF0aGJme1h9KV57LTF9XG1hdGhiZntYfV5UXG1hdGhiZntZfVxcCiY9JiBcbWF0aGJme0hZfQpcZW5ke2FycmF5fQokJAp3aXRoICRcbWF0aGJme0h9JCB0aGUgcHJvamVjdGlvbiBtYXRyaXggYWxzbyByZWZlcnJlZCB0byBhcyB0aGUgaGF0IG1hdHJpeC4KCgpgYGB7cn0KWCA8LSBtb2RlbC5tYXRyaXgofngsZGF0YSkKWApgYGAKCmBgYHtyfQpYdFggPC0gdChYKSUqJVgKWHRYCmBgYAoKYGBge3J9Clh0WGludiA8LSBzb2x2ZSh0KFgpJSolWCkKWHRYaW52CmBgYAoKYGBge3J9CkggPC0gWCAlKiUgWHRYaW52ICUqJSB0KFgpCkgKYGBgCgoKYGBge3J9ClkgPC0gZGF0YSR5ClloYXQgPC0gSCUqJVkKWWhhdApgYGAKCiMjIFdoYXQgZG8gdGhlc2UgcHJvamVjdGlvbnMgbWVhbiBnZW9tZXRyaWNhbGx5PwoKVGhlIG90aGVyIHBpY3R1cmUgdG8gbGluZWFyIHJlZ3Jlc3Npb24gaXMgdG8gY29uc2lkZXIgJFhfMCQsICRYXzEkIGFuZCAkWSQgYXMgdmVjdG9ycyBpbiB0aGUgc3BhY2Ugb2YgdGhlIGRhdGEgJFxtYXRoYmJ7Un1ebiQsIGhlcmUgJFxtYXRoYmJ7Un1eMyQgYmVjYXVzZSB3ZSBoYXZlIHRocmVlIGRhdGEgcG9pbnRzLgoKCmBgYHtyfQpvcmlnaW5SbiA8LSBkYXRhLmZyYW1lKFgxPTAsWDI9MCxYMz0wKQpkYXRhJHgwIDwtIDEKZGF0YVJuIDwtIGRhdGEuZnJhbWUodChkYXRhKSkKCmxpYnJhcnkocGxvdGx5KQoKcDEgPC0gcGxvdF9seSgKICAgIG9yaWdpblJuLAogICAgeCA9IH4gWDEsCiAgICB5ID0gfiBYMiwKICAgIHo9IH4gWDMpICU+JQogIGFkZF9tYXJrZXJzKHR5cGU9InNjYXR0ZXIzZCIpICU+JQogIGxheW91dCgKICAgIHNjZW5lID0gbGlzdCgKICAgICAgYXNwZWN0bW9kZT0iY3ViZSIsCiAgICAgIHhheGlzID0gbGlzdChyYW5nZT1jKC00LDQpKSwgeWF4aXMgPSBsaXN0KHJhbmdlPWMoLTQsNCkpLCB6YXhpcyA9IGxpc3QocmFuZ2U9YygtNCw0KSkKICAgICAgKQogICAgKQpwMSA8LSBwMSAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYygwLDEpLAogICAgeSA9IGMoMCwwKSwKICAgIHogPSBjKDAsMCksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAiZ3JleSIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikgJT4lCiAgYWRkX3RyYWNlKAogICAgeCA9IGMoMCwwKSwKICAgIHkgPSBjKDAsMSksCiAgICB6ID0gYygwLDApLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gImdyZXkiKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JQogIGFkZF90cmFjZSgKICAgIHggPSBjKDAsMCksCiAgICB5ID0gYygwLDApLAogICAgeiA9IGMoMCwxKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJncmV5IiksCiAgICB0eXBlPSJzY2F0dGVyM2QiKSAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYygwLDEpLAogICAgeSA9IGMoMCwxKSwKICAgIHogPSBjKDAsMSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAiYmxhY2siKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JQogICAgYWRkX3RyYWNlKAogICAgeCA9IGMoMCwxKSwKICAgIHkgPSBjKDAsMiksCiAgICB6ID0gYygwLDMpLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gImJsYWNrIiksCiAgICB0eXBlPSJzY2F0dGVyM2QiKQpgYGAKYGBge3J9CnAyIDwtIHAxICU+JQogIGFkZF90cmFjZSgKICAgIHggPSBjKDAsWVsxXSksCiAgICB5ID0gYygwLFlbMl0pLAogICAgeiA9IGMoMCxZWzNdKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJyZWQiKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JQogIGFkZF90cmFjZSgKICAgIHggPSBjKDAsWWhhdFsxXSksCiAgICB5ID0gYygwLFloYXRbMl0pLAogICAgeiA9IGMoMCxZaGF0WzNdKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJyZWQiKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JSBhZGRfdHJhY2UoCiAgICB4ID0gYyhZWzFdLFloYXRbMV0pLAogICAgeSA9IGMoWVsyXSxZaGF0WzJdKSwKICAgIHogPSBjKFlbM10sWWhhdFszXSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAicmVkIiwgZGFzaD0iZGFzaCIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikKcDIKYGBgCgoKIyMjIEhvdyBkb2VzIHRoaXMgcHJvamVjdGlvbiB3b3JrPwoKJCQKXGJlZ2lue2FycmF5fXtsY2x9ClxoYXR7XG1hdGhiZntZfX0gJj0mIFxtYXRoYmZ7WH0gKFxtYXRoYmZ7WH1eVFxtYXRoYmZ7WH0pXnstMX1cbWF0aGJme1h9XlRcbWF0aGJme1l9XFwKJj0mIFxtYXRoYmZ7WH0oXG1hdGhiZntYfV5UXG1hdGhiZntYfSleey0xLzJ9KFxtYXRoYmZ7WH1eVFxtYXRoYmZ7WH0pXnstMS8yfVxtYXRoYmZ7WH1eVFxtYXRoYmZ7WX1cXAomPSYgXG1hdGhiZntVfVxtYXRoYmZ7VX1eVFxtYXRoYmZ7WX0KXGVuZHthcnJheX0KJCQKCgotICRcbWF0aGJme1V9JCBpcyBhIG5ldyBvcnRob25vcm1hbCBiYXNpcyBpbiAkXG1hdGhiYntSfV4yJCwgYSBzdWJzcGFjZSBvZiAkXG1hdGhiYntSfV4zJAoKLSBUaGUgc3BhY2Ugc3Bhbm5lZCBieSBVIGFuZCBYIGlzIHRoZSBjb2x1bW4gc3BhY2Ugb2YgWCwgZS5nLiBpdCBjb250YWlucyBhbGwgcG9zc2libGUgbGluZWFyIGNvbWJpbmFudGlvbnMgb2YgWC4KJFxtYXRoYmZ7VX1edFxtYXRoYmZ7WX0kIGlzIHRoZSBwcm9qZWN0aW9uIG9mIFkgb24gdGhpcyBuZXcgb3J0aG9ub3JtYWwgYmFzaXMKCmBgYHtyfQplaWdlblh0WCA8LSBlaWdlbihYdFgpClh0WGludlNxcnQgPC0gZWlnZW5YdFgkdmVjdG9ycyAlKiVkaWFnKDEvZWlnZW5YdFgkdmFsdWVzXi41KSUqJXQoZWlnZW5YdFgkdmVjdG9ycykKVSA8LSBYICUqJSBYdFhpbnZTcXJ0CgpwMyA8LSBwMSAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYygwLFVbMSwxXSksCiAgICB5ID0gYygwLFVbMiwxXSksCiAgICB6ID0gYygwLFVbMywxXSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAiYmx1ZSIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikgJT4lCiAgYWRkX3RyYWNlKAogICAgeCA9IGMoMCxVWzEsMl0pLAogICAgeSA9IGMoMCxVWzIsMl0pLAogICAgeiA9IGMoMCxVWzMsMl0pLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gImJsdWUiKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpCgpwMwpgYGAKCgotICRcbWF0aGJme1V9XlRcbWF0aGJme1l9JCBpcyB0aGUgcHJvamVjdGlvbiBvZiAkXG1hdGhiZntZfSQgaW4gdGhlIHNwYWNlIHNwYW5uZWQgYnkgJFxtYXRoYmZ7VX0kLgotIEluZGVlZCAkXG1hdGhiZntVfV8xXlRcbWF0aGJme1l9JAoKYGBge3J9CnA0IDwtIHAzICU+JQogIGFkZF90cmFjZSgKICAgIHggPSBjKDAsWVsxXSksCiAgICB5ID0gYygwLFlbMl0pLAogICAgeiA9IGMoMCxZWzNdKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJyZWQiKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JQogIGFkZF90cmFjZSgKICAgIHggPSBjKDAsVVsxLDFdKihVWywxXSUqJVkpKSwKICAgIHkgPSBjKDAsVVsyLDFdKihVWywxXSUqJVkpKSwKICAgIHogPSBjKDAsVVszLDFdKihVWywxXSUqJVkpKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJyZWQiLGRhc2g9ImRhc2giKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JSBhZGRfdHJhY2UoCiAgICB4ID0gYyhZWzFdLFVbMSwxXSooVVssMV0lKiVZKSksCiAgICB5ID0gYyhZWzJdLFVbMiwxXSooVVssMV0lKiVZKSksCiAgICB6ID0gYyhZWzNdLFVbMywxXSooVVssMV0lKiVZKSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAicmVkIiwgZGFzaD0iZGFzaCIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikKcDQKYGBgCgotIGFuZCAkXG1hdGhiZntVfV8yXlRcbWF0aGJme1l9JApgYGB7cn0KcDUgPC0gcDQgJT4lCiAgYWRkX3RyYWNlKAogICAgeCA9IGMoMCxZWzFdKSwKICAgIHkgPSBjKDAsWVsyXSksCiAgICB6ID0gYygwLFlbM10pLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gInJlZCIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikgJT4lCiAgYWRkX3RyYWNlKAogICAgeCA9IGMoMCxVWzEsMl0qKFVbLDJdJSolWSkpLAogICAgeSA9IGMoMCxVWzIsMl0qKFVbLDJdJSolWSkpLAogICAgeiA9IGMoMCxVWzMsMl0qKFVbLDJdJSolWSkpLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gInJlZCIsZGFzaD0iZGFzaCIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikgJT4lIGFkZF90cmFjZSgKICAgIHggPSBjKFlbMV0sVVsxLDJdKihVWywyXSUqJVkpKSwKICAgIHkgPSBjKFlbMl0sVVsyLDJdKihVWywyXSUqJVkpKSwKICAgIHogPSBjKFlbM10sVVszLDJdKihVWywyXSUqJVkpKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJyZWQiLCBkYXNoPSJkYXNoIiksCiAgICB0eXBlPSJzY2F0dGVyM2QiKQpwNQpgYGAKCmBgYHtyfQpwNiA8LSBwNSAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYygwLFloYXRbMV0pLAogICAgeSA9IGMoMCxZaGF0WzJdKSwKICAgIHogPSBjKDAsWWhhdFszXSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAicmVkIiksCiAgICB0eXBlPSJzY2F0dGVyM2QiKSAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYyhZWzFdLFloYXRbMV0pLAogICAgeSA9IGMoWVsyXSxZaGF0WzJdKSwKICAgIHogPSBjKFlbM10sWWhhdFszXSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAicmVkIiwgZGFzaD0iZGFzaCIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikgJT4lCiAgYWRkX3RyYWNlKAogICAgeCA9IGMoVVsxLDFdKihVWywxXSUqJVkpLFloYXRbMV0pLAogICAgeSA9IGMoVVsyLDFdKihVWywxXSUqJVkpLFloYXRbMl0pLAogICAgeiA9IGMoVVszLDFdKihVWywxXSUqJVkpLFloYXRbM10pLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gInJlZCIsIGRhc2g9ImRhc2giKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYyhVWzEsMl0qKFVbLDJdJSolWSksWWhhdFsxXSksCiAgICB5ID0gYyhVWzIsMl0qKFVbLDJdJSolWSksWWhhdFsyXSksCiAgICB6ID0gYyhVWzMsMl0qKFVbLDJdJSolWSksWWhhdFszXSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAicmVkIiwgZGFzaD0iZGFzaCIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikKcDYKYGBgCgojIyBUaGUgRXJyb3IgdmVjdG9yCgpOb3RlLCB0aGF0IGl0IGlzIGFsc28gY2xlYXIgZnJvbSB0aGUgZXF1YXRpb24gaW4gdGhlIGRlcml2YXRpb24gb2YgdGhlIGxlYXN0IHNxdWFyZXMgc29sdXRpb24gdGhhdCB0aGUgcmVzaWR1YWwgaXMgb3J0aG9nb25hbCBvbiB0aGUgY29sdW1uIHNwYWNlOgoKXFsKIC0yIFxtYXRoYmZ7WH1eVChcbWF0aGJme1l9LVxtYXRoYmZ7WH1cYm9sZHN5bWJvbHtcYmV0YX0pID0gMApcXQoKCiMgQ3Vyc2Ugb2YgZGltZW5zaW9uYWxpdHk/CgotIEltYWdpbmUgd2hhdCBoYXBwZW5zIHdoZW4gcCBhcHByb2FjaGVzIG4gJHA9biQgb3IgYmVjb21lcyBtdWNoIGxhcmdlciB0aGFuIHAgPj4gbiEhIQoKLSBTdXBwb3NlIHRoYXQgd2UgYWRkIGEgcHJlZGljdG9yICRcbWF0aGJme1h9XzIgPSBbMiwwLDFdXlQkPwoKJCQKXG1hdGhiZntZfT1cbGVmdFsKXGJlZ2lue2FycmF5fXtjfQoxXFwKMlxcCjJcXApcZW5ke2FycmF5fVxyaWdodF0sClxxdWFkClxtYXRoYmZ7WH09IFxsZWZ0WwpcYmVnaW57YXJyYXl9e2NjY30KMSYxJjJcXAoxJjImMFxcCjEmMyYxXFwKXGVuZHthcnJheX0KXHJpZ2h0XSwKXHF1YWQgXGJvbGRzeW1ib2x7XGJldGF9ID0gXGxlZnRbClxiZWdpbnthcnJheX17Y30KXGJldGFfMFxcClxiZXRhXzFcXApcYmV0YV8yClxlbmR7YXJyYXl9ClxyaWdodF0KXHF1YWQKXHRleHR7YW5kfQpccXVhZApcYm9sZHN5bWJvbHtcZXBzaWxvbn09ClxsZWZ0WwpcYmVnaW57YXJyYXl9e2N9ClxlcHNpbG9uXzFcXApcZXBzaWxvbl8yXFwKXGVwc2lsb25fMwpcZW5ke2FycmF5fQpccmlnaHRdCiQkCgoKYGBge3J9CmRhdGEkeDIgPC0gYygyLDAsMSkKZml0IDwtIGxtKHl+eCt4MixkYXRhKQojIHByZWRpY3QgdmFsdWVzIG9uIHJlZ3VsYXIgeHkgZ3JpZAp4MXByZWQgPC0gc2VxKC0xLCA0LCBsZW5ndGgub3V0ID0gMTApCngycHJlZCA8LSBzZXEoLTEsIDQsIGxlbmd0aC5vdXQgPSAxMCkKeHkgPC0gZXhwYW5kLmdyaWQoeCA9IHgxcHJlZCwKeDIgPSB4MnByZWQpCnlwcmVkIDwtIG1hdHJpeCAobnJvdyA9IDMwLCBuY29sID0gMzAsCmRhdGEgPSBwcmVkaWN0KGZpdCwgbmV3ZGF0YSA9IGRhdGEuZnJhbWUoeHkpKSkKCmxpYnJhcnkocGxvdDNEKQoKCiMgZml0dGVkIHBvaW50cyBmb3IgZHJvcGxpbmVzIHRvIHN1cmZhY2UKdGg9MjAKcGg9NQpzY2F0dGVyM0QoZGF0YSR4LAogIGRhdGEkeDIsCiAgWSwKICBwY2ggPSAxNiwKICBjb2w9ImRhcmtibHVlIiwKICBjZXggPSAxLAogIHRoZXRhID0gdGgsCiAgdGlja3R5cGUgPSAiZGV0YWlsZWQiLAogIHhsYWIgPSAieDEiLAogIHlsYWIgPSAieDIiLAogIHpsYWIgPSAieSIsCiAgY29sdmFyPUZBTFNFLAogIGJ0eSA9ICJnIiwKICB4bGltPWMoLTEsMyksCiAgeWxpbT1jKC0xLDMpLAogIHpsaW09YygtMiw0KSkKCgp6LnByZWQzRCA8LSBvdXRlcigKICB4MXByZWQsCiAgeDJwcmVkLAogIGZ1bmN0aW9uKHgxLHgyKQogIHsKICAgIGZpdCRjb2VmWzFdICsgZml0JGNvZWZbMl0qeDErZml0JGNvZWZbMl0qeDIKICB9KQoKeC5wcmVkM0QgPC0gb3V0ZXIoCiAgeDFwcmVkLAogIHgycHJlZCwKICBmdW5jdGlvbih4LHkpIHgpCgp5LnByZWQzRCA8LSBvdXRlcigKICB4MXByZWQsCiAgeDJwcmVkLAogIGZ1bmN0aW9uKHgseSkgeSkKCnNjYXR0ZXIzRChkYXRhJHgsCiAgZGF0YSR4MiwKICBkYXRhJHksCiAgcGNoID0gMTYsCiAgY29sPSJkYXJrYmx1ZSIsCiAgY2V4ID0gMSwKICB0aGV0YSA9IHRoLAogIHRpY2t0eXBlID0gImRldGFpbGVkIiwKICB4bGFiID0gIngxIiwKICB5bGFiID0gIngyIiwKICB6bGFiID0gInkiLAogIGNvbHZhcj1GQUxTRSwKICBidHkgPSAiZyIsCiAgeGxpbT1jKC0xLDQpLAogIHlsaW09YygtMSw0KSwKICB6bGltPWMoLTIsNCkpCgpzdXJmM0QoCiAgeC5wcmVkM0QsCiAgeS5wcmVkM0QsCiAgei5wcmVkM0QsCiAgY29sPSJibHVlIiwKICBmYWNldHM9TkEsCiAgYWRkPVRSVUUpCmBgYAoKTm90ZSwgdGhhdCB0aGUgbGluZWFyIHJlZ3Jlc3Npb24gaXMgbm93IGEgcGxhbmUuCgpIb3dldmVyLCB3ZSBvYnRhaW4gYSBwZXJmZWN0IGZpdCBhbmQgYWxsIHRoZSBkYXRhIHBvaW50cyBhcmUgZmFsbGluZyBpbiB0aGUgcGxhbmUhIGByIHNldC5zZWVkKDQpO2Vtbzo6amkoImZlYXIiKWAKClRoaXMgaXMgb2J2aW91cyBpZiB3ZSBsb29rIGF0IHRoZSBjb2x1bW4gc3BhY2Ugb2YgWCEKCmBgYHtyfQpYIDwtIGNiaW5kKFgsYygyLDAsMSkpClh0WCA8LSB0KFgpJSolWAplaWdlblh0WCA8LSBlaWdlbihYdFgpClh0WGludlNxcnQgPC0gZWlnZW5YdFgkdmVjdG9ycyAlKiVkaWFnKDEvZWlnZW5YdFgkdmFsdWVzXi41KSUqJXQoZWlnZW5YdFgkdmVjdG9ycykKVSA8LSBYICUqJSBYdFhpbnZTcXJ0CgpwNyA8LSBwMSAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYygwLDIpLAogICAgeSA9IGMoMCwwKSwKICAgIHogPSBjKDAsMSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAiZGFya2dyZWVuIiksCiAgICB0eXBlPSJzY2F0dGVyM2QiKQpwNwpgYGAKCmBgYHtyfQpwOCA8LSBwNyAlPiUKICBhZGRfdHJhY2UoCiAgICB4ID0gYygwLFVbMSwxXSksCiAgICB5ID0gYygwLFVbMiwxXSksCiAgICB6ID0gYygwLFVbMywxXSksCiAgICBtb2RlID0gImxpbmVzIiwKICAgIGxpbmUgPSBsaXN0KHdpZHRoID0gNSwgY29sb3IgPSAiYmx1ZSIpLAogICAgdHlwZT0ic2NhdHRlcjNkIikgJT4lCiAgYWRkX3RyYWNlKAogICAgeCA9IGMoMCxVWzEsMl0pLAogICAgeSA9IGMoMCxVWzIsMl0pLAogICAgeiA9IGMoMCxVWzMsMl0pLAogICAgbW9kZSA9ICJsaW5lcyIsCiAgICBsaW5lID0gbGlzdCh3aWR0aCA9IDUsIGNvbG9yID0gImJsdWUiKSwKICAgIHR5cGU9InNjYXR0ZXIzZCIpICU+JQogIGFkZF90cmFjZSgKICAgIHggPSBjKDAsVVsxLDNdKSwKICAgIHkgPSBjKDAsVVsyLDNdKSwKICAgIHogPSBjKDAsVVszLDNdKSwKICAgIG1vZGUgPSAibGluZXMiLAogICAgbGluZSA9IGxpc3Qod2lkdGggPSA1LCBjb2xvciA9ICJibHVlIiksCiAgICB0eXBlPSJzY2F0dGVyM2QiKQoKcDgKYGBgCgotIFRoZSBjb2x1bW4gc3BhY2Ugbm93IHNwYW5zIHRoZSBlbnRpcmUgICRcbWF0aGJie1J9XjMkIQotIFdpdGggdGhlIGludGVyY2VwdCBhbmQgdGhlIHR3byBwcmVkaWN0b3JzIHdlIGNhbiB0aHVzIGZpdCBldmVyeSBkYXRhc2V0IHRoYXQgb25seSBoYXMgMyBvYnNlcnZhdGlvbnMgZm9yIHRoZSBwcmVkaWN0b3JzIGFuZCB0aGUgcmVzcG9uc2UuCi0gU28gdGhlIG1vZGVsIGNhbiBubyBsb25nZXIgYmUgdXNlZCB0byBnZW5lcmFsaXNlIHRoZSBwYXR0ZXJucyBzZWVuIGluIHRoZSBkYXRhIHRvd2FyZHMgdGhlIHBvcHVsYXRpb24gKG5ldyBvYnNlcnZhdGlvbnMpLgotIFByb2JsZW0gb2Ygb3ZlcmZpdHRpbmchISEKCi0gSWYgJHAgPj4gbiQgdGhlbiB0aGUgcHJvYmxlbSBnZXRzIGV2ZW4gd29yc2UhIFRoZW4gdGhlcmUgaXMgZXZlbiBubyBsb25nZXIgYSB1bmlxdWUgc29sdXRpb24gdG8gdGhlIGxlYXN0IHNxdWFyZXMgcHJvYmxlbS4uLgo= *** This work is licensed under the [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0) licence.