Responding to a query from Rolf Turner on r-sig-mixed-models@r-project.org.
tl;dr the error messages are spurious/due to a singular fit. Considerable speed-ups can be achieved by (1) switching optimizers, (2) using nAGQ=0, (3) using glmmTMB or Julia ...
to do
compare variance-covariance matrix estimates (they vary a lot, although it doesn't seem to have much effect on the fixed effects?
play with/fight with Julia more: speed? try fast! (corresponding to nAGQ=0)?
try estimation with treatment and treatment-by-x effects either as real random effects (i.e., with shrinkage) or with large fixed variance (for computational efficiency)
caveats
Julia timing is still questionable; I have previously gotten results as fast as 7 seconds for the full model fit (vs. the 36 seconds reported below)
Load the Julia packages to be used
julia> using DataFrames, GLM, MixedModels, RCalll
and load the R packages through RCall
julia> R""" ## fitting packages suppressMessages(library(lme4)) library(glmmTMB) library(brms) """;
Read the data previously generated in R. At present the MixedModels package does not support the two-column response format for a binomial and the total and propostion must be calculated explicitly.
const df = readtable("artSim.csv", makefactors=true); df[:Tot] = float.(df[:Alive] .+ df[:Dead]); df[:prop] = df[:Dead] ./ df[:Tot];
Push the data to the R process
@rput df
and examine the structure
R"str(df)"
'data.frame': 1080 obs. of 7 variables: $ x : int 0 2 4 6 8 10 12 14 16 18 ... $ Trt : Factor w/ 24 levels "A","B","C" ,"D",..: 1 1 1 1 1 1 1 1 1 1 ... $ Rep : int 1 1 1 1 1 1 1 1 1 1 ... $ Dead : int 0 0 2 2 1 1 1 0 3 2 ... $ Alive: int 25 25 23 23 24 24 24 25 22 23 ... $ Tot : num 25 25 25 25 25 25 25 25 25 25 ... $ prop : num 0 0 0.08 0.08 0.04 0.04 0.04 0 0.12 0.08 ...
The model that is to be fit to these data is represented by the formula
frm = @formula(prop ~ 0 + Trt + Trt&x + (1 + x|Rep))
There will be 3 parameters in the covariance matrix for the random effects term and 48 fixed-effects coefficients. This is a case where the glmer option nAGQ=0 should produce a much faster fit because the constrained nonlinear optimization will be over the 3 parameters from the covariance matrix. Without this option the 3-parameter optimization is done first then followed by a constrained nonlinear optimization over both the covariance parameters and the fixed-effects, a total of 39 parameters.
When timing a Julia fit it is important to distinguish between the initial call to a function, which may cause "Just In Time" or JIT compilation, and subsequent executions. Depending upon the exact sequence of calls the first call can take much longer.
First use the fast=true option to the fit! call. This is equivalent to using nAGQ=0 in glmer
show(fit!(glmm(frm, df, Binomial(), CloglogLink(), wt=Array(df[:Tot])), fast=true))
Generalized Linear Mixed Model fit by minimizing the Laplace approximation
to the deviance
Formula: prop ~ 0 + Trt + Trt & x + ((1 + x&#
41; | Rep)
Distribution: Distributions.Binomial{Float64}
Link: GLM.CloglogLink()
Deviance (Laplace approximation): 888.4787
Variance components:
Column Variance Std.Dev. Corr.
Rep (Intercept) 0.058839572456 0.242568696
x 0.000102296623 0.010114179 -0.59
Number of obs: 1080; levels of grouping factors: 72
Fixed-effects parameters:
Estimate Std.Error z value P(>|z|)
Trt: A -3.14366 0.270468 -11.6231 <1e-30
Trt: B -2.80304 0.213021 -13.1585 <1e-38
Trt: C -2.58196 0.197764 -13.0558 <1e-38
Trt: D -2.31225 0.205731 -11.2392 <1e-28
Trt: E -2.54773 0.220614 -11.5484 <1e-30
Trt: F -2.12062 0.222265 -9.54095 <1e-20
Trt: G -2.11696 0.199161 -10.6294 <1e-25
Trt: H -2.03373 0.185538 -10.9612 <1e-27
Trt: I -2.23967 0.195561 -11.4525 <1e-29
Trt: J -1.81182 0.201949 -8.97166 <1e-18
Trt: K -1.31475 0.198401 -6.62672 <1e-10
Trt: L -1.29853 0.202599 -6.40935 <1e-9
Trt: M -1.0978 0.168796 -6.50369 <1e-10
Trt: N -0.924785 0.16666 -5.54893 <1e-7
Trt: O -0.98236 0.180703 -5.43632 <1e-7
Trt: P -0.816064 0.186235 -4.38191 <1e-4
Trt: Q -0.615693 0.192595 -3.19683 0.0014
Trt: R -0.152161 0.187649 -0.810879 0.4174
Trt: S -0.217567 0.159767 -1.36178 0.1733
Trt: T -0.283216 0.168113 -1.68468 0.0921
Trt: U 0.106325 0.177251 0.599856 0.5486
Trt: V 0.0794044 0.187463 0.423573 0.6719
Trt: W 0.427792 0.198575 2.15431 0.0312
Trt: X 0.262161 0.19459 1.34725 0.1779
Trt: A & x 0.0513203 0.0133104 3.85564 0.0001
Trt: B & x 0.108349 0.00967386 11.2002 <1e-28
Trt: C & x 0.143247 0.00942057 15.2057 <1e-51
Trt: D & x 0.197537 0.0131269 15.0482 <1e-50
Trt: E & x 0.235773 0.015821 14.9026 <1e-49
Trt: F & x 0.287073 0.0217839 13.1782 <1e-38
Trt: G & x 0.0555802 0.00940497 5.90967 <1e-8
Trt: H & x 0.101309 0.008601 11.7787 <1e-31
Trt: I & x 0.164692 0.0107969 15.2537 <1e-51
Trt: J & x 0.219653 0.016244 13.5221 <1e-40
Trt: K & x 0.241308 0.0205554 11.7394 <1e-31
Trt: L & x 0.269954 0.023979 11.2579 <1e-28
Trt: M & x 0.054908 0.00781037 7.03014 <1e-11
Trt: N & x 0.0914613 0.00830168 11.0172 <1e-27
Trt: O & x 0.162726 0.0138346 11.7622 <1e-31
Trt: P & x 0.202323 0.0193266 10.4686 <1e-24
Trt: Q & x 0.260135 0.0292534 8.89246 <1e-18
Trt: R & x 0.23447 0.0321262 7.29842 <1e-12
Trt: S & x 0.0530068 0.0076192 6.95701 <1e-11
Trt: T & x 0.107314 0.0113858 9.42529 <1e-20
Trt: U & x 0.148021 0.0210124 7.04447 <1e-11
Trt: V & x 0.229408 0.0363827 6.30541 <1e-9
Trt: W & x 0.3901 0.0986662 3.95373 <1e-4
Trt: X & x 0.317157 0.0651612 4.86727 <1e-5
@time g3 = fit!(glmm(frm, df, Binomial(), CloglogLink(), wt=Array(df[:Tot])), fast=true);
0.930170 seconds (590.95 k allocations: 32.492 MiB, 0.61% gc time )
@show loglikelihood(g3)
loglikelihood(g3) = -1413.2838981380403
show(g3.LMM.optsum)
Initial parameter vector: [1.0, 0.0, 1.0] Initial objective value: 1480.8990814709841 Optimizer (from NLopt): LN_BOBYQA Lower bounds: [0.0, -Inf, 0.0] ftol_rel: 1.0e-12 ftol_abs: 1.0e-8 xtol_rel: 0.0 xtol_abs: [1.0e-10, 1.0e-10, 1.0e-10] initial_step: [0.75, 1.0, 0.75] maxfeval: -1 Function evaluations: 116 Final parameter vector: [0.242569, -0.00594272, 0.00818418] Final objective value: 888.4786718210885 Return code: FTOL_REACHED
The two-stage fit is a bit slower but produces a slightly greater log-likelihood at the estimate.
@time g4 = fit!(glmm(frm, df, Binomial(), CloglogLink(), wt=Array(df[:Tot])));
4.859606 seconds (3.05 M allocations: 158.008 MiB, 0.64% gc time& #41;
@show loglikelihood(g4)
loglikelihood(g4) = -1413.2069452041058
show(g4.LMM.optsum)
Initial parameter vector: [-3.14366, -2.80304, -2.58196, -2.31225, -2.5 4773, -2.12062, -2.11696, -2.03373, -2.23967, -1.81182, -1.31475, -1.29853, -1.0978, -0.924785, -0.98236, -0.816064, -0.615693, -0.152161, -0.217567, -0.283216, 0.106325, 0.0794044, 0.427792, 0.262161, 0.0513203, 0.108349, 0. 143247, 0.197537, 0.235773, 0.287073, 0.0555802, 0.101309, 0.164692, 0.2196 53, 0.241308, 0.269954, 0.054908, 0.0914613, 0.162726, 0.202323, 0.260135, 0.23447, 0.0530068, 0.107314, 0.148021, 0.229408, 0.3901, 0.317157, 0.24256 9, -0.00594272, 0.00818418] Initial objective value: 888.4786714335524 Optimizer (from NLopt): LN_BOBYQA Lower bounds: [-Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -I nf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -I nf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, 0.0] ftol_rel: 1.0e-12 ftol_abs: 1.0e-8 xtol_rel: 0.0 xtol_abs: [1.0e-10, 1.0e-10, 1.0e-10] initial_step: [0.0901558, 0.0710071, 0.0659213, 0.0685769, 0.0735379, 0.0740882, 0.066387, 0.0618461, 0.0651872, 0.0673164, 0.0661338, 0.0675331, 0.0562654, 0.0555534, 0.0602344, 0.0620782, 0.0641984, 0.062549 7, 0.0532555, 0.0560376, 0.0590838, 0.0624878, 0.0661918, 0.0648632, 0.0044 3681, 0.00322462, 0.00314019, 0.00437564, 0.00527366, 0.00726129, 0.0031349 9, 0.002867, 0.00359896, 0.00541467, 0.0068518, 0.007993, 0.00260346, 0.002 76723, 0.00461154, 0.0064422, 0.00975114, 0.0107087, 0.00253973, 0.00379525 , 0.00700414, 0.0121276, 0.0328887, 0.0217204, 0.05, -0.00148568, 0.0020460 4] maxfeval: -1 Function evaluations: 619 Final parameter vector: [-3.15538, -2.81229, -2.59413, -2.32564, -2.5 6195, -2.13411, -2.12464, -2.04179, -2.25215, -1.82313, -1.32507, -1.30899, -1.10267, -0.932395, -0.990863, -0.824555, -0.624065, -0.157924, -0.221121 , -0.289599, 0.103097, 0.0754325, 0.426482, 0.258959, 0.0513934, 0.108647, 0.143932, 0.1987, 0.237116, 0.288876, 0.0557233, 0.10168, 0.165637, 0.22098 4, 0.243059, 0.271927, 0.0550891, 0.0921122, 0.163939, 0.203958, 0.262643, 0.237327, 0.0533389, 0.108413, 0.149718, 0.232559, 0.400008, 0.32316, 0.242 706, -0.00594923, 0.0082008] Final objective value: 888.3247659533648 Return code: FTOL_REACHED
For reference, the version of Julia and the packages are
versioninfo()
Julia Version 0.6.0 Commit 9036443 (2017-06-19 13:05 UTC) Platform Info: OS: Linux (x86_64-pc-linux-gnu) CPU: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge&# 41; LAPACK: libopenblas64_ LIBM: libopenlibm LLVM: libLLVM-3.9.1 (ORCJIT, ivybridge)
Pkg.status.(["GLM", "MixedModels"])
- GLM 0.7.0+ master - MixedModels 0.10.0+ master
The glmmTMB package for R provides the function of the same name.
R""" t1 <- system.time(fit1 <- glmmTMB(prop ~ (Trt + 0)/x + (x | Rep), weights = Tot, family=binomial(link="cloglog"),data=df)) print(t1) print(summary(fit1)) """;
user system elapsed
12.940 0.204 12.900
Family: binomial ( cloglog )
Formula: prop ~ (Trt + 0)/x + (x | Rep)
Data: df
Weights: Tot
AIC BIC logLik deviance df.resid
2934.0 3188.2 -1416.0 2832.0 1029
Random effects:
Conditional model:
Groups Name Variance Std.Dev. Corr
Rep (Intercept) 0.0911953 0.30199
x 0.0002018 0.01421 -0.85
Number of obs: 1080, groups: Rep, 72
Conditional model:
Estimate Std. Error z value Pr(>|z|)
TrtA -3.128386 0.288867 -10.830 < 2e-16 ***
TrtB -2.795983 0.237114 -11.792 < 2e-16 ***
TrtC -2.594271 0.224446 -11.559 < 2e-16 ***
TrtD -2.315585 0.231151 -10.018 < 2e-16 ***
TrtE -2.550779 0.244096 -10.450 < 2e-16 ***
TrtF -2.127179 0.247135 -8.607 < 2e-16 ***
TrtG -2.129283 0.225136 -9.458 < 2e-16 ***
TrtH -2.034643 0.213341 -9.537 < 2e-16 ***
TrtI -2.232149 0.219991 -10.147 < 2e-16 ***
TrtJ -1.815797 0.226365 -8.022 1.04e-15 ***
TrtK -1.343958 0.226206 -5.941 2.83e-09 ***
TrtL -1.317193 0.228411 -5.767 8.08e-09 ***
TrtM -1.106273 0.198377 -5.577 2.45e-08 ***
TrtN -0.937612 0.197265 -4.753 2.00e-06 ***
TrtO -0.996496 0.208469 -4.780 1.75e-06 ***
TrtP -0.821662 0.214596 -3.829 0.000129 ***
TrtQ -0.628222 0.219452 -2.863 0.004201 **
TrtR -0.169397 0.216490 -0.782 0.433939
TrtS -0.229032 0.191142 -1.198 0.230827
TrtT -0.294068 0.197341 -1.490 0.136184
TrtU 0.100323 0.205449 0.488 0.625330
TrtV 0.070562 0.216039 0.327 0.743958
TrtW 0.416483 0.225147 1.850 0.064339 .
TrtX 0.251028 0.221212 1.135 0.256464
TrtA:x 0.051194 0.014474 3.537 0.000405 ***
TrtB:x 0.108206 0.011283 9.590 < 2e-16 ***
TrtC:x 0.143226 0.011130 12.868 < 2e-16 ***
TrtD:x 0.198396 0.014400 13.778 < 2e-16 ***
TrtE:x 0.235215 0.016906 13.914 < 2e-16 ***
TrtF:x 0.290013 0.023029 12.593 < 2e-16 ***
TrtG:x 0.055388 0.011063 5.006 5.54e-07 ***
TrtH:x 0.100285 0.010395 9.647 < 2e-16 ***
TrtI:x 0.164710 0.012149 13.557 < 2e-16 ***
TrtJ:x 0.219449 0.017037 12.881 < 2e-16 ***
TrtK:x 0.241748 0.021976 11.000 < 2e-16 ***
TrtL:x 0.269755 0.024818 10.869 < 2e-16 ***
TrtM:x 0.056040 0.009708 5.773 7.80e-09 ***
TrtN:x 0.093264 0.010223 9.123 < 2e-16 ***
TrtO:x 0.160666 0.014882 10.796 < 2e-16 ***
TrtP:x 0.208033 0.020638 10.080 < 2e-16 ***
TrtQ:x 0.262988 0.030264 8.690 < 2e-16 ***
TrtR:x 0.234389 0.034003 6.893 5.45e-12 ***
TrtS:x 0.055086 0.009585 5.747 9.10e-09 ***
TrtT:x 0.107724 0.012583 8.561 < 2e-16 ***
TrtU:x 0.148538 0.021540 6.896 5.35e-12 ***
TrtV:x 0.231748 0.038057 6.089 1.13e-09 ***
TrtW:x 0.407326 0.102894 3.959 7.54e-05 ***
TrtX:x 0.323814 0.065854 4.917 8.78e-07 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Notice that the log-likelihood, -1416.0, for these estimates is slightly smaller than that from the glmm fits, -1413.28 and -1413.21.
The estimates of the fixed-effects coefficients and the covariance matrix for the random effects are similar to those from the fit in Julia.
The deviance displayed in this summary is quite different from that in the Julia fits. The value in this summary is simply -2 * log-likelihood, which does not take into account the saturated model, which is described as the
maximum achieveable log-likelihood (McCullagh and Nelder, Generalized Linear Models (2nd ed), p. 118)
This is why the model fits are compared using the log-likelihood, which should be defined consistently.
Using the default optimizer and nAGQ=0 glmer converges quickly but to a singular covariance matrix with a much smaller log-likelihood, -1451.7
julia> R""" t2 <- system.time(fit2 <- glmer(cbind(Dead,Alive) ~ (Trt + 0)/x + (x | Rep), family=binomial(link="cloglog"), data=df, nAGQ=0L)) print(t2) summary(fit2) """ user system elapsed 0.352 0.000 0.355 RCall.RObject{RCall.VecSxp} Generalized linear mixed model fit by maximum likelihood (Adaptive Gauss-Hermite Quadrature, nAGQ = 0) [glmerMod] Family: binomial ( cloglog ) Formula: cbind(Dead, Alive) ~ (Trt + 0)/x + (x | Rep) Data: df AIC BIC logLik deviance df.resid 3005.3 3259.6 -1451.7 2903.3 1029 Scaled residuals: Min 1Q Median 3Q Max -5.0956 -0.2925 0.0000 0.3645 2.7530 Random effects: Groups Name Variance Std.Dev. Corr Rep (Intercept) 0.0000000 0.00000 x 0.0002326 0.01525 NaN Number of obs: 1080, groups: Rep, 72 Fixed effects: Estimate Std. Error z value Pr(>|z|) TrtA -3.12250 0.23059 -13.541 < 2e-16 *** TrtB -2.80040 0.16040 -17.459 < 2e-16 *** TrtC -2.57445 0.13942 -18.465 < 2e-16 *** TrtD -2.31281 0.15046 -15.372 < 2e-16 *** TrtE -2.54488 0.17026 -14.947 < 2e-16 *** TrtF -2.11982 0.17253 -12.287 < 2e-16 *** TrtG -2.10874 0.14127 -14.927 < 2e-16 *** TrtH -2.03019 0.12165 -16.689 < 2e-16 *** TrtI -2.27603 0.13817 -16.472 < 2e-16 *** TrtJ -1.81062 0.14536 -12.456 < 2e-16 *** TrtK -1.27570 0.13618 -9.368 < 2e-16 *** TrtL -1.27497 0.14407 -8.850 < 2e-16 *** TrtM -1.09395 0.09406 -11.631 < 2e-16 *** TrtN -0.92772 0.09060 -10.240 < 2e-16 *** TrtO -0.98662 0.11268 -8.756 < 2e-16 *** TrtP -0.81628 0.12265 -6.656 2.82e-11 *** TrtQ -0.61601 0.13223 -4.659 3.18e-06 *** TrtR -0.16271 0.12223 -1.331 0.183128 TrtS -0.22316 0.07618 -2.929 0.003396 ** TrtT -0.29812 0.09266 -3.217 0.001295 ** TrtU 0.08327 0.10664 0.781 0.434884 TrtV 0.05672 0.12220 0.464 0.642526 TrtW 0.42618 0.14042 3.035 0.002404 ** TrtX 0.25690 0.13481 1.906 0.056703 . TrtA:x 0.04941 0.01490 3.316 0.000913 *** TrtB:x 0.10818 0.01171 9.240 < 2e-16 *** TrtC:x 0.14289 0.01149 12.431 < 2e-16 *** TrtD:x 0.19835 0.01470 13.490 < 2e-16 *** TrtE:x 0.23567 0.01713 13.761 < 2e-16 *** TrtF:x 0.28697 0.02275 12.615 < 2e-16 *** TrtG:x 0.05517 0.01148 4.805 1.55e-06 *** TrtH:x 0.10128 0.01086 9.327 < 2e-16 *** TrtI:x 0.16903 0.01283 13.177 < 2e-16 *** TrtJ:x 0.21958 0.01752 12.536 < 2e-16 *** TrtK:x 0.23385 0.02073 11.282 < 2e-16 *** TrtL:x 0.26510 0.02432 10.903 < 2e-16 *** TrtM:x 0.05473 0.01022 5.356 8.51e-08 *** TrtN:x 0.09199 0.01063 8.651 < 2e-16 *** TrtO:x 0.16596 0.01530 10.845 < 2e-16 *** TrtP:x 0.20232 0.02040 9.920 < 2e-16 *** TrtQ:x 0.26029 0.03000 8.677 < 2e-16 *** TrtR:x 0.22405 0.03103 7.221 5.14e-13 *** TrtS:x 0.05280 0.01005 5.253 1.50e-07 *** TrtT:x 0.11057 0.01327 8.335 < 2e-16 *** TrtU:x 0.14664 0.02137 6.863 6.74e-12 *** TrtV:x 0.22058 0.03516 6.274 3.53e-10 *** TrtW:x 0.38493 0.09740 3.952 7.75e-05 *** TrtX:x 0.31591 0.06512 4.851 1.23e-06 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
The singular covariance matrix is on the boundary of the parameter space.
julia> @show getθ(g3) getθ(g3) = [0.242569, -0.00594272, 0.00818418] 3-element Array{Float64,1}: 0.242569 -0.00594272 0.00818418 julia> @show getθ(g4) getθ(g4) = [0.242706, -0.00594923, 0.0082008] 3-element Array{Float64,1}: 0.242706 -0.00594923 0.0082008 julia> show(R"""getME(fit2, "theta")""") RCall.RObject{RCall.RealSxp} Rep.(Intercept) Rep.x.(Intercept) Rep.x 0.0000000000 -0.0152457093 0.0003684742
We can use the converged estimates from the glmm fits as the starting estimates for the glmer fit to show that the log-likelihood is reproduced.
julia> g3th = getθ(g3) 3-element Array{Float64,1}: 0.242569 -0.00594272 0.00818418 julia> @rput g3th 3-element Array{Float64,1}: 0.242569 -0.00594272 0.00818418 julia> R""" t3 <- system.time(fit3 <- glmer(cbind(Dead,Alive) ~ (Trt + 0)/x + (x | Rep), family=binomial(link="cloglog"), data=df, start=g3th, nAGQ=0L)) print(t3) summary(fit3) """ user system elapsed 0.848 0.000 0.850 RCall.RObject{RCall.VecSxp} Generalized linear mixed model fit by maximum likelihood (Adaptive Gauss-Hermite Quadrature, nAGQ = 0) [glmerMod] Family: binomial ( cloglog ) Formula: cbind(Dead, Alive) ~ (Trt + 0)/x + (x | Rep) Data: df AIC BIC logLik deviance df.resid 2928.6 3182.8 -1413.3 2826.6 1029 Scaled residuals: Min 1Q Median 3Q Max -8.6192 -0.2567 0.0000 0.3327 2.5421 Random effects: Groups Name Variance Std.Dev. Corr Rep (Intercept) 0.0588388 0.24257 x 0.0001023 0.01011 -0.59 Number of obs: 1080, groups: Rep, 72 Fixed effects: Estimate Std. Error z value Pr(>|z|) TrtA -3.143661 0.270467 -11.623 < 2e-16 *** TrtB -2.803040 0.213021 -13.159 < 2e-16 *** TrtC -2.581960 0.197763 -13.056 < 2e-16 *** TrtD -2.312249 0.205730 -11.239 < 2e-16 *** TrtE -2.547726 0.220613 -11.548 < 2e-16 *** TrtF -2.120615 0.222264 -9.541 < 2e-16 *** TrtG -2.116955 0.199160 -10.629 < 2e-16 *** TrtH -2.033729 0.185537 -10.961 < 2e-16 *** TrtI -2.239672 0.195561 -11.453 < 2e-16 *** TrtJ -1.811820 0.201949 -8.972 < 2e-16 *** TrtK -1.314749 0.198401 -6.627 3.43e-11 *** TrtL -1.298529 0.202599 -6.409 1.46e-10 *** TrtM -1.097798 0.168795 -6.504 7.84e-11 *** TrtN -0.924785 0.166659 -5.549 2.87e-08 *** TrtO -0.982359 0.180702 -5.436 5.44e-08 *** TrtP -0.816064 0.186234 -4.382 1.18e-05 *** TrtQ -0.615693 0.192594 -3.197 0.001389 ** TrtR -0.152159 0.187648 -0.811 0.417437 TrtS -0.217569 0.159766 -1.362 0.173261 TrtT -0.283216 0.168112 -1.685 0.092049 . TrtU 0.106323 0.177249 0.600 0.548604 TrtV 0.079404 0.187463 0.424 0.671876 TrtW 0.427799 0.198571 2.154 0.031210 * TrtX 0.262170 0.194584 1.347 0.177873 TrtA:x 0.051320 0.013310 3.856 0.000115 *** TrtB:x 0.108349 0.009674 11.200 < 2e-16 *** TrtC:x 0.143247 0.009421 15.206 < 2e-16 *** TrtD:x 0.197537 0.013127 15.048 < 2e-16 *** TrtE:x 0.235773 0.015821 14.903 < 2e-16 *** TrtF:x 0.287073 0.021784 13.178 < 2e-16 *** TrtG:x 0.055580 0.009405 5.910 3.43e-09 *** TrtH:x 0.101309 0.008601 11.779 < 2e-16 *** TrtI:x 0.164692 0.010797 15.254 < 2e-16 *** TrtJ:x 0.219653 0.016244 13.522 < 2e-16 *** TrtK:x 0.241308 0.020555 11.739 < 2e-16 *** TrtL:x 0.269954 0.023979 11.258 < 2e-16 *** TrtM:x 0.054908 0.007810 7.030 2.06e-12 *** TrtN:x 0.091461 0.008302 11.017 < 2e-16 *** TrtO:x 0.162726 0.013834 11.762 < 2e-16 *** TrtP:x 0.202323 0.019327 10.469 < 2e-16 *** TrtQ:x 0.260135 0.029253 8.892 < 2e-16 *** TrtR:x 0.234470 0.032125 7.299 2.91e-13 *** TrtS:x 0.053007 0.007619 6.957 3.47e-12 *** TrtT:x 0.107314 0.011386 9.425 < 2e-16 *** TrtU:x 0.148021 0.021010 7.045 1.85e-12 *** TrtV:x 0.229408 0.036383 6.305 2.87e-10 *** TrtW:x 0.390086 0.098632 3.955 7.65e-05 *** TrtX:x 0.317150 0.065143 4.869 1.12e-06 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1