Maximum Likelihood Model output

Model output

The Maximum Likelihood Models object generates a rather large amount of output:

Worksheet output
Plot output
Text output
Additional output
Additional Simulation output
ODE error messages

Worksheet output

Phoenix model worksheet output depends on the run mode selected and whether the model is population or individual.

BootOmega (Population; Bootstrap): Mean estimated Omega matrix over all sample runs. Includes the mean of the covariance matrix of the random effects and the mean Correlation matrix. Bootstrap scenario results will have a suffix of ‘(B)’.

BootOmegaStderr (Population; Bootstrap): Estimated standard errors for the Omega matrix over all sample runs. The standard errors are derived from the variance of each omega element over all bootstrap sample runs. Bootstrap scenario results will have a suffix of ‘(B)’.

BootOverall (Population; Bootstrap): Reports the replicate number, the model return code, as well as the likelihood (LL) for all bootstrap runs. The table includes runs that were unsuccessful. Bootstrap scenario results will have a suffix of ‘(B)’.

BootSecondary (Population; Bootstrap): Reports secondary parameters over all replicates of a bootstrapped population model. The mean, standard error, CV%, median and 2.5% and 97.5% percentiles are included. Bootstrap scenario results will have a suffix of ‘(B)’.

BootTheta (Population; Bootstrap): Reports fixed effects over all replicates. Includes the mean, standard error, CV%, median and 2.5% and 97.5% percentiles. Bootstrap scenario results will have a suffix of ‘(B)’.

BootVarCovar (Population; Bootstrap): Reports variance-covariance matrix for fixed effects over all replicates. Bootstrap scenario results will have a suffix of ‘(B)’.

ConvergenceData (Population; Simple, Scenarios): Lists values for each model parameter at each iteration.

Doses (Individual/Population; Simple, Scenarios, Predictive Check, Simulation): Reports dosing information for each individual used in the model.

Eta (Population; Simple, Scenarios, Predictive Check, Simulation): Reports post hoc or EBE (empirical Bayesian estimates) of the random effects (eta) for each individual.

EtaCov (Population; Simple, Scenarios, Predictive Check, Simulation): Same information as EtaCovariate and EtaCovariateCat worksheets, but covariates appear as columns so there is one row per subject.

EtaCovariate (Population; Simple, Scenarios, Predictive Check, Simulation): Table by subject of individual continuous covariates and individual post-hoc or EBE of the random effects values.
This worksheet is used to create scatter plots to visualize potential covariate effects.

EtaCovariateCat (Population; Simple, Scenarios, Predictive Check, Simulation): Table by subject of individual categorical covariates and individual post-hoc or EBE of the random effects values.
This worksheet is used to create box-plots to visualize potential covariate effects.

EtaEta (Population; Simple, Scenarios, Predictive Check, Simulation): Same information as the Eta worksheet, but presented in a different format to facilitate plots.

Initial Estimates (Individual/Population; Simple, Scenarios, Bootstrap, Predictive Check, Simulation): Reports the initial estimates used.

NonParEta (Population; Simple, Scenarios, Predictive Check, Simulation): Reports post hoc random effects estimates for each subject when the NonParametric Method is selected. This eta vector for each subject is estimated from the nonparametric algorithm, not from the original fit.

NonParOverall (Population; Simple, Scenarios, Predictive Check, Simulation): Reports post hoc random effects mean and variance/covariance estimates for the population when the NonParametric Method is selected. These are computed directly as the mean and variance/covariance matrix of the nonparametric distribution defined by the nonparametric support points and associated probabilities. In the parametric case, the etas are assumed to have a normal distribution with mean zero and a variance/covariance matrix Omega. A nonparametric mean that is significantly different than zero, or a nonparametric Omega matrix that is significantly different than the parametric Omega computed under the normality assumption, provides evidence that challenges the normality assumption. This table is created only when the NonParametric Method is selected.

NonParSupport (Population; Simple, Scenarios, Predictive Check, Simulation): Reports a set of support points and their probabilities, which add up to 1, that define the discrete nonparametric distribution obtained by the NonParametric Method. Unlike the post hoc eta values found by the parametric method under the normality assumption, the nonparametric eta support vectors do not have a one to one correspondence with specific subjects. The number of supports is determined by the fitting algorithm and never exceeds the number of subjects (it is often considerably less). While the fitted nonparametric distribution is discrete, conceptually the true eta distribution is usually regarded as continuous. The support point positions and associated probabilities in the discrete distribution give an indication of where the most likely regions of the true eta distribution lie. This table is created only when the NonParametric Method is selected.

Observations (Population; Predictive Check): Reports the time elapsed and value for each continuous observation. Available for Predictive check only.

Observations_Categorical (Population; Predictive Check): Reports the time elapsed and value for each categorical and/or count observation. Available for Predictive check only.

Omega (Population; Simple, Scenarios, Predictive Check, Simulation): The estimated Omega matrix, the covariance matrix of the random effects multivariate normal distribution, correlation and summary eta shrinkage statistics. Eta shrinkage data is computed both on an overall summary basis (i.e., for each eta_j a summary shrinkage is computed for all subjects) and on an individual subject basis (i.e., for each subject). Refer to “Shrinkage calculation” for details. The summary eta shrinkage statistics also appear at the end of the Core Status file.
The individual shrinkage statistics are reported in the dmp.txt file, as well as in the output text file bluptable.dat.

OmegaStderr (Population; Simple, Scenarios, Predictive Check, Simulation): The estimated standard errors of the Omega estimators.

Overall (Individual/Population; Simple, Scenarios, Stepwise, Shotgun, Predictive Check, Simulation): Reports model fit diagnostics. Lists the following columns of information:

RetCode: Return code indicating the success of the convergence

LogLik: The log likelihood

-2LL: twice the negative log likelihood

AIC: Akaike information Criterion, a goodness of fit measure based on the log likelihood adjusted for the number of parameters (degrees of freedom) in the fit. It is computed as: AIC = 2k – 2ln(L), where: k is the number of parameters in the model, L is the maximized value of the likelihood function for the model

BIC: Bayes Information Criterion; similar to AIC but using a different dof adjustment (AIC and BIC are only meaningful during comparison of models. The smaller the value, the better the model.)

nParam: Number of parameters (fixed effects parameters+random effects parameters+residual error (eps) parameters)

nObs: Number of observations

nSub: Number of subjects

EpsShrinkage: Epsilon shrinkage, computed as 1-standard deviation (IWRES) for the IWRES values shown on the Residuals worksheet. In cases of multiple dependent variables, the value in the output represents the shrinkage for the last one. Refer to the Core Status tab for the values for each epsilon.

Condition: An estimate of the degree of collinearity in the linearized design matrix. A high condition number may be an indicator of a poor experimental design.

Note: For covariate searches, this is the only output worksheet generated. To generate the full results, change the Run Mode to Scenarios in the Run Options tab and re-execute. Since the best scenario is selected automatically after the covariate search, it will be used during the Scenarios run.

PartialDerivatives (Individual; Simple, Simulation): Reports the partial derivative of each prediction for each fixed effect.

Posthoc (Population; Simple, Predictive Check, Simulation): Reports post hoc values. The posthoc parameters are defined in the form of PML in stparm statements, as for example:

stparm(V=tvV*exp(nV))
stparm(Cl=tvCl*exp(nCl))

Here, tvV and tvCl are fixed effects, and nV and nCl are random effects. The numerical values at the posterior mode for each subject (or in the case of QRPEM, the posterior mean) of the empirical Bayesian distribution of nV and nCl are known as the posthoc values of the random effects nV and nCl. The table of posthoc parameters is created from the corresponding stparm formulas with using the optimal value of the fixed effects at the completion of the fit together with the posthoc values of the random effects.

PredCheck_BQLFraction (Population; Predictive Check): Created when a categorical covariate and BQL are involved, and BQL Fraction is specified. In addition to the stratum-bins, binning minimum and maximum and IVAR values, the observed and predicted fractions, the name of the observation and the observed quantile levels (Q), are reported.

PredCheck_Cat_ObsQ (Population; Predictive Check): Created when the models involve categorical and/or count observation data. All the actual observations for each categorical/count observed variable are collected at each stratum-bin from the original dataset, and the fraction of observations for each of its observed category or grouped level is calculated.

PredCheck_Cat_SimQ (Population; Predictive Check): Created when the models involve categorical and/or count observation data. The simulated observations for each categorical/count observed variable are collected at each stratum-bin, and the quantile values for the requested quantile levels (column Q) are obtained by summarizing the simulated fractions of observations over all the replicates for each simulated category or grouped level.

PredCheck_ObsQ_SimQCI (Population; Predictive Check): Created when the models involve continuous observed variables. All the actual observations are collected at each stratum-bin from the original dataset, sorted, and the quantiles for the requested percentiles are identified. These are the observed quantiles values summarized at each bin time (Column IVAR) and listed in the DV0 column. Minimum and maximum values for the bins are listed (Columns Bin_Min and Bin_Max). If confidence intervals for the simulated quantiles were requested, they are summarized (Column DV) and both the level of the confidence interval (Column QE) and the quantile for the confidence intervals (Column QI) are reported. Since each simulated replicate is like the original dataset, quantiles can be calculated in each replicate. Quantiles of each stratum-bin-observed quantile are the secondary confidence intervals. So, for example, the row with QI=50%, QE=10% is the 10% confidence interval of the prediction of the 50% quantile.

PredCheck_SimQ (Population; Predictive Check): Created the models involve continuous observed variables. The requested quantiles are collected for each stratum and bin. In that stratum-bin, all the simulated observations are sorted and the quantiles are calculated and reported. This worksheet lists the quantiles values (Column DV) for the requested quantiles levels (Column Q) summarized over all the replicates and all IDs. Minimum and maximum values for the bins are listed (Columns Bin_Min and Bin_Max). Quantiles are summarized at each bin time (Column IVAR) and if applicable for each stratification level.

PredCheck_TTE (Population; Predictive Check): Created when the Time-to-event option is checked on the observation tab. Presents Kaplan-Meyer plot for the chosen dependent variable.

PredCheckAll (Population; Predictive Check, Simulation): For each replicate (Column Replicate) and observation name (Column Obsname), it lists the prediction (Column DV) at each time point (Column IVAR) per individual (usually Column ID5 if only 1 ID column) per sort (Column ID1 to ID4) and per stratum (Column STRAT). The STRAT column contains the actual covariate value. If no stratification is selected the STRAT column will have values of 0 for all rows.

Profile (Population; Profile): Summaries of profile runs indicating the parameter, estimate, log-likelihood, return code, delta (perturbation amount) and percent (perturbation percentage) are reported.

Resid2 (Individual/Population; Simple, Scenarios): When two residual errors are requested, this worksheet reports model predictions and residuals for the second residual error.

Residuals (Individual/Population; Simple, Scenarios): Reports model predictions and residuals. Residuals are “noise” that is not explained by the model.

Secondary (Individual/Population; Simple, Scenarios, Predictive Check, Simulation): Reports secondary parameters specified by a user, including:

– estimated secondary parameters (Columns Parameter and Estimate)
– units (Column Units)
– standard errors (Column Stderr)
– relative standard error (also called coefficient of variation) percentage (Column CV%, calculated as 100* Stderr/ParameterValue)
– confidence limits (upper and lower 95%, 2.5% CI, and 97.5%CI)
– variance inflation factor (Column Var. Inf. Factor)

Simulation (Individual; Simulation): The sort variables, requested IVAR, Simulated DV for the requested Y variables and the name of the Y variables (YVarName) are reported.

Simulation Table 01, Simulation Table 02, … (Individual/Population; Predictive Check, Simulation): Created when simulation tables are requested or the TTE checkbox is active in Predictive Check mode.

StrCovariate (Population; Simple, Scenarios, Predictive Check, Simulation): Table by subject of individual continuous covariates and individual model estimated structural parameter values. This worksheet is used to create scatter plots of covariate versus structural parameters to visualize potential covariate effects.

StrCovariateCat (Population; Simple, Scenarios, Predictive Check, Simulation): Table by subject of individual categorical covariates and individual model estimated structural parameter values. This worksheet is used to create box-plots of covariate versus structural parameters to visualize potential covariate effects.

Table01 to Table05 (Optional) (Simple): Optional tables created by the simple run mode or simulation mode for individual models if the user requests them under the Tables Tab. The content depends on what the user enters.

Theta (Individual/Population; Simple, Scenarios, Predictive Check, Simulation): Reports the following:

– estimated fixed effects and standard deviation (eps) parameters (Columns Parameter and Estimate)
– units (Column Units)
– standard errors (Column Stderr)
– relative standard error (or coefficient of variation) percentage (Column CV%), calculated as 100* Stderr/ParameterValue
– confidence limits (upper and lower 95%, Columns 2.5% CI and 97.5%CI)
– variance inflation factor (Column Var. Inf. Factor)

ThetaCorrelation (Individual/Population; Simple, Scenarios, Predictive Check, Simulation): Submatrix of overall parameter correlation matrix corresponding to theta (fixed effect and eps parameters).

ThetaCovariance (Individual/Population; Simple, Scenarios, Predictive Check, Simulation): Submatrix of overall parameter covariance matrix corresponding to theta (fixed effect and eps parameters).

VarCovar (Population; Simple, Scenarios): Variance-Covariance matrix of all the parameter estimators (thetas and omega).