The Phoenix Model Object

Phoenix Model output

The Phoenix Model object generates a rather large amount of output:

Worksheet output

Phoenix model worksheet output depends on the run mode selected and whether the model is population or individual. The first table indicates what output is created by each run mode and the second table describes the content of each worksheet.

Worksheet Output	Content
BootOmega Population	The mean estimated Omega matrix over all sample runs. Includes the mean of the covariance matrix of the random effects and the mean Correlation matrix. Results created by bootstrapping always display a scenario of '(B)’.
BootOmegaStacked Population	Estimated Omega matrix for each replication. Taking the average over the replicates per parameter yields the information presented in BootOmega worksheet. Results created by bootstrapping always display a scenario of '(B)'.
BootOmegaStderr Population	Estimated standard errors for the Omega matrix over all sample runs. The standard errors are derived from the variance of each omega element over all bootstrap sample runs. Results created by bootstrapping always display a scenario of ‘(B)’.
BootOverall Population	Reports the replicate number, the model return code, as well as the likelihood (LL) for all bootstrap runs. The table includes runs that were unsuccessful. Results created by bootstrapping always display a scenario of '(B)'.
BootSecondary Population	Reports secondary parameters over all replicates of a bootstrapped population model. The mean, standard deviation (eps) parameters, standard errors, median and 2.5% and 97.5% percentiles are included. Results created by bootstrapping always display a scenario of '(B)'.
BootTheta Population	Reports fixed effects over all replicates. Includes the mean, standard deviation (eps) parameters, standard errors, median and 2.5% and 97.5% percentiles. Results created by bootstrapping always display a scenario of ‘(B)’.
BootThetaStacked Population	Same information as BootTheta worksheet, but stacked by thetas.
ConvergenceData Population	Lists values for each model parameter at each iteration.
Doses Individual/Population	Reports dosing information for each individual used in the model.
Eta Population	Reports post hoc or EBE (empirical Bayesian estimates) of the random effects (eta) for each individual.
EtaCov Population	Same information as EtaCovariate and EtaCovariateCat worksheets, but covariates appear as columns so there is one row per subject.
EtaCovariate Population	Table by subject of individual continuous covariates and individual post-hoc or EBE of the random effects values. This worksheet is used to create scatter plots to visualize potential covariate effects.
EtaCovariateCat Population	Table by subject of individual categorical covariates and individual post-hoc or EBE of the random effects values. This worksheet is used to create box-plots to visualize potential covariate effects.
EtaEta Population	Same information as the Eta worksheet, but presented in a different format to facilitate plots.
EtaStacked Population	Same as Eta worksheet, but stacked by parameter.
Ind.Quantiles Population	Reports the requested quantiles (Column Q) values (Column DV) for the predictive check simulation per ID and IVAR (independent variable) over all the replicates. If the predictive check was stratified, then the column STRAT will display the stratification value.
InitialEstimates Individual/Population	Reports the initial estimates used.
NonParEta Population	Reports post hoc random effects estimates for each subject when the NonParametric Method is selected. This eta vector for each subject is estimated from the nonparametric algorithm, not from the original fit.
NonParOverall Population	Reports post hoc random effects mean and variance/covariance estimates for the population when the NonParametric Method is selected. These are computed directly as the mean and variance/covariance matrix of the nonparametric distribution defined by the nonparametric support points and associated probabilities. In the parametric case, the etas are assumed to have a normal distribution with mean zero and a variance/covariance matrix Omega. A nonparametric mean that is significantly different than zero, or a nonparametric Omega matrix that is significantly different than the parametric Omega computed under the normality assumption, provides evidence that challenges the normality assumption. This table is created only when the NonParametric Method is selected.
NonParStacked Population	Reports marginal and cumulative probabilities of the support point values for each eta, sorted in ascending order and stacked by eta value. This table is created only when the NonParametric Method is selected.
NonParSupport Population	Reports a set of support points and their probabilities, which add up to 1, that define the discrete nonparametric distribution obtained by the NonParametric Method. Unlike the post hoc eta values found by the parametric method under the normality assumption, the nonparametric eta support vectors do not have a one to one correspondence with specific subjects. The number of supports is determined by the fitting algorithm and never exceeds the number of subjects and often is considerably less. While the fitted nonparametric distribution is discrete, conceptually the true eta distribution is usually regarded as continuous. The support point positions and associated probabilities in the discrete distribution give an indication of where the most likely regions of the true eta distribution lie. This table is created only when the NonParametric Method is selected.
Observations Population	Reports the time elapsed and value for each observation.
Omega Population	The estimated Omega matrix, the covariance matrix of the random effects multivariate normal distribution, correlation and summary eta shrinkage statistics. Eta shrinkage data is computed both on an overall summary basis (i.e., for each eta_j a summary shrinkage is computed for all subjects) and on an individual subject basis (i.e., for each subject). Refer to “Shrinkage calculation” for details. The summary eta shrinkage statistics also appear at the end of the Core Status file. The individual shrinkage statistics are reported in the dmp.txt file, as well as in the output text file bluptable.dat.
OmegaStderr Population	The estimated standard errors of the Omega estimates.
Overall Individual/Population	Reports model fit diagnostics. Lists the following columns of information:
	•RetCode: Return code indicating the success of the convergence
	•LogLik: The log likelihood
	•-2LL: twice the negative log likelihood
	•AIC: Akaike information Criterion, a goodness of fit measure based on the log likelihood adjusted for the number of parameters (degrees of freedom) in the fit. It is computed as: , where: k is the number of parameters in the model, L is the maximized value of the likelihood function for the model
	•BIC: Bayes Information Criterion; similar to AIC but using a different dof adjustment (AIC and SBC are only meaningful during comparison of models. A smaller value is better, negative is better than positive, and a more negative value is even better.)
	•nParam: Number of parameters (fixed effects parameters+random effects parameters+residual error (eps) parameters)
	•nObs: Number of observations
	•nSub: Number of subjects
	•EpsShrinkage: Epsilon shrinkage, computed as 1-standard deviation (IWRES) for the IWRES values shown on the Residuals worksheet
	•Condition: An estimate of the degree of collinearity in the linearized design matrix. A high condition number may be an indicator of a poor experimental design.
	Note: For covariate searches, this is the only output worksheet generated. To generate the full results, change the Run Mode to Scenarios in the Run Options tab and re-execute. Since the best scenario is selected automatically after the covariate search, it will be used during the Scenarios run.
PartialDerivatives Individual/Population	Reports the partial derivative of each prediction for each fixed effect.
Posthoc Individual/Population	Reports post hoc values. The posthoc parameters are defined in the form of PML in stparm statements, as for example:
	•stparm(V=tvV*exp(nV))
	•stparm(Cl=tvCl*exp(nCl))
	Here, tvV and tvCl are fixed effects, and nV and nCl are random effects. The numerical values at the posterior mode for each subject (or in the case of QRPEM, the posterior mean) of the empirical Bayesian distribution of nV and nCl are known as the posthoc values of the random effects nV and nCl. The table of posthoc parameters is created from the corresponding stparm formulas with using the optimal value of the fixed effects at the completion of the fit together with the posthoc values of the random effects.
PosthocStacked Individual/Population	Same as Posthoc worksheet, but stacked by parameter.
PredCheckAll Population	Created when Predictive Check mode is used. For each replicate (Column Replicate) and observation name (Column Obsname), it lists the prediction (Column DV) at each time point (Column IVAR) per individual (usually Column ID5 if only 1 ID column) per sort (Column ID1 to ID4) and per stratum (Column STRAT). The STRAT column contains the actual covariate value. If no stratification is selected the STRAT column will have values of 0 for all rows.
PredCheck_TTE Population	Created when Predictive Check mode is used and the Time-to-event option is specified on the observation tab.
PredCheck_ObsQ_SimQCI Population	Created when Predictive Check mode is used. All the actual observations are collected at each stratum-bin from the original dataset, sorted and the quantiles for the requested percentiles are identified. These are the observed quantiles values summarized at each bin time (IVAR) and listed in the DV0 column. In addition, if the option of confidence intervals for the simulated quantiles were requested, they are also presented in this worksheet. The confidence intervals are summarized under the column DV. The level of the confidence interval is column QE and the column QI indicates the quantile for the confidence intervals. Since each simulated replicate is like the original dataset, quantiles can be calculated in each replicate. Quantiles of the each stratum-bin-observed quantile, are the secondary confidence intervals. So, for example, the row with QI=50%, QE=10% is the 10% confidence interval of the prediction of the 50% quantile.
PredCheck_SimQ Population	Created when Predictive Check mode is used. The requested quantiles are collected for each stratum and bin. In that stratum-bin, all the simulated observations are sorted and the quantiles are calculated and reported. This worksheet lists the quantiles values (Column DV) for the requested quantiles levels (Column Q) summarized over all the replicates and all IDs. Quantiles are summarized at each bin time (Column IVAR) and if applicable for each stratification level.
Profile Population	In Profile runs, summaries of profile runs indicating the parameter, estimate, log-likelihood, return code, delta (perturbation amount) and percent (perturbation percentage) are reported.
Residuals Individual/Population	Reports model predictions and residuals. (A Resid2 worksheet is also generated if there are two residual errors requested.) Residuals are “noise” that is not explained by the model.
Secondary Individual/Population	Reports secondary parameters specified by a user, including: - estimated secondary parameters (Columns Parameter and Estimate) - units (Column Units) - standard errors (Column Stderr) - standard error percentage (Column CV%, calculated as 100* Stderr/ParameterValue) - confidence limits (upper and lower 95%, 2.5% CI, and 9.5%CI) - variance inflation factor (Column Var. Inf. Factor)
Simulation Individual	The sort variables, requested IVAR, Simulated DV for the requested Y variables and the name of the Y variables (YVarName) are reported.
Simulation Table 01, Simulation Table 02, ... Population	Created for simulation TTE when Predictive Check mode is used and the Time-to-event is selected.
StrCovariate Population	Table by subject of individual continuous covariates and individual model estimated structural parameter values. This worksheet is used to create scatter plots of covariate versus structural parameters to visualize potential covariate effects.
StrCovariateCat Population	Table by subject of individual categorical covariates and individual model estimated structural parameter values. This worksheet is used to create box-plots of covariate versus structural parameters to visualize potential covariate effects.
Table01 to Table05 (Optional) Individual	Optional tables created by the simple run mode or simulation mode for individual models if the user requests them under the Tables Tab. The content depends on what the user enters.
Theta Individual/Population	Reports the following: - estimated fixed effects and standard deviation (eps) parameters (Columns Parameter and Estimate) - units (Column Units) - standard errors (Column Stderr) - standard error percentage (Column CV%, calculated as 100* Stderr/ParameterValue) - confidence limits (upper and lower 95%, Columns 2.5% CI and 9.5%CI) - variance inflation factor (Column Var. Inf. Factor)
ThetaCorrelation Individual/Population	Submatrix of overall parameter correlation matrix corresponding to fixed effect and eps parameters.
ThetaCovariance Individual/Population	Submatrix of overall parameter covariance matrix corresponding to fixed effect and eps parameters.
VarCovar Population	Variance-Covariance matrix of estimates of thetas and omega.

Return Codes

The success of a Phoenix model fit can usually be diagnosed by the engine return code. The return code is a numeric integer value, usually from 1 to 7, but it can also be zero or negative in special cases when convergence is not requested or achieved. Return codes are presented in the Overall Worksheet(s) in the RetCode column, as well as in the Core Output and Core Status text output.

All Phoenix NLME methods, except QRPEM, are based on the direct numerical optimization of a log likelihood or approximate log likelihood function. The fundamental optimization algorithm used is the well-known unconstrained BFGS quasi-Newton UNCMIN optimizer presented as pseudo-code in Dennis, J.E. and Schnabel, R.B., Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM, 1996 (re-issue of classic 1983 book). Several customized versions of UNCMIN based on this pseudo-code have been implemented in Phoenix for the various engines. The return codes in Phoenix for all engines, except QRPEM, are usually based directly on the ITRMCD return codes of 1 through 5 in UNCMIN. Additional codes of 0, 6, 7, and negative return codes that do not appear in the original UNCMIN have been added to deal with special situations and, in some cases, the meaning of a return code of 4 has been changed. The actual methods of determining the return codes in UNCMIN are rather technical. What follows is a brief summary and the interested user should refer to the reference mentioned above for more detail.

Basically, UNCMIN return codes of 1, 2, and 3 indicate that the optimizer was ‘probably successful’ in reaching at least a local optimum of the log likelihood function, with 1 giving the strongest evidence of success and 3 the weakest. More specifically:

1 means that the ‘scaled gradient’ was reduced below a threshold

2 means that further progress does not seem possible due to relative changes in the parameters becoming too small on successive iterations

3 means that the line search step in the quasi-Newton direction failed to locate a sufficiently better objective value than the current value

4 means that the user-defined maximum number of iterations was reached before any other termination criterion was achieved (note that the meaning of this code has been changed as described below for the FO, Naive pooled, FOCE L-B, and IT2S-EM engines)

5 means that the UNCMIN algorithm failed to compute an acceptable step-size within the allowable internal limit on number of trials.

The FO, Naive pooled, FOCE L-B, and IT2S-EM methods in Phoenix require a sequence of UNCMIN optimizations of a log-likelihood or approximate log-likelihood function. Typically, successful convergence is recognized if the final two members of this sequence give essentially identical results with a UNCMIN return code of 1, 2, or 3. Thus, at least two ‘successful’ UNCMIN optimizations are required for convergence. If convergence is achieved, the return code of 1, 2, or 3 for these engines is the return code from UNCMIN for the final run in the sequence. If these engines do not converge within the user-specified number of UNCMIN attempts (for these engines, the iteration counter is the number of runs of UNCMIN), a return code of 4 is made. If the sequence of UNCMIN optimization is determined to be non-convergent before hitting the maximum number of iterations, a negative return code is made, where the absolute value of the return code is the UNCMIN return code from the final UNCMIN run in the sequence.

Unlike the other UNCMIN-based engines, the FOCE ELS method can sometimes be determined to have converged based on a single UNCMIN run, if the UNCMIN return code is 1 or 2. For a return code of 3, the FOCE ELS algorithm attempts a restart from the final solution so obtained. Here, the iteration counter counts the total number of line searches made throughout all restarts, and thus a return code of 4 means that the total number of such line searches over all attempts has hit the user-specified maximum. If the FOCE ELS algorithm cannot improve upon its original attempt that resulted in a return code of 3, then the results of that attempt are reported with a return code of 3. If, however, better (higher overall log-likelihood) results are obtained, the current attempt then becomes the reference and a new UNCMIN run attempt is made to improve the log-likelihood. The final return code reflects the return code of the best attempt.

To these basic UNCMIN return codes, several others have been added, include 0, 6, 7, and various negative return codes indicating non-convergence. The following table summarizes the meaning of the non-negative integer return codes within the context of each method. Negative return codes generally apply in situations where convergence is not achieved and are also described below.

Return Code	Meaning
0	The user ran the engine in evaluation model only, no optimization was requested, and the solution that is returned was the same one entered as a starting solution. (Not used in the original UNCMIN algorithm.)
1	Convergence was achieved and the final UNCMIN optimization successfully terminated with a scaled gradient below an internal tolerance (this is the ‘best’ possible return code). For QRPEM, a return code of 1 indicates successful convergence.
2	Convergence was achieved and optimization of the final UNCMIN run terminated when parameter change fell below internal tolerance on successive iterations of the final UNCMIN run. QRPEM does not use a return code of 2.
3	Convergence was achieved and optimization of the final UNCMIN run terminated when insufficient objective function improvement was made during the line search step along the quasi-Newton direction. QRPEM does not use a return code of 3.
4	For FO, FOCE L-B, Naive pooled, and IT2S-EM, the maximum number of UNCMIN runs was reached before any other termination criterion was achieved. For FOCE ELS, the maximum total number of linear searches over all UNCMIN runs was reached before convergence was achieved. For QRPEM, the maximum number of QRPEM iterations was reached before convergence was achieved, where a QRPEM iteration is one complete cycle through sampling all subjects and making a parameter update based on the samples.
5	Optimization was terminated due to internal failure of UNCMIN to find a suitable step size (this is an error termination). QRPEM does not use a return code of 5.
6	The user manually terminated the engine from the user interface Stop Early button in the Progress window before convergence. (Not used in the original UNCMIN algorithm.)
7	If the line search procedure gets into a region where some bad/inappropriate function values are obtained and it fails to find a good one after several attempts, then the engine stops proceeding further and reports the last good results to the user.

Negative return codes generally indicate some kind of convergence failure and often a more definitive reason for the convergence failure can be found in the Core Status file in the Text Output section of the Results tab or in the file nlme7engine.log (if command line invocation is used). For UNCMIN-based engines, the absolute value of the return code represents the UNCMIN return code on the UNCMIN run that resulted in a determination of convergence failure. QRPEM uses a return code of -4 if non-convergence is detected.

Relatively uncommon, the user can see the return code 40 for FOCE ELS. It indicates that the engine has gone through numerous cycles and was unable to find a solution better than the previous solution by a certain tolerance (.01).

For the Naive Pool engine only, Variance Inflation Factors (VIF) are computed per time entry during simulation runs and presented in several worksheets that tabulate parameter estimates. These are calculated in the same matter as WinNonlin Classical models but are computed regardless of whether the model is simulated or if a fit is run.

VIFs are based on a simple linearization of the model. If the Naive pool engine is run with Number of Iterations > 0, then the linearization is done at the final fitted parameter values. If it is just a simulation, where the model evaluation with Number of iterations=0, then the linearization is done at the initial input values of the parameters. In this case the observed data values never enter into the VIF computation.

For simulations VIFs can be used to compare experimental designs. For fits, if the square roots of the VIFs are multiplied by the epsilon estimate it gives standard error estimates based on the linearized model. Standard error of estimates are normally computed from the Hessian of the objective function. If the Hessian is not successful, the Phoenix Model will then automatically report the standard errors based on successful VIF computation instead. If this occurs, it is indicated in the Core Status text output.