The Phoenix Model Object

Plot output

Phoenix Model plot output depends on the run mode selected and whether the model is population or individual. Most plots for non-population models are prefixed with “Ind” while population plots are prefixed with “Pop”. Users can always disable and enable plots manually in the Plots tab. The table below describes the content of each potential plot that the Phoenix Model creates.

Users can double-click a plot in the Results tab to display it in a separate window for editing. (See the menu options discussion in the Plots chapter of the Data Tools and Plots Guide for plot editing options.)

Double-clicking a point in one of these plots will open the worksheet used to create the plot in a separate window and highlight the corresponding row of data.

Plot Output	Content
Boot Omega Histogram Population	Histogram plot of omega elements for all bootstrap replicates.
Boot Theta Histogram Population	Histogram plot of fixed effects and secondary parameters for all bootstrap replicates.
Convergence Population	Plots the values for each model parameter at each iteration.
Covariate Box Plots Population	Box plots of specified categorical covariates vs Etas. Covariates of interest need to be specified but covariate effects do not need to be part of the model to obtain these plots. Note: The labels in these plots will contain the values of the categorical covariate, even if a categorical name was specified in the Covar. Type tab.
Covariate Plots Population	Scatter plots of specified continuous covariates vs Etas. Covariates of interest need to be specified but covariate effects do not need to be part of the model to obtain these plots. These plots are created for all covariates but if a covariate changes over time the first value is selected for plotting purposes which might make that specific plot meaningless. This will often occur with plots where dose is a covariate.
CWRES Histogram Population	Histogram plot of the conditional weighted residual values. If scenarios are present each histogram is presented by scenario.
CWRES vs IVAR Population	Plot of CWRES (conditional weighted residuals; a proposed replacement for the classical WRES (weighted residuals) goodness of fit statistic) against IVAR (the independent variable; typically time in a PK fit, concentration or dose in a PD fit). Values of CWRES should be approximately N(0,1) and hence concentrated between y= –2 and y=+2. Values significantly above 3 or below –3 are suspect and may indicate a lack of fit and/or model misspecification.
CWRES vs PRED Population	Same as CWRES vs IVAR, but with the population predictions (i.e., the predictions obtained by setting the random effect values to zero) used for the X-axis.
CWRES vs TAD Population	Same as CWRES vs IVAR, but independent variable is Time After Dose.
DV vs IPRED Individual/Population	Plot of the dependent variable (DV, e.g., concentrations for PK models) versus individual predicted values (IPRED, e.g., predicted concentrations). Individual prediction obtained by setting random effects to the 'post hoc' or empirical Bayesian estimate of the random effects for the individual from which the DV observation was made. Thus the plot shows observed vs fitted values of the model function. Ideally, points should fall close to the line of unity y=x.
DV vs IPRED Lattice Individual	Same as DV vs IPRED plots for individual models, but latticed by sort.
DV vs IPRED Log Population	DV vs IPRED plot for population models, with log transformed DV and IPRED axes.
DV vs PRED Population	Analog of DV vs IPRED with population predictions used instead of individual predictions. Since population predictions are typically less accurate, this plot will show larger deviations around the y=x line of unity than DV vs IPRED.
DV vs PRED Log Population	DV versus PRED plot with log transformed DV and PRED axes.
DV vs TAD Population	Plot of observations vs Time After Dose.
DV, IPRED vs IVAR Individual/Population	Plot of the dependent variable (DV), and individual predicted estimates (IPRED) versus the independent variable (IVAR, e.g., time). Ideally, points should fall close to the line of unity y=x.
DV, IPRED vs IVAR Lattice Individual/Population	Same plot as DV, IPRED vs IVAR, but latticed by sort.
DV, IPRED vs DV2, IPRED2 Lattice Individual/Population	Plots (latticed by individual) of DV (dependent variable), IPRED (individual prediction) vs a second dependent variable (DV2), and a second individual prediction (IPRED2). Generated when there are two residual errors requested.
DV2, IPRED2 vs DV, IPRED Lattice Individual/Population	Inverse of DV, IPRED vs DV2, IPRED2 Lattice. Generated when there are two residual errors requested.
DV, IPRED vs TAD Individual/Population	Plot of dependent variable and individual prediction vs Time After Dose.
DV, IPRED vs TAD Lattice Individual/Population	Same plot as DV, IPRED vs TAD, but latticed by sort.
DV, IPRED,PRED vs IVAR Lattice Population	Plots (latticed by individual) containing all population observations, population prediction, and individual predictions vs the independent variable.
DV, IPRED,PRED vs TAD Lattice Population	Plots (latticed by individual) containing observations, population prediction, and individual predictions vs Time After Dose.
DV, PRED vs IVAR Population	Plots containing observations and population prediction vs the independent variable.
DV, PRED vs IVAR Lattice Population	Same plot as DV, PRED vs IVAR, but latticed by individual.
DV, PRED vs TAD Population	Plots containing observations and population prediction vs Time After Dose.
DV, PRED vs TAD Lattice Population	Same as DV, PRED vs TAD, but latticed.
Eta Histogram Population	Histogram plot of the eta values. If scenarios are present, each histogram is presented by scenario.
Eta QQ Population	Quantile-quantile plot for each eta in the model. If the components of eta are well described by a normal distribution, plotted values will fall roughly along a straight line of unity y=x. Significant deviations from normality, particularly in the tails of the distribution, can be seen by deviations from this line.
Eta Scatter Population	Scatter plot of all combinations of etas also known as scatter plot matrix of etas. This allows visual confirmation of correlations between random effects.
IWRES vs IPRED Individual/Population	Plot of individual weighted residuals (IWRES) versus individual predicted values (IPRED, e.g., predicted concentrations). Ideally, the blue line should be at 0 and the red line (with its negative reflection) should not show any fanning. Fanning indicates room for improving the distribution of residuals.
IWRES vs IPRED Lattice Individual	Plots (latticed by sort) of individual weighted residuals (IWRES) versus individual predicted values (IPRED, e.g., predicted concentrations).
IWRES vs IVAR Individual	Plot of individual weighted residuals (IWRES) versus the independent variable (IVAR, e.g., time). Ideally, the blue line should be at 0 and the red line (with its negative reflection) should not show any fanning. Fanning indicates room for improving the distribution of residuals.
IWRES vs IVAR Lattice Individual	Plots (latticed by sort) individual weighted residuals (IWRES) versus the independent variable (IVAR, e.g., time).
IWRES vs TAD Individual	Plot of individual weighted residuals (IWRES) versus time after dose.
IWRES vs TAD Lattice Individual	Plots (latticed by sort) of individual weighted residuals (IWRES) versus time after dose.
NP Covariate Box Plots Population	Box plots of specified covariates vs nonparametric Eta.
NP Covariate Plots Population	Scatter plots of specified covariates vs nonparametric Eta.
NP Eta Scatter Population	Scatter plot of all combinations of nonparametric Etas.
Partial Derivatives Individual	Plot of the partial derivative of each prediction with respect to each fixed effect (which is 1-to-1 with structural parameters).
PCWRES vs IVAR Population	Plot of PCWRES (predictive check weighted residuals; a proposed replacement for the classical WRES (weighted residuals) goodness of fit statistic) against IVAR (the independent variable; typically time in a PK fit, concentration or dose in a PD fit). Values of CWRES should be approximately N(0,1) and hence concentrated between y= –2 and y=+2. Values significantly above 3 or below –3 are suspect and may indicate a lack of fit and/or model misspecification.
PCWRES vs PRED Population	Same as PCWRES vs IVAR, but with the population predictions (i.e., the predictions obtained by setting the random effect values to zero) used for the X-axis.
PCWRES vs TAD Population	Same as PCWRES vs IVAR, but independent variable is Time After Dose.
Posthoc Histogram Population	Histogram plot of the posthoc values. If scenarios are present, each histogram is presented by scenario.
PredCheck_TTE Population	Created when Predictive Check mode is used and the Time-to-event option is specified on the observation tab. This is a Kaplan-Meier plot that describes the fraction of patients which reached some state to a specific point of time with confidence interval of observations.
PredCheckCat ObsQ_SimQ Population	A series of plots created when Predictive Check mode is used with categorical observations. The plots show the observed fraction of each category level versus the chosen X-axis and the corresponding confidence interval.
PredCheck_ObsQ_SimQ Population	Plot created only for Predictive Check mode. For each stratification, the observed quantiles are superimposed with the predictive check quantiles over the observed data. The default color scheme is as follows: - Red lines: observed quantiles - Black lines: predicted quantiles - Blue symbols: observed data
PredCheck_ObsQ_SimQCI Population	Plot created only for Predictive Check mode. For each stratification, the observed quantiles and the confidence intervals of the simulated quantiles are plotted (with observed data overlaid). The default color scheme is as follows: - Black lines: observed quantiles - Gray lines: confidence intervals of the predicted quantiles - Blue symbols: observed data The meaning of the default legend is: - 05%.05%=5% confidence interval of the 5% quantile of simulated data - 05%, 95%=95% CI on the 5% quantile of simulated data - 05%=5% quantile of observed (raw observations) and so on.
QQ CWRES Population	A qq (quantile-quantile) plot of the components of CWRES. If the components of CWRES are well described by a normal distribution, plotted values will fall roughly along a straight line of unity y=x. Significant deviations from normality, particularly in the tails of the distribution, can be seen by deviations from this line. Can be considered as a diagnostic of model mis-specification.
QQ IWRES Individual/Population	Quantile-quantile plot of the individual weighted residuals. If the components of IWRES are well described by a normal distribution, plotted values will fall roughly along a straight line of unity y=x. Significant deviations from normality, particularly in the tails of the distribution, can be seen by deviations from this line. Can be considered as a diagnostic of model misspecification.
QQ IWRES Lattice Individual	Quantile-quantile plot (latticed by sort) of the individual weighted residuals (IWRES). If the components of IWRES are well described by a normal distribution, plotted values will fall roughly along a straight line of unity y=x. Significant deviations from normality, particularly in the tails of the distribution, can be seen by deviations from this line. Can be considered as a diagnostic of model misspecification.
QQ PCWRES Population	Same as QQ CWRES but with PCWRES replacing CWRES.
QQ WRES Population	Same as QQ CWRES but with WRES replacing CWRES.
Simulation Individual	In Simulation run mode, plots (latticed by subject) the simulated and observed dependent variable versus independent variable (IVAR).
Str Covariate Box Plots Population	Box plots of the specified categorical covariates vs fixed effects (i.e., model structural parameters). Covariates of interest need to be specified but covariate effects do not need to be part of the model to obtain these plots.
Str Covariate Plots Population	Plots of the specified continuous covariates vs fixed effects (i.e., model structural parameters). Covariates of interest need to be specified but covariate effects do not need to be part of the model to obtain these plots.
WRES Histogram Population	Histogram plot of the weighted residuals.
WRES vs IVAR Population	WRES (weighted residuals) vs IVAR (independent variable) plot.
WRES vs PRED Population	WRES (weighted residuals) vs PRED plot.
WRES vs TAD Population	Plot of the weighted residuals vs time after dose.

Note:Default plots from library Phoenix models will always generate plots with Time After Dose (TAD) in the X-axis. These plots are generated by default, even if time is not the x-variable in the model (for example pharmacodynamic Emax models) or if dose is an inappropriate concept in the model. In these cases, these plots just duplicate the plots with IVAR in the X-axis.

IWRES is the Individual Weighted RESidual.

IWRES=(DV – IPRED)/STD

(1)

where IPRED is the individual prediction and STD is the estimate of the standard deviation of the observation.

Assuming the residual error model is correct, IWRES should be a N(0,1) random variable. Then:

STD=stdev(eps)*sqrt(var(IPRED))

(2)

where stdev(eps) is the estimated standard deviation for the normal residual error eps and var(IPRED) is the variance function.

For example, for an additive residual error model:

DV=IPRED+eps
var=1
STD=stdev(eps)

(3)

For a multiplicative (proportional) residual error model:

DV=IPRED+IPRED*eps
var=IPRED*IPRED
STD=abs(IPRED)*eps

(4)

WRES vs CWRES: WRES (weighted residuals) is a standardized estimate of the components of the population residual vector (observations-population predictions). If a Gaussian observation and random effect model is correct, the components of the population residual vector for a given individual are approximately multivariate normally distributed with a covariance matrix reflecting correlation induced by the fact that all observations from a given individual share a common set of random effect values specific to that individual. The computation of WRES decorrelates these values and standardizes them to unit variance, so the components of WRES are approximately independent N(0,1) random variables. Plots of WRES against population predictions (or QQ plots of WRES) are often used as a diagnostic of model misspecification, excessive deviations from the nominal N(0,1) distribution being regarded as indicative of model misspecification.

Recently, Hooker, Staatz, and Karlsson (2007). noted that WRES is not always a reliable indicator of model misspecification when an FOCE method is used and gave an example where the WRES plot from a highly mis-specified model was better, or closer to what would be expected from the nominal N(0,1) case, than the WRES plot from a correctly specified model. They proposed a modification to WRES as CWRES (“conditional WRES”) which, in the case of correctly specified models, often results in a statistic for which the N(0,1) approximation is better than in the WRES case. In the example, CWRES correctly differentiates between the correctly specified and mis-specified model whereas WRES is completely misleading. CWRES has been gaining acceptance and is now one of the diagnostic outputs for POP PK modeling suggested by the latest European Medicines Agency guidelines.

Text output

Text Output	Content
Compiler Output	The output from compiling the model.c file, the C language source program that results from building a model in the DME or in Phoenix Modeling Language text. Any errors in compilation will be noted here. If there are any errors, the model fitting process will not run.
Core Output	A text version of output from the running the fitting process. Includes fitted parameter values, eigenvalues, condition number of the matrix of partial derivatives (square root of the ratio of the largest to the smallest eigenvalue), log likelihoods, post hoc estimates, and a table of goodness-of-fit items (data values, individual and population predictions and residuals, etc.).
Core Status	A text version which lists the minimization process, the minimization engine used, as well as a summary of the Optimal Parameter Estimates and eta shrinkage statistics. (See the Omega worksheet description for information on shrinkage computations.)
Model Text	The PML (Phoenix Modeling Language) code that is generated as the model is specified, plus the column definitions generated when a dataset is mapped.
Settings	The settings sent into the compiler and runtime engine. Includes mappings and model code.
Status Window Text	Lists the text that appeared in the Status Window during execution.
Stepwise Text	Stepwise Run Mode only. Explains decision on which covariates to add or subtract in the next step of the covariate search.
Warnings and Errors	If any runtime errors are encountered, they are written to this file.

Additional output

Additional output consists of files that are not be viewed within Phoenix but can be exported and viewed in other software. Additional output file remain part of a Phoenix project but they cannot be used as input to any Phoenix object downstream. Often files created as additional output have the potential to be very large (e.g., simulation). If the user wishes to export and import back these files into Phoenix, caution should be taken to ensure sufficient memory resources.

Output Name	Content
BootSubj(B).csv	Created for bootstrap mode run. Table showing the subjects that were randomly included in each sample as well as the numbers of tries for those samples.
dmp.txt	These files are created for population models with the Simple mode. The intention is for R users to be able to post-process these files. The dmp.txt files can be loaded into R and they can be used for processing of Phoenix NLME results in R. Individual eta shrinkage statistics are listed at the end of the file. See the Omega worksheet description for information on shrinkage computations.
Rawout.csv	These files list the estimates of Thetas, Omega, and Log-Likelihood for each executed sample.
Rawsimout.csv	These files list the simulation estimates for each executed sample. When running simulation in an individual model, one Rawsimout.csv file is generated for each subject.
Rawsimtbl01.csv–Rawsimtbl05.csv	Optional tables created by the Predictive Check mode, if Tables are requested under the Tables tab. The content depends on what the user enters.

Additional Simulation output

Optionally, if a user selects a folder in which to copy simulation results then csv files are saved to that directory. This is only available for population models in Simulation run mode.

Output Name	Content
Predout.csv	The same as the PredCheckAll worksheet described above (see “Worksheet output”).
simtbl01.csv–simtbl05.csv	Optional tables created by the Simulation mode, if Add Sim Table is used on the Options tab. The content depends on what the user enters.

ODE error messages

When the ODE solver returns an error code, Phoenix NLME reports the error messages to the user so that appropriate actions may be taken. The error messages may appear in either Core Status text output or Warnings and Errors text output or both. If any ODE error message appears in the Warnings and Errors text output, then the corresponding results obtained, if there are, should not be trusted and/or be interpreted with care.

For the estimation mode, the ODE error message may appear in either Core Status text output or both Core Status text output and Warnings and Errors text output. If there are occasional occurrences of ODE error messages during some early or intermediate iterations, then it is probably due to some unrealistic parameter values found during the intermediate search, and hence there is typically no need to worry about these (as the optimization is eventually able to go to the right direction). However, if the error message appears during almost all the iterations, then the engine may stuck in a bad/inappropriate region, and hence the estimation results obtained may not be reliable. Moreover, if the error message continues to show up during the standard error calculations step and/or during the final stage for preparing worksheet outputs (e.g., Residual and Overall worksheets), then the corresponding results obtained should not be trusted.

For the simulation modes (including VPC), a 0-iteration fit is performed before the simulation run to populate worksheet outputs. Errors occurred during either the 0-iteration fit or simulation runs are recorded in some specific files and reported in the Warnings and Errors text output, and the corresponding results obtained, if there are, should not be trusted. For example, if ODE errors occurred during generation of simulation table worksheets, then simulation tables should be taken with care; if ODE errors occurred during predictive check step, predcheck outputs should be taken with care. It is worth pointing out that 0-iteration fit error files are wiped by subsequent simulation runs. To see them, the user should re-run the model in simple mode.

Typically, the ODE errors can be avoided by taking appropriate actions. For example, if the error message is about maximum number of function evaluations exceeded, then one can increase the value of “ODE max step” (by clicking the “Advanced <<” button in the Run Option tab) to be sufficiently large to avoid such error. While, for some cases, the easiest/best way is just to switch to a different solver. For example, if the model is suspected to be stiff, then one needs to switch to auto-detect or stiff solver (from the "max ODE" menu located in the Run Options tab).