The success of a Phoenix model fit can usually be diagnosed by the engine return code. The return code is a numeric integer value, usually from 1 to 7, but it can also be zero or negative in cases when convergence is not requested or achieved. Return codes are presented in the Overall Worksheet(s) in the RetCode column, as well as in the Core Output and Core Status text output.
All Phoenix NLME methods, except QRPEM, are based on the direct numerical optimization of a log likelihood or approximate log likelihood function. The fundamental optimization algorithm used is the well-known unconstrained BFGS quasi-Newton UNCMIN optimizer presented as pseudo-code in Dennis, J.E. and Schnabel, R.B., Numerical Methods for Unconstrained Optimization and Nonlinear Equations, SIAM, 1996 (re-issue of classic 1983 book). Several customized versions of UNCMIN based on this pseudo-code have been implemented in Phoenix for the various engines. The return codes in Phoenix for all engines, except QRPEM, are usually based directly on the ITRMCD return codes of 1 through 5 in UNCMIN. Additional codes of 0, 6, 7, and negative return codes that do not appear in the original UNCMIN have been added to deal with special situations and, in some cases, the meaning of a return code of 4 has been changed. The actual methods of determining the return codes in UNCMIN are technical. What follows is a summary, refer to the reference mentioned above for more detail.
UNCMIN return codes of 1, 2, and 3 indicate that the optimizer was ‘probably successful’ in reaching at least a local optimum of the log likelihood function, with 1 giving the strongest evidence of success and 3 the weakest. More specifically:
1 = The ‘scaled gradient’ was reduced below a threshold
2 = Further progress does not seem possible due to relative changes in the parameters becoming too small on successive iterations
3 = The line search step in the quasi-Newton direction failed to locate a sufficiently better objective value than the current value
4 = The user-defined maximum number of iterations was reached before any other termination criterion was achieved (note that the meaning of this code has been changed as described below for the FO, Naive pooled, FOCE L-B, and IT2S-EM engines)
5 = The UNCMIN algorithm failed to compute an acceptable step-size within the allowable internal limit on number of trials.
The FO, Naive pooled, FOCE L-B, and IT2S-EM methods in Phoenix require a sequence of UNCMIN optimizations of a log-likelihood or approximate log-likelihood function. Typically, successful convergence is recognized if the final two members of this sequence give essentially identical results with a UNCMIN return code of 1, 2, or 3. Thus, at least two ‘successful’ UNCMIN optimizations are required for convergence. If convergence is achieved, the return code of 1, 2, or 3 for these engines is the return code from UNCMIN for the final run in the sequence. If these engines do not converge within the user-specified number of UNCMIN attempts (for these engines, the iteration counter is the number of runs of UNCMIN), a return code of 4 is made. If the sequence of UNCMIN optimization is determined to be non-convergent before hitting the maximum number of iterations, a negative return code is made, where the absolute value of the return code is the UNCMIN return code from the final UNCMIN run in the sequence.
Unlike the other UNCMIN-based engines, the FOCE ELS method can sometimes be determined to have converged based on a single UNCMIN run, if the UNCMIN return code is 1 or 2. For a return code of 3, the FOCE ELS algorithm attempts a restart from the final solution so obtained. Here, the iteration counter counts the total number of line searches made throughout all restarts, and thus a return code of 4 means that the total number of such line searches over all attempts has hit the user-specified maximum. If the FOCE ELS algorithm cannot improve upon its original attempt that resulted in a return code of 3, then the results of that attempt are reported with a return code of 3. If, however, better (higher overall log-likelihood) results are obtained, the current attempt then becomes the reference and a new UNCMIN run attempt is made to improve the log-likelihood. The final return code reflects the return code of the best attempt.
To these basic UNCMIN return codes, several others have been added, including 0, 6, 7, and various negative return codes indicating non-convergence. The following summarizes the meaning of the non-negative integer return codes within the context of each method. Negative return codes apply in situations where convergence is not achieved and are also described below.
0: The engine was run in evaluation model only, no optimization was requested, and the solution that is returned was the same one entered as a starting solution. (Not used in the original UNCMIN algorithm.)
1: Convergence was achieved and the final UNCMIN optimization successfully terminated with a scaled gradient below an internal tolerance (this is the ‘best’ possible return code). For QRPEM, a return code of 1 indicates successful convergence.
2: Convergence was achieved and optimization of the final UNCMIN run terminated when parameter change fell below internal tolerance on successive iterations of the final UNCMIN run. QRPEM does not use a return code of 2.
3: Convergence was achieved and optimization of the final UNCMIN run terminated when insufficient objective function improvement was made during the line search step along the quasi-Newton direction. QRPEM does not use a return code of 3.
4: For FO, FOCE L-B, Naive pooled, and IT2S-EM, the maximum number of UNCMIN runs was reached before any other termination criterion was achieved. For FOCE ELS, the maximum total number of linear searches over all UNCMIN runs was reached before convergence was achieved. For QRPEM, the maximum number of QRPEM iterations was reached before convergence was achieved, where a QRPEM iteration is one complete cycle through sampling all subjects and making a parameter update based on the samples.
5: Optimization was terminated due to internal failure of UNCMIN to find a suitable step size (this is an error termination). QRPEM does not use a return code of 5.
6: The engine was manually terminated from the user interface Stop Early button in the Progress window before convergence. (Not used in the original UNCMIN algorithm.)
7: Line search failed using central difference gradient or central difference gradient failed. After several attempts, the function value remained out of range. The last function value within range is reported.
Negative return codes indicate convergence failure and often a more definitive reason for the convergence failure can be found in the Core Status file in the Text Output section of the Results tab or in the file nlme7engine.log (if command line invocation is used). For UNCMIN-based engines, the absolute value of the return code represents the UNCMIN return code on the UNCMIN run that resulted in a determination of convergence failure. QRPEM uses a return code of -4 if non-convergence is detected.
The return code 40 for FOCE ELS is uncommon. A code of 40 indicates that the engine went through numerous cycles and was unable to find a solution better than the previous solution by a certain tolerance (.01).
For the Naive Pool engine only, Variance Inflation Factors (VIF) are computed per time entry during simulation runs and presented in several worksheets that tabulate parameter estimates. These are calculated in the same manner as Least-Squares Regression models but are computed regardless of whether the model is simulated or if a fit is run.
VIFs are based on a simple linearization of the model. If the Naive pool engine is run with Number of Iterations > 0, then the linearization is done at the final fitted parameter values. If it is just a simulation, where the model evaluation with Number of iterations=0, then the linearization is done at the initial input values of the parameters. In this case, the observed data values never enter the VIF computation.
For simulations, VIFs can be used to compare experimental designs. For fits, if the square roots of the VIFs are multiplied by the epsilon estimate it gives standard error estimates based on the linearized model. Standard error of estimates are normally computed from the Hessian of the objective function. If the Hessian is not successful, the Maximum Likelihood Models object will then automatically report the standard errors based on successful VIF computation instead. If this occurs, it is indicated in the Core Status text output.
Legal Notice | Contact Certara
© Certara USA, Inc. All rights reserved.