gretl uses double-precision arithmetic throughout — except for the multiple-precision plugin invoked by the menu item "Model/High precision OLS" which represents floating-point values using a number of bits given by the environment variable GRETL_MP_BITS (default value 256). The normal equations of Least Squares are solved via Choleski decomposition, which is accurate enough for most purposes. The program has been tested rather thoroughly on the statistical reference datasets provided by NIST (the U.S. National Institute of Standards and Technology) and a full account of the results may be found on the gretl website (follow the link "Numerical accuracy").
In October 2002 I had a useful exchange with Giovanni Baoicchi and Walter Disasto, who were writing a review of gretl for the Journal of Applied Econonetrics, and James McKinnon, software review editor for the journal. I am grateful to Baoicchi and Disasto for their careful examination of the program, which prompted the following modifications.
The reviewers pointed out that there was a bug in gretl's "p-value finder", whereby the program printed the complement of the correct probability for negative values of z. This was fixed in version 0.998 of the program (released July 9, 2002).
They also noted that the p-value finder produced inaccurate results for extreme values of x (e.g. values of around 8 to 10 in the t distribution with 100 degrees of freedom). This too was fixed in gretl version 0.998, with a switch to more accurate probability distribution code.
The reviewers noted a flaw in the presentation of regression coefficients in gretl, whereby some coefficients could be printed to an unacceptably small number of significant figures. This was fixed in version 0.999 (released August 25, 2002): now all the statistics associated with a regression are printed to 6 significant figures.
It transpired from the reviewer's tests that the numerical accuracy of gretl on MS Windows was less than on Linux, where I had done my testing. For example, on the Longley data — a well-known "ill-conditioned" dataset often used for testing econometrics programs — the Windows version of gretl was getting some coefficients wrong at the 7th digit while the same coefficients were correct on Linux. This anomaly was fixed in gretl version 1.0pre3 (released October 10, 2002).
The current version of the gretl source code package contains a tests subdirectory, with a test suite based on the NIST datasets. This is invoked if you do make check in the top source directory. You are warned if the numerical accuracy falls short of standard. Please consult the file README in the tests directory for details.
The NIST test suite is also distributed with the MS Windows version of gretl. You can run the tests by invoking the program nisttest.exe.
As mentioned above, all regression statistics are printed to 6 significant figures in the current version of gretl (except when the multiple-precision plugin is used, then results are given to 12 figures). If you want to examine a particular value more closely, first save it (see the Section called genr in Chapter 9) then print it using print -t (see Chapter 9). This will show the value to 10 digits.