musl - atanhf(x) Slight Accuracy Improvements

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LRH.2.02.1903132105140.4826@key0.esi.com.au>
Date: Wed, 13 Mar 2019 22:18:41 +1100 (AEDT)
From: Damian McGuckin <damianm@....com.au>
To: musl@...ts.openwall.com
Subject: atanhf(x) Slight Accuracy Improvements

Hi all,

Currently this routine does not achieve accuracy < ULP across all of its 
domain. For half of its domain, it worst error exceeds that limit by 
about 10%.

By tweaking some algebra, this can be made more accurate. A comparison of 
the more accurate version and the original is noted below.

 	ATANF ... FROM ... TO     WORST ERROR  % > EPS/2   MEAN ERROR
 	--------------------------------------------------------------
 	Accurate:0.00000..0.17000 1.00023*EPS   0.76907%   0.01967*EPS
 	Original:0.00000..0.17000 1.09363*EPS   0.71616%   0.01941*EPS
 	Accurate:0.17000..0.55000 0.99151*EPS   5.52206%   0.21620*EPS
 	Original:0.17000..0.55000 1.12779*EPS   8.87471%   0.23922*EPS
 	Accurate:0.55000..1.00000 0.68151*EPS   1.34789%   0.19159*EPS
 	Original:0.55000..1.00000 0.68372*EPS   1.40897%   0.19207*EPS

I see a reduction in the worst error across the entire spectrum and reduce 
slightly the percentage exceeding 0.5*ULP in most cases. The mean error is 
much the same. I have yet to rework the double version.

However, across a subset of its argument range, namely

 	[0 ..(sqrt(2)-1)/(sqrt(2)+1)]

I cannot crack the 1.0*ULP barrier if the computation of the argument
reduction

 	f = 2*(y + (y*y)/(1-y))  <---- LITTLE PROBLEM

is done in single precision. The error in the 23rd bit causes me grief.

Doing that sole calculation in double precision and then storing it as a 
float brings the worst error to 0.99*ULP. I want to avoid any extended
precision.

Note that I used the same accuracy tweak for log1p as done in log2 to 
avoid the cancellation error seen in

 	f - (f*f)/2

Any suggestions are welcome on how to get one extra bit of accuracy in my 
calculation of 'LITTLE PROBLEM' above. Using the approach as seen in 'sq() 
in 'hypotf' makes things worse unless I am doing something really wrong.

Thanks - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.