numeric calculation problem?

[shriram]

The program

fun square(n): n * n end

fun f(x, y):
  (333.75 * num-expt(y, 6))
  +
  (square(x)
    *
    ((11 * square(x) * square(y))
      -
      num-expt(y, 6)
      -
      (121 * num-expt(y, 4))
      -
      2))
  +
  (5.5 * num-expt(y, 8))
  +
  (x / (2 * y))
end

v = f(77617, 33096)

binds v to -2.3611832414348226e+21.

I may have transcribed this incorrectly.

[blerner]

33096 ^ 6 itself will drastically overflow (according to Wolfram Alpha,
it's roughly 1.3 * 10^27, which makes sense, since 3.3 * 10^4 should be
a bit more than 1300 * 10^24), so I suspect the computation fails
there. Dunno how good our BigNum implementation actually is; we just
took that library as given.

[jpolitz]

My guess is that, since we get an exact answer for parts of the computation like 33096 ^ 6, and then combine them with floating-point calculations after (which are assumed inexact by the numbers library), we get a different answer than either the fully-exact or fully-floating point calculations would give.

[schanzer]

The BigNum implementation has a ton of missing functionality. The basics are there, but quite a few APIs just errored out as undefined. I'm not sure how old your branch of the library is, but this past spring I inserted an exactness-losing hack that coerces BigNums into floats when an undefined function is called (see bootstrapworld/wescheme-compiler2012@91f4e48). This could be causing the calculation errors you're seeing, when the prior implementation would have just thrown an error.

[ds26gte]

BTW, this produces the correct result (i.e. -54767/66192) with the roughnum code in place. All non-roughnum calculations should be precise henceforth. The Bignum (aka BigInteger) code in the library seems fine.

[blerner]

@ds26gte this one also should track roughnum's merge