integer division

[krismc]

I can't figure out why integer division is giving me a different answer to regular division?

>>> 100/.8
125.0
>>> 100//.8
124.0

[scruss]

MicroPython, like CPython, has an integer division operator that performs floor division.

The constant .8 is represented as a binary floating point number very slightly larger than 0.8 (0.80000001192, approximately). We can demonstrate this in Micropython with the modulo operator:

>>> 100%.8
0.7999985

If the constant had been stored as the exact value, the result would be zero. With floor division, the quotient is 124, plus a remainder of 0.7999985

[GitHubsSilverBullet]

While there is an exact representation of 0.8 in IEEE 754 single precision (0x3f4ccccd, or 1 × 2^-1 × 1.6 according to Float Toy),

Hmmm, not sure whether that's correct.
For 0.8, the binary pattern of the mantissa looks like 110011001100110011001100… and it continues forever.

#!/usr/bin/python3
# -*- coding: UTF-8 -*-
# vim: fileencoding=utf-8: ts=4: sw=4: expandtab:

from decimal import *

getcontext().prec = 100
test = Decimal("0.8")
D1 = Decimal(1)


y = Decimal(2)
result = Decimal(0)
for i in range(1,24):
    divisor = D1 / y
    if result + D1/divisor <= test:
        print('1', end='')
        result += D1 / divisor
    else:
        print('0', end='')
    y /= 2
print()
print('float result: ', result)

y = Decimal(2)
result = Decimal(0)
for i in range(1,54):
    divisor = D1 / y
    if result + D1/divisor <= test:
        print('1', end='')
        result += D1 / divisor
    else:
        print('0', end='')
    y /= 2
print()
print('double result:', result)

[scruss]

From a Raspberry Pi Pico running v1.25.0:

>>> import array,struct
>>> hex(struct.unpack('<I',bytes(array.array('f', [0.8]))))
'0x3f4ccccd'

I guess float toy is wrong, then. But still, the result from divmod() shows the problem.

Using a less-broken version of Float Toy, 0x3f4ccccd is more correctly parsed as 1 × 2^-1 × 1.60000002384185791015625 = 0.800000011920928955078125

[GitHubsSilverBullet]

'0x3f4ccccd'
1 – 11111010 – 0 1100 1100 1100 1100 1101
Sign – Exponent – Mantissa
And right at the far right end rears a rounding problem its ugly head.

[scruss]

wouldn't the sign be 0?

[GitHubsSilverBullet]

wouldn't the sign be 0?

Good catch. You're correct. Copy'n'Paste error on my side, suppressing the two leading zeros.
Should have been:
0 - 01111110 – 100 1100 1100 1100 1100 1101
But the conclusion remains the same.