A common "thick numbers" package?

@giordano

I've not been Watching this package but it looks like interesting and important changes are afoot, and I wonder if it's an opportunity for some shared development among packages like Measurements and possibly even dual-number packages like ForwardDiff (and probably more, feel free to CC others). All of these packages share a somewhat-awkward desire to do math on objects that act like numbers but also have some "thickness" to them. (In ForwardDiff the thickness is infinitesimal, so maybe it's not quite as relevant to this discussion.) All of these packages have been tempted to inherit from Real, but many have recognized that this has some issues. I wonder if it's time to create a core package---let me throw out ThickNumbers.jl as a possible name---that defines abstract types and exports common operators, traits, fallbacks, errors, etc., but leaves all concrete implementations to downstream packages.

To make this a bit concrete, here's an example from the development (master) branch of IntervalArithmetic:

julia> using IntervalArithmetic, IntervalArithmetic.Symbols

julia> 1..3 == 1..3
ERROR: ArgumentError: == is purposely not supported, use ≛ instead
Stacktrace:
 [1] ==(::Interval{Float64}, ::Interval{Float64})
   @ IntervalArithmetic ~/.julia/dev/IntervalArithmetic/src/intervals/interval_operations/boolean.jl:31
 [2] top-level scope
   @ REPL[2]:1

julia> 1..3 ≛ 1..3
true

julia> 1..3 ≛ 1..3.1
false

Note the use of a new operator ≛. My understanding is that IntervalArithmetic is trying to adhere more strictly to the fundamental principle that f(X, Y) (here f is ==) includes all possible values of f(x, y) for any x ∈ X, y ∈ Y; since almost any independent choice of x, y are not equal, testing interval equality is nearly useless. One could make a case for Interval{Bool}(false, true), but having the operation be undefined is at least safe, and gets you to the same result (an error) that putting a == b inside an if statement would generate even if it did return Interval{Bool}(false, true) (because if requires a Bool). This is just one example among many.

I tend to think of Measurement as kind of like an interval, but less strict and less affected by the dependency problem. I actually have a practical use case where I'd like to be able to support users choosing whether they want to perform a computation using IntervalArithmetic (guaranteed but probably overly-conservative bounds) or Measurements (approximate but probably tighter bounds) and hence it would be interesting to consider making the two packages more compatible.

While I haven't actually written a line of code, I wonder if a shared "abstract package" might standardize some of the difficult decisions we have in common, or at least provide a social gathering point for discussing such issues. Julia's own base/numbers.jl and base/operators.jl might serve as inspiration for what could go in there, as well as noticing and sharing relevant patterns that are currently in the packages that might eventually use ThickNumbers.jl.

@Kolaru

Thanks for the opening this issue!
It would be nice indeed to have a common interface. But I am not sure on the design either; to do this we would need to write down what are the expected features of "thick numbers" and hopefully see if we can reach a consensus.

To add context: interval arithmetic has stringent specifications which play poorly with generic code intended for real numbers (in the mathematical sense). To illustrate, let me copy-paste @Kolaru's recent example

f(x, y) =  (x == y) ? 1 : exp(x - y)

which fails for the reasons mentioned already in @timholy's comment.

So to prevent silent failures, we have to fight quite a bit against Julia's flexibility (that's the reason for the recent changes on master) and this is why having Interval <: Real is not great. On master, we still have Interval <: Real but we removed all comparisons ==, <, etc. and conversion/promotion with Number; in PR #572 we are also discussing if intersect, union, etc. should follow the Base definition or just error to disambiguates with their interval arithmetic meaning.

It may also be worth mentioning here that recently we opened #584 as a way to recover interoperability with the ecosystem by exploiting the decoration system. This came up when thinking about conversion, see recently #580. One scenario could be to define

struct Interval{T} <: Real
    interval   :: BareInterval{T}
    decoration :: Decoration 
end

struct BareInterval{T}
    lo :: T
    hi :: T
end

Then, Interval can be made more permissive since a decoration could flag "unsafe operations" (converting a Number to an Interval for instance). On the other hand, BareInterval will be very conservative.

@OlivierHnt

To compare with other similar packages having the same problem (Unitful.jl, Measurements.jl and ForwardDiff.jl for what I know), I think it may be useful to explicit what we want or not from the Real supertype. That way we may get a clearer view of what we have in common.

What we want:

Arithmetic.
Mathematical functions.
Interoperability with the ecosystem ^^'.

What we do not want:

Automatic promotion to or from float or other Real types. Correctness can simply not be guaranteed in general.
Number as 1 element 0-size arrays (broadcastable, valid for things like iteration, and set operation like intersect). We don't want that because it is ambiguous since intervals are sets by their own rights, which conflict with Base definition of e.g. intersect for a Real. Being broadcastable alone would be fine, maybe.

What we can kind of live with:

Ordering (< and such) and comparison (==). We can in principle use ternary logic for that, that would be guaranteed, but it is still slightly ambiguous (especially for ==).
Mixing different Real types. Float64 * Interval has unclear interpretation, because we don't know if the Float64 is an exact trustable number. As @OlivierHnt mentioned we could tackle that by extending the concept of DecoratedInterval, and tag computation that may have lost validity that way.

I am pretty sure that I am missing some elements, but that's what come to mind right now.

Does it leaves enough in common with other packages ? I am not sure.

Update (sorry for the delay, I've been immersed in other things): I've realized that Measurements.jl doesn't implement the arithmetic rules I want:

julia> using Measurements

julia> (0 ± 1) * (0 ± 1)
0.0 ± 0.0

Instead I want something consistent with the algebra of random variables (what I think of as "soft intervals"). AFAICT there isn't really a package doing what I want, so I may start one (locally for now), and as I develop it I'll look for common interests. I may have a more informed view on this in a couple of weeks.

@baggepinnen

CC @baggepinnen for MonteCarloMeasurements.jl .

I don't think it's particularly easy to define a common generic interface. For example I feel quite strongly about making Measurements.Measurement <: AbstractFloat, which I don't think would make sense for IntervalArithmetic.

@timholy Measurements.jl implements linear error propagation, which is a simple and commonly used method in Physics, but it relies on the first derivative of the function being the "dominant" one, hypothesis which can locally fail for simple functions like sin/cos (think of their stationary points). One could extend the propagation to second-order derivatives to work around this limitation, another approach is Monte Carlo sampling, which should be equivalent to linear error propagation when its condition is satisfied (albeit much slower), and provide more "meaningful" results otherwise.

There's an initial implementation at https://github.com/timholy/ThickNumbers.jl. Durng this preliminary phase, I'll host it in my personal GitHub account, but if we decide to adopt it I'll transfer it to JuliaMath.

It's preliminary in the sense that I've not actually tried it in practical usage, and I don't plan to register it until I've created or rebased at least two packages on it, so we know if the generalizations "work" in real-world practice.

However, it's got a lot going for it already:

extensive documentation (still waiting for it to be deployed at the time of writing, so I haven't checked availability)
default implementations for a subset of IEEE Std 1788-2015 (more presumably to come)
a subdir package, ThickNumbersInterfaceTests, which can be used to determine whether a concrete subtype satisfies the interface.
support for ForwardDiff across all ThickNumber subtypes, even though they are <: Number rather than <: Real.

Note that it mirrors the "new" direction for IntervalArithmetic, but with a couple changes in terms of unicode operators. I decided to aim for "dot" consistency, riffing off of ⪽. So most of the operators I chose have a "dot" in them. I'm happy to discuss and modify these choices.

There were some initial issues with doc depoloyment, but those have been fixed and additional improvements made.

Toy package serving as a proof-of-principle: https://github.com/HolyLab/GaussianRandomVariables.jl

@AnderGray

CC To @AnderGray, who has experience and may add something to the discussion.

I think a package implementing a base uncertain number type, with common interface, would be very useful. I would have used it when making ProbabilityBoundsAnalysis.jl and MomentArithmetic.jl.

Note: if you're looking for an easy-ish way to reduce puffiness in interval arithmetic, centred from interval arithmetic is still rigorous but uses the first derivative (we can use ForwardDiff.jl) to reduce puffiness. I think when you use higher orders, it generalises to Taylor Models.

Unfortunately gaussian random variables aren't closed under arithmetic operations (except linear transformations), so the distribution's shape will change for anything non-linear. Essentially why first-order error propagation is only accurate for linear-enough functions.

You can however perform computations with moments, or at least bound them in non-linear transformations. So unless you additionally track the shapes, the transformed moments become intervals. Afterwards however, probabilities can be bounded from the moment, using e.g. the Chebyshev inequality.

If useful, we did publish open access article on the topic, and a sort of repository/package. Inside the paper you can find some tables containing the arithmetic rules we worked out (similar to the algebra of random variables)

Reducing puffiness is all about correlations:

julia> using MomentArithmetic, IntervalArithmetic

julia> a = Moments(0, 1, -1 .. 1)  # mean 0, var 1, range [-1, 1]
julia> b = Moments(0, 1, -1 .. 1)

julia> a - b   # Default is unknown correlation
moment: 	  ~ ( mean  = 0, var = [0.0,4.0] , range = [-2.0,2.0] )

julia> subIndep(a, b)  # Independence gives precise variance
moment: 	  ~ ( mean  = 0, var = 2 , range = [-2.0,2.0] )

At the end of the paper there is some discussion on using covariances to reduce puffiness in arithmetic like in the above.

Thanks, this looks useful. In case you're not aware, I tried building MomentArithmetic on 1.10 and got this:

julia> using MomentArithmetic
Precompiling MomentArithmetic
        Info Given MomentArithmetic was explicitly requested, output will be shown live
WARNING: could not import ProbabilityBoundsAnalysis.plot into MomentArithmetic
WARNING: Method definition (::Type{MomentArithmetic.Moments})() in module MomentArithmetic at /home/tim/.julia/packages/MomentArithmetic/x2Mva/src/Moments.jl:43 overwritten at /home/tim/.julia/packages/MomentArithmetic/x2Mva/src/Moments.jl:52.
ERROR: LoadError: Method overwriting is not permitted during Module precompile.
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/MomentArithmetic/x2Mva/src/Moments.jl:52
...

1.10 is pickier about turning things that may have been warnings on previous releases into errors, because it was found that in some cases unfixed warnings were the source of serious, very difficult-to-track-down bugs. (In other cases they are fairly harmless, and this may be such a case.)

I also like this line:

There are no infinitely massive body weights (despite recent trends in western dietary health).

😆

I've gotten back to this issue and made enough progress with our GaussianRandomVariables package to think that it's nearly certain to be useful for some applications. Because we're also working on optimization routines that can either use intervals (for rigorous bounds) or gaussians (for "statistical" bounds), having both types of numbers based on a common ThickNumbers API is basically a requirement to make that optimization code sane. Consequently I am likely to register both an intervals package and the gaussian numbers package. I have more than a little reluctance about doing this (I'd rather not fragment the ecosystem with two interval-arithmetic packages), but I don't currently see a great alternative. Our interval-arithmetic package's README will strongly direct users to IntervalArithmetic.jl unless they really need the ThickNumbers API, and I expect ours will likely remain a pale shadow of this one (but good enough to tackle some interesting problems).

Any real reservations about this? I expect to move on this fairly soon.

@tmcgrath325

CC @tmcgrath325

I have not looked at this in a while, but what is the main obstruction making ThickNumber and IntervalArithmetic incompatible? The subtyping ThickNumber <: Number?
If it's only a matter of API, can a package extension help?

It's that the interval type has to be a subtype of ThickNumber: https://github.com/HolyLab/GaussianRandomVariables.jl/blob/309f20e65d06ba7c97019ed301f7b052b175b779/src/GaussianRandomVariables.jl#L37

And then this implies that the type must satisfy the ThickNumbersInterfaceTests: https://github.com/HolyLab/ThickNumbers.jl/blob/main/ThickNumbersInterfaceTests/src/ThickNumbersInterfaceTests.jl

The interface is documented here: https://holylab.github.io/ThickNumbers.jl/dev/#The-Fundamental-Principle-of-ThickNumbers-(FPTN) and the section below that.

we have to provide an unsafe, and non-standard compliant flavor of interval, for which many checks are disabled.

Is that on the table? If so, I'd be happy to work on implementing it: I don't want to fragment the ecosystem. But I'd only do it if there were a reasonable chance of it being merged, and I wouldn't want to push you folks into something you're reluctant to take on. A lot of people turn to intervals because they want that certificate, and this would explicitly break that.

I'm probably a much less common potential user: since we're trying to solve practical engineering problems rather than prove theorems, we don't really need the certificate. What attracts me about interval arithmetic is that it's so well-grounded from a conceptual standpoint, and not many alternatives are. The giant challenge to practical usage is the dependency problem, and so from this standpoint I have no real concern about carving out a floating-point exception as being "in the weeds." Indeed, at the last stage of our computation, when I'm assembling results from many contributions, to narrow the dependency problem I'm quite likely to use statistical accumulation of errors rather than summing errors. If I'm willing to sacrifice that kind of guarantee, then floating-point roundoff is the least of my concerns. So I really like the idea of a mode that's safe and conceptually sound in principle, but doesn't go to heroics to handle the oddities of floating point because you know that's not your biggest problem.

So I really like the idea of a mode that's safe and conceptually sound in principle, but doesn't go to heroics to handle the oddities of floating point because you know that's not your biggest problem.

So it sounds like you'd like to comply with the IEEE 1788 interval arithmetic standard with no rounding (which we control via IntervalArithmetic.configure(; rounding=:none)).

The standard imposes a bunch of behavior (what to do with empty interval, nai, decorations, normalizing zero for the output of inf(...) etc.) and all of that will always involve extra checks, completely independent from the rounding strategy.

So to achieve compliance with interval arithmetic, you'll have to be willing to sacrifice performance compared to a 0-check interval implementation. How much performance is acceptable is the next question 😅
Note that the code of IA.jl has never been optimized, and I would not be surprised if we are being too naive. For instance, we perhaps use inf / sup just because it reads well but these functions do some extra normalization which might be completely skipped once we are in a branch with no empty intervals.

So before talking about implementing a different interval flavor, seeing if we can shrink the gap for hull on BareInterval will be helpful here because it is not related to rounding, just formatting the output.
These are the offensive lines for a single call to hull with two args

IntervalArithmetic.jl/src/intervals/interval_operations/set_operations.jl

Lines 71 to 74 in 7f3e73b

    
           function hull(x::BareInterval{T}, y::BareInterval{T}) where {T<:NumTypes} 
        
               isempty_interval(x) & isempty_interval(y) && return x 
        
               return _unsafe_bareinterval(T, min(inf(x), inf(y)), max(sup(x), sup(y))) 
        
           end

Can the check for empty intervals be done in a cheaper way? Do we need to use inf/sup which involve some checks with NaN etc.?
For hull(x, y, z...), could all the necessary checks be done at the beginning on each argument, rather than reproduced at every call of hull on each pair?

So it sounds like you'd like to comply with the IEEE 1788 interval arithmetic standard with no rounding (which we control via IntervalArithmetic.configure(; rounding=:none)).

I may have mis-spoken. What I want is the performance of the current ThickNumbers-based implementation. You can see there that hull does only the minimum:

hull(a::TN, b::TN) where TN<:ThickNumber =
    lohi(TN, min(loval(a), loval(b)), max(hival(a), hival(b)))

Fair enough. Then the statements

[...] well-grounded from a conceptual standpoint [...]
[...] a mode that's safe and conceptually sound in principle [...]

must now all be handled on your side, since you can't rely on the interval arithmetic standard. I imagine you already have this under control to justify that you avoid any branching check for your interval computations.

Maybe the only assumption you are making is that there is no concept of empty interval (so you don't deal with intersection for instance)?

Side note: if we internally store the low and high bounds of an empty interval as (Inf, -Inf) the hull function should be fixed as the code would be the same as yours I believe.
Currently we store its bounds as (NaN, NaN) (but inf/ sup returns Inf / -Inf as asked by the standard). We probably just decided to do this because NaN felt safer.
If that allows us to remove a bunch of checks and gain some speed we might want to consider doing this.

Maybe the only assumption you are making is that there is no concept of empty interval (so you don't deal with intersection for instance)?

No, I define that: https://github.com/HolyLab/ThickNumbers.jl/blob/778b20c0f3d603f88c36d91e2471171f8070db33/src/ThickNumbers.jl#L206-L227

julia> using ThickNumbers, IntervalFastMath

julia> iv = emptyset(Interval{Float64})
Inf .. -Inf

And yes, I'm comfortable with defining what I mean with 'conceptually well-grounded'. In essence, I like the analysis forced by interval arithmetic, but I am willing to place bets on the fact that it's going to be conservative. So for this package, the needed change would largely be to take floating-point arithmetic at face value and not "waste" a bunch of time worrying about ulps. As always, only for those who opt-in to this behavior.

Overflow is a tricky case, as you do have to handle it sometimes in non-interval code.

Oh that's good, since that matches our behavior, except that you are consistent about what is stored internally.
One quick test could be to change

IntervalArithmetic.jl/src/intervals/interval_operations/constants.jl

Line 12 in 7f3e73b

    
           emptyinterval(::Type{BareInterval{T}}) where {T<:AbstractFloat} = _unsafe_bareinterval(T, convert(T, NaN), convert(T, NaN))

into

# bypass default
IntervalArithmetic.emptyinterval(::Type{BareInterval{T}}) where {T<:AbstractFloat} =
    _unsafe_bareinterval(T, typemax(T), typemin(T))

Then just test

function my_hull(x::BareInterval{T}, y::BareInterval{T}) where {T}
    xlo, xhi = x.lo, x.hi
    ylo, yhi = y.lo, y.hi
    return IntervalArithmetic._unsafe_bareinterval(T, min(xlo, ylo), max(xhi, yhi)) 
end

You'll still hit one normalization path here

IntervalArithmetic.jl/src/intervals/construction.jl

Lines 64 to 65 in 7f3e73b

    
           _unsafe_bareinterval(::Type{T}, a::T, b::T) where {T<:AbstractFloat} = 
        
               new{T}(_normalisezero(a), _normalisezero(b))

But let's see first if that matters. It is probably also possible to avoid that and make _unsafe_bareinterval be essentially a no-op.

Btw, thanks for sticking with this discussion. I hope it's not too much of a drag on your side. Regardless whether we can get things sufficiently on track for you to use IntervalArithmetic.jl, it's making us question a few things about our performance and hopefully it'll lead to some improvements 🙂

PS: fyi we are conservative with

mixing standard floats and intervals (eg 1.0 + bareinterval(1) is an error)
comparison / boolean functions such as ==, in, etc. (as you already noticed)

EDIT: on my laptop I get

julia> x = bareinterval(2); y = bareinterval(4);

julia> @btime hull(Ref($x)[], Ref($y)[]);
  3.291 ns (0 allocations: 0 bytes)

julia> @btime my_hull(Ref($x)[], Ref($y)[]);
  2.625 ns (0 allocations: 0 bytes)

Btw, thanks for sticking with this discussion. I hope it's not too much of a drag on your side. Regardless whether we can get things sufficiently on track for you to use IntervalArithmetic.jl, it's making us question a few things about our performance and hopefully it'll lead to some improvements 🙂

I'd say the same thing in reverse--I'm questioning some decisions I made too, and regardless of outcome here I will make improvements that also affect our GaussianRandomVariables package.

I'd be happy to give a first crack at a "fast but less safe" mode. I might split it into two pieces: places where the existing performance could be improved without the mode change, and places where the new mode is needed. Do you prefer, say, two PRs or many smaller ones focused on individual issues?

Two PRs are fine. By mode you mean the flavor I suppose. I never looked too much into this, but it is a concept defined in the IEEE standard.

First PR (#771) of a total of 3 anticipated so far. More are likely to come after we see where things stand.

perf: Speed up `hull` and `intersect_interval`

perf: Store empty intervals as `(typemax, typemin)`

perf: Speed up `hull` and `intersect_interval`

Despite the progress on hull, I'm struggling to get the performance I currently have with IFM. As one example of the obstacles I'm facing: a key element of an upcoming manuscript I'm working on needs to compute the third derivative wrt t of ϕ(t) = f(x + t*w) where x and t are (thin) Reals (or, in the case of x, a vector thereof) and w an Interval or BareInterval (or vector thereof). It seems that only BareInterval could give me the performance I want/need, but it deliberately doesn't support combining with Reals and decorations add a ton of overhead.

So this leads me to ask: how receptive (or not) would you be to UnsafeInterval that doesn't worry about provenance at all and takes floating-point math at face value?

We made exact (which should have been called literal I suppose...) for this reason.

julia> @exact f(x) = sin(2x + 1)
f (generic function with 1 method)

julia> x = exact(1); w = exact(2);

julia> f(x + bareinterval(0, 1) * w)
[-1.0, 0.656987]

Note: exact(0.1) is not giving $1/10$; it treats the input float as exact.

@lbenet

In the last JuliaInterval call, we had quite some discussion about having a useful common interval type that can be used as a backend by other packages.

A key problem is that in general, intervals can have a lot of different traits:

Being a Number
Correct rounding
Carrying flags (typically decorations or guaranteed flag)
Having open/closed boundaries
Being compatible with set operations from Base
Different ways to deal with corner cases (like division by interval(0))

Now, except for subtyping Number, all the other traits are properties that could be stored as type parameters, and thus compiled out at no cost when disabled. And so a "bare" interval could be retrieved by disabling everything, and many flavors could be built by choosing different combinations.

Having a trait system here could be nice, as traits could be added by users, the default being the barest possible interval.

However, what we still have to think about it is whether this would be over-engineering the problem: as much as it would be nice to implements operations in common between IntervalArithmetic.jl, ThickNumbers.jl and even Intervals.jl, it is unclear (at least to me) whether it is worth it.

@lbenet @OlivierHnt @dpsanders

Navigation Menu

Repository navigation

A common "thick numbers" package? #585

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	function hull(x::BareInterval{T}, y::BareInterval{T}) where {T<:NumTypes}
	isempty_interval(x) & isempty_interval(y) && return x
	return _unsafe_bareinterval(T, min(inf(x), inf(y)), max(sup(x), sup(y)))
	end

	_unsafe_bareinterval(::Type{T}, a::T, b::T) where {T<:AbstractFloat} =
	new{T}(_normalisezero(a), _normalisezero(b))

A common "thick numbers" package? #585

Description

Activity

OlivierHnt commented on Sep 28, 2023

Kolaru commented on Sep 28, 2023

timholy commented on Oct 4, 2023

giordano commented on Oct 7, 2023

timholy commented on Oct 8, 2023

timholy commented on Oct 9, 2023

timholy commented on Oct 11, 2023

lbenet commented on Oct 11, 2023

AnderGray commented on Oct 11, 2023

timholy commented on Oct 12, 2023

timholy commented on Oct 12, 2023

timholy commented on Jul 15, 2026

timholy commented on Jul 15, 2026

OlivierHnt commented on Jul 15, 2026

timholy commented on Jul 16, 2026

14 remaining items

timholy commented on Jul 27, 2026

OlivierHnt commented on Jul 27, 2026

timholy commented on Jul 27, 2026

OlivierHnt commented on Jul 28, 2026

timholy commented on Jul 28, 2026

OlivierHnt commented on Jul 28, 2026

timholy commented on Jul 28, 2026

OlivierHnt commented on Jul 30, 2026

timholy commented on Jul 30, 2026

timholy commented on Aug 5, 2026

OlivierHnt commented on Aug 6, 2026

Kolaru commented on Sep 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions