Provide testing macros/functions like assert_close?

In Python, the NumPy library provides nice array-comparison functions for unit tests etc., which I make heavy use of:

import numpy as np

a = np.arange(5, dtype=np.float32) + 1

np.testing.assert_allclose(a, 1e-10 + np.sqrt(a**2))  # Succeeds.
np.testing.assert_allclose(a, 1e-1 + np.sqrt(a**2))  # Fails.

Would it be useful/appropriate to add some of these to this library? I could be interested in working on it, but thought I'd check first.

(Seems like some such functions are used internally, but don't seem to be exported for external use.)

The reason to not export them publically is simple, then I'd need to ensure they have high quality.

Having these as macros sounds good, especially for the usual assertion as a macro benefits and possibly supporting different number of arguments or other options.

I think it would be important to support relative and absolute error, and weirdly enough that's not even a function anywhere. But it exists at this point in the code (again, of dubious quality)

https://github.com/bluss/rust-ndarray/blob/8218867ab15a9b05a7dc43b84165d24630bdbd16/numeric-tests/tests/accuracy.rs#L241-L250

I'd like to point out, that there is a crate called approx dedicated to that purpose. One could depend on it or use it as a source of inspiration.

@bluss

@bluss Cool. Yes definitely agree tolerances should be configurable.

@aepsil0n Thanks for pointing that out!

So I was thinking a nice option might be to integrate with the Rust port of Hamcrest: add some Hamcrest "matchers" for approximate equality of arrays in, say, a testing submodule of ndarray. Thoughts?

Below are some initial tests I wrote, which hopefully provide a general feel for the potential API. (Haven't done the Hamcrest integration yet.)

#[cfg(test)]
mod tests {
  use lib::ndarray;

  use lib::util::*;

  #[test]
  fn test_absolute_tolerance() {
    let a = ndarray::Array1::<f32>::range(1., 4., 1.);
    assert_that(&a, almost_equals(&a));
    assert_that(&a, almost_equals(&a + 1e-8));
    assert_that(&a, almost_equals(&a + 1e-6).with_atol(1e-5));
    assert_that(&a, not(almost_equals(&a + 1e-6)));
    assert_that(array![1., 2., 3.], almost_equals(a))
  }

  #[test]
  fn test_relative_tolerance() {
    let b = ndarray::Array1::<f32>::range(1., 4., 1.) * 1e6;
    assert_that(&b, almost_equals(&b + 1.).with_rtol(1e-4));
    assert_that(
      &b,
      not(almost_equals(&b + 1.).with_atol(1e-4).with_rtol(0.)),
    );
  }

  #[test]
  fn test_matrix() {
    let c = ndarray::Array2::<f32>::zeros((4, 5)) + 5.;
    assert_that(&c, almost_equals(&c));
    assert_that(&c, almost_equals(array![5.]));
    assert_that(c, almost_equals(ndarray::arr0(4.)).with_atol(1.));
  }
}

No hamcrest integration, we don't bring on major deps for minor features.

Ok. You haven't provided any guidance, though -- suggestions? Optional dep? Make a little framework sans hamcrest? Or not worth bothering with at all? I use the numpy.testing module quite frequently so thought something similar would be nice here.

I suggest for better allclose functionality in ndarray 1 macro for allclose and 1 for assert_allcose that can handle the absolute/relative and combination. It can also be a function for allclose if that's enough for a good API. If you want to develop bigger and more fluent API in the style of hamcrest that sounds awesome, but please then publish it as a separate crate.

Functions for allclose and assert_allclose sound good to me! What about trying to add a fluent testing API behind a feature in Cargo, though? Opt in to begin with? It's only a few hundred lines of code so I'm reluctant to make a whole new crate for this small feature...

@jturner314

Isn't this handled by #581? @jturner314 @joshburkart

Yes that looks like the right way to go!

Fixed by #581.

Navigation Menu

Repository navigation

Provide testing macros/functions like assert_close? #413

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Provide testing macros/functions like assert_close? #413

Description

Activity

bluss commented on Jan 28, 2018

bluss commented on Jan 28, 2018

milibopp commented on Jan 29, 2018

joshburkart commented on Jan 29, 2018

joshburkart commented on Feb 7, 2018

bluss commented on Feb 7, 2018

joshburkart commented on Feb 7, 2018

bluss commented on Feb 9, 2018

joshburkart commented on Feb 11, 2018

LukeMathWalker commented on Mar 17, 2019

joshburkart commented on Mar 17, 2019

jturner314 commented on May 6, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions