Skip to content

Add test cases for antispam rules, fix bug in attachments rule & test - #733

Merged
SebastiaanZ merged 15 commits into
python-discord:masterfrom
kwzrd:unittest-antispam-rules
Feb 21, 2020
Merged

SebastiaanZ merged 15 commits into
python-discord:masterfrom
kwzrd:unittest-antispam-rules

Conversation

@kwzrd

@kwzrd kwzrd commented Jan 26, 2020 •

Copy link
Copy Markdown
Contributor

This PR introduces a series of unit test cases for the bot's antispam rules. This addresses the following issues.

Closes #595 via 1f17d0e.
Closes #596 via b0713be.
Closes #597 via f15f318.
Closes #598 via 4bfe30d.
Closes #603 via f32ffaf.

I also sneak in a quick bugfix in 4199b12: it appears that the already existing rule for attachments contains a slight bug where in the printout, the configured 'max' key is looked up instead of 'interval'. The same commit also fixes the key in the rule's unit test, which was incorrectly expecting the wrong key.

Finally, aedbd6f is a refactor of the name msg into make_msg which prevents a harmless, but unnecessary name collision in the msg for msg in ... generator expressions down the module.

Edit:
Closes #599 via 54e8c98.
Closes #602 via cae7f25.

kwzrd added 7 commits January 26, 2020 22:20
The rule was incorrectly printing out the maximum amount of allowed attachments
instead of the configured interval. This commit also adjusts the rule's unit
test case.
The name msg is less descriptive and creates a needless name conflict in local gen exp.
@kwzrd kwzrd added the a: tests Related to tests (e.g. unit tests) label Jan 26, 2020
@kwzrd

kwzrd commented Jan 28, 2020

Copy link
Copy Markdown
Contributor Author

The committed code is considered ready for review. I'm currently awaiting a response from a user in #599 - if they're no longer interested, I'd like to include that issue in this PR's scope as well.

@MarkKoz MarkKoz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the code for the tests is extremely similar, even the cases (the burst tests are kind of outliers). The main difference is always just in the make_msg function and the output string.

Comment thread tests/bot/rules/test_burst.py Resolved
@kwzrd

kwzrd commented Jan 31, 2020

Copy link
Copy Markdown
Contributor Author

The test cases are similar, yes - the rules are too. What do you propose can be done to make this better? Would it make sense to implement some kind of an "executor" on the side, and then just feed it the test cases from each module?

@MarkKoz

MarkKoz commented Jan 31, 2020

Copy link
Copy Markdown
Contributor

Two helper functions: one for "allows" tests and one for "disallows". They should be passed the apply function and the subtest arguments. The helpers will do the loop, calling of apply, and the assertions.

As I mentioned, some of the subtest arguments follow a similar pattern across test suites. To facilitate this, a function could be defined which will take a make_msg function and return subtest arguments. This function can shared among some test suites to generate subtest arguments while other suites with more divergent subtest can still use their own subtest arguments. My only concern is that such function adds a layer of indirection which has the consequence of making unclear what the actual subtest values are.

I believe such a design will reduce redundancy but still remain flexible should test suites need to diverge in code in the future.

kwzrd added 3 commits February 2, 2020 18:25
This will serve as an ABC for tests for individual rules.
The base class provides runners for allowed and disallowed
cases, and the children classes then only provide the cases
and implementations of helper methods specific to each rule.
This will prevent child classes to be instantiated unless they implement
all abstract methods, leading to a more descriptive error message.
@kwzrd

kwzrd commented Feb 2, 2020

Copy link
Copy Markdown
Contributor Author

93d19f3 adds an ABC (9758e32 actually adds the metaclass=ABCMeta), b89f9c5 then adjusts the existing rules to use the ABC. I decide to make the DisallowedCase (previously Case) more generic by having an n_violations attr instead of a rule-specific one. This way, the subTest kwargs can be generic as well.

Let me know how this looks, and thanks for the feedback.

@jb3
jb3 requested review from a team, kosayoda and tagptroll1 and removed request for a team February 2, 2020 20:32
@kwzrd kwzrd added the s: WIP Work In Progress label Feb 2, 2020
@jb3
jb3 requested a review from a team as a code owner February 2, 2020 22:27

@MarkKoz MarkKoz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great.

Comment thread tests/bot/rules/__init__.py Resolved
@MarkKoz MarkKoz added status: needs review and removed s: WIP Work In Progress labels Feb 3, 2020
@kwzrd

kwzrd commented Feb 4, 2020

Copy link
Copy Markdown
Contributor Author

I've added tests for duplicates (54e8c98) and newlines (cae7f25).

As the latter is more complex than the other rules, yielding two distinct error reports, I decided to implement two subclasses of the RuleTest ABC for it. Each checks one type of violation.

If the given messages violate both total count and max consecutive, the rule will fail on the total as that is checked first. Since I consider this an implementation detail and not "desired functionality", I do not test it, and each disallowed test case only violates the rule one way or the other, so that the tests still pass should the two checks switch order.

This PR now provides 100% coverage for the entire rules package - I've updated the PR description to mention the two extra issues it closes.

@SebastiaanZ
SebastiaanZ merged commit 59618bb into python-discord:master Feb 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

a: tests Related to tests (e.g. unit tests)

Projects

None yet

4 participants