You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Experiment versions cause confusion with allow_multiple_experiments='control' #612
Describe the bug
When we reset an experiment from dashboard, starting numbered versions of experiments in data, it makes confusion in User.max_experiments_reached? and considers the current experiment as already taken.
The symptom is that subsequent calls to ab_test will always return control, once the user starts to be excluded from alternative choice by max_experiments_reached.
To Reproduce
Steps to reproduce the behavior:
I wrote an example app that reproduces the error. It's in rails because it was easier to me.
Go to home page. You'll see the alternatives returned by multiple calls to ab_test. This time it works as usual;
Go to split dashboard and click in reset data, so to start a new version in the experiment
Now refreshes the app until the first alternative is choosed as other.
In this case, the result you see is oher, control, control showing that, after that first call subsequent ones will always return control
Additional context
I've already debugged it. The problem is in User#max_experiments_reached?. The count in L31 doesn't clean up experiment key, that now has :<version> at the end.
Activity
andrehjr commentedon Apr 28, 2020
Thanks for the detailed report and reproduction steps @brunopedroso 🚀
I'll check it out. In the meantime, feel free to submit a PR 😄
Fix #max_experiments_reached? when using allow_multiple_experiments=c…
Fix #max_experiments_reached? when using allow_multiple_experiments=c…
Fix #max_experiments_reached? when using allow_multiple_experiments=c…
🩹Selectively pull from splitrb/split to defer major version update
Change redis.exists to redis.exists? to avoid problems with redis 4.3
v4.0.0