One of my favourite maths podcasts, Wrong But Useful, posed this puzzle a while back (based on an original tweet from @troyhunt):
"Your password needs to contain at least four distinct digits, but no digit more than three times. You pick a nine-digit number at random. What’s the probability you picked a valid password?"
Ignoring the fact that this is clearly a ridiculous set of requirements, having solved the puzzle (via a inelegant spreadsheet) a further question occurred to me - given the password restrictions
1) Password is 9 digits long
2) Password contains at least 4 distinct digits
3) Password contains no digit more than three times
how would I go about testing this?
For the purposes of this keeping this post short I'm not going to do any checks on the first requirement and assume we are always entering a nine digit code (so I won't test for longer/shorter strings, alphabetic/special characters, or anything more complicated like script injection). As an opening shot, I want to check that the code isn't completely broken, so I would submit something that fits the requirements and check it is accepted (I'll put the expected result with any explanation in brackets after the test value):
Test 1: 423578196 (accept)
Now, without a developer to talk to, it's impossible to know how they would code the checks, but I'll assume they are carried out in sequence, 2) then 3), so my testing will follow the same sequence - submitting a range of values for each condition in turn to check the correct result is returned.
In order to minimise the amount of testing I'll stick to boundaries; so I'll concentrate on tests for 1, 3, 4, 5 and 9 distinct digits (the smallest and largest possible cases plus the actual requirement and one either side).
Test 2: 333333333 (only 1 distinct digit, reject)
Test 3: 222255599 (only 3 distinct digits, reject)
Test 4: 000446677 (4 distinct digits, accept)
Test 5: 832835825 (4 distinct digits, accept)
Test 6: 779933400 (5 distinct digits, accept)
Test 7: 123456789 (9 distinct digits, accept)
Tests 4 and 5 both test the same condition but fulfil the condition in two different ways - one with the digits grouped and the other with them interspersed. This ensures that no assumption on the order of the digits is assumed in the code. Test 4 also checks that a string with leading zeros is handled correctly - if it is parsed as an integer they may get cut off ("446677") which would change the result. It's possible this is overloading the test with too much responsibility, but we'll let that pass.
So, on to the other requirement: we can now assume all the strings will be 9 digits with at least 4 distinct digits. Again, test at the boundaries, so strings containing digits that are entered at most 1, 2, 3, 4 or 6 times (6 is the most you can achieve if you have at most 4 distinct digits):
Test 8: 012345678 (max repeats 1, accept)
Test 9: 112323445 (max repeats 2, accept)
Test 10: 777949494 (max repeats 3, accept)
Test 11: 333888810 (max repeats 4, reject)
Test 12: 285848258 (max repeats 4, reject)
Test 13: 494424044 (max repeats 6, reject)
Test 14: 363363663 (max repeats 5, reject)
Test 8 is a variation on Test 7, but again with a leading zero to see if that affects the result. If we already knew Test 4 had passed then Test 8 might be redundant, but since we're writing these up front I think it makes sense to keep it in.
Test 11 and Test 12 both test for 4 repeats of a digit: the first checks for a continuous run of four numbers mid-string (with a red herring of three identical digits first), and the second test the scenario when the digits occur separately.
I added Test 14 because I could see it was possible for there to be two lots of 4 repeats in the same string, which might cause an issue in the code.
I think 14 tests is reasonable for the given requirements; I might add some further variety if the opportunity arose (e.g. if cross-browser testing was required or while testing other functionality that requires the password to be entered first).
Of course, there are other ways to come up with tests for this, which may be preferable if you know a little more about the system. For example, I have assumed there is no distinction between rejecting a password based on max repeats or distinct items - if instead there are separate messages for each infringement you could replace tests 2-14 with the following minimal set of tests that cover all of the boundary points for those two requirements:
Test 2a: 012345678 (9 distinct, max repeat 1, accept)
Test 3a: 662298811 (5 distinct, max repeat 2, accept)
Test 4a: 777119922 (4 distinct, max repeat 3, accept)
Test 5a: 333300999 (3 distinct, max repeat 4, reject)
Test 6a: 777777777 (1 distinct, max repeat 9, reject)
Obviously there are a few combinations not covered here, but it should be sufficient to establish the basic principles are being applied correctly. If you wanted to be more rigorous, you could look at all the possible combinations of numbers of repeats and number of distinct digits (it turns out 25 of the 81 pairs can are valid) - you'd need to add some of the variety I've included above to those which would push the number up even further. Running all those would be a bit excessive for a manual test, and if we assume the developer has taken a fairly sensible approach (!) a lot of those tests would just be exercising the same parts of the code and would not provide any new information.
Of course, you could just run all 1000000000 possible digit combinations, but it might take a while to write the expected outcomes matrix...
"Your password needs to contain at least four distinct digits, but no digit more than three times. You pick a nine-digit number at random. What’s the probability you picked a valid password?"
Ignoring the fact that this is clearly a ridiculous set of requirements, having solved the puzzle (via a inelegant spreadsheet) a further question occurred to me - given the password restrictions
1) Password is 9 digits long
2) Password contains at least 4 distinct digits
3) Password contains no digit more than three times
how would I go about testing this?
For the purposes of this keeping this post short I'm not going to do any checks on the first requirement and assume we are always entering a nine digit code (so I won't test for longer/shorter strings, alphabetic/special characters, or anything more complicated like script injection). As an opening shot, I want to check that the code isn't completely broken, so I would submit something that fits the requirements and check it is accepted (I'll put the expected result with any explanation in brackets after the test value):
Test 1: 423578196 (accept)
Now, without a developer to talk to, it's impossible to know how they would code the checks, but I'll assume they are carried out in sequence, 2) then 3), so my testing will follow the same sequence - submitting a range of values for each condition in turn to check the correct result is returned.
In order to minimise the amount of testing I'll stick to boundaries; so I'll concentrate on tests for 1, 3, 4, 5 and 9 distinct digits (the smallest and largest possible cases plus the actual requirement and one either side).
Test 2: 333333333 (only 1 distinct digit, reject)
Test 3: 222255599 (only 3 distinct digits, reject)
Test 4: 000446677 (4 distinct digits, accept)
Test 5: 832835825 (4 distinct digits, accept)
Test 6: 779933400 (5 distinct digits, accept)
Test 7: 123456789 (9 distinct digits, accept)
Tests 4 and 5 both test the same condition but fulfil the condition in two different ways - one with the digits grouped and the other with them interspersed. This ensures that no assumption on the order of the digits is assumed in the code. Test 4 also checks that a string with leading zeros is handled correctly - if it is parsed as an integer they may get cut off ("446677") which would change the result. It's possible this is overloading the test with too much responsibility, but we'll let that pass.
So, on to the other requirement: we can now assume all the strings will be 9 digits with at least 4 distinct digits. Again, test at the boundaries, so strings containing digits that are entered at most 1, 2, 3, 4 or 6 times (6 is the most you can achieve if you have at most 4 distinct digits):
Test 8: 012345678 (max repeats 1, accept)
Test 9: 112323445 (max repeats 2, accept)
Test 10: 777949494 (max repeats 3, accept)
Test 11: 333888810 (max repeats 4, reject)
Test 12: 285848258 (max repeats 4, reject)
Test 13: 494424044 (max repeats 6, reject)
Test 14: 363363663 (max repeats 5, reject)
Test 8 is a variation on Test 7, but again with a leading zero to see if that affects the result. If we already knew Test 4 had passed then Test 8 might be redundant, but since we're writing these up front I think it makes sense to keep it in.
Test 11 and Test 12 both test for 4 repeats of a digit: the first checks for a continuous run of four numbers mid-string (with a red herring of three identical digits first), and the second test the scenario when the digits occur separately.
I added Test 14 because I could see it was possible for there to be two lots of 4 repeats in the same string, which might cause an issue in the code.
I think 14 tests is reasonable for the given requirements; I might add some further variety if the opportunity arose (e.g. if cross-browser testing was required or while testing other functionality that requires the password to be entered first).
Of course, there are other ways to come up with tests for this, which may be preferable if you know a little more about the system. For example, I have assumed there is no distinction between rejecting a password based on max repeats or distinct items - if instead there are separate messages for each infringement you could replace tests 2-14 with the following minimal set of tests that cover all of the boundary points for those two requirements:
Test 2a: 012345678 (9 distinct, max repeat 1, accept)
Test 3a: 662298811 (5 distinct, max repeat 2, accept)
Test 4a: 777119922 (4 distinct, max repeat 3, accept)
Test 5a: 333300999 (3 distinct, max repeat 4, reject)
Test 6a: 777777777 (1 distinct, max repeat 9, reject)
Obviously there are a few combinations not covered here, but it should be sufficient to establish the basic principles are being applied correctly. If you wanted to be more rigorous, you could look at all the possible combinations of numbers of repeats and number of distinct digits (it turns out 25 of the 81 pairs can are valid) - you'd need to add some of the variety I've included above to those which would push the number up even further. Running all those would be a bit excessive for a manual test, and if we assume the developer has taken a fairly sensible approach (!) a lot of those tests would just be exercising the same parts of the code and would not provide any new information.
Of course, you could just run all 1000000000 possible digit combinations, but it might take a while to write the expected outcomes matrix...
Comments
Post a Comment