Testing a Shuffling Function

Testing a function which produces output based on random values presents challenges not present when dealing with a strictly deterministic function. Here I will be using a function which simulates shuffling a deck of cards to illustrate some of these considerations.

The shuffle function I am using (written in C#) is as follows:

    public Card[] ShuffleCards(Card[] deck)
    {
        Card[] cards = (Card[])deck.Clone();
        for (int a = 0; a < cards.Length; a++)
        {
            int b = Rand.Next(a, cards.Count());
            Card selectedCard = cards[b];
            cards[b] = cards[a];
            cards[a] = selectedCard;
        }
    
        return cards;
    }

For my first test, I will check to ensure that the function returns the correct number of elements. For example, if a deck starts with 60 cards, it should end with 60 cards.
(Note: test cases designed to work with Nunit )

   [Test]
    public void Shuffle_isCorrectLengthAfter()
    {
        CardFunctions cf = new CardFunctions();
        Card[] TestDeck = new Card[5] 
                { new Card(), new Card(), new Card(), new Card(), new Card() };

        TestDeck = cf.ShuffleCards(TestDeck);

        Assert.AreEqual(TestDeck.Length, 5);
    }

The second test ensures that all elements are still present after the shuffle. This test is specifically to catch instances where one card is duplicated replacing one or more other cards. For example, a deck started out 1, 2, 3, 4, 5 but ended up 1, 2, 2, 2, 2.

    [Test]
    public void Shuffle_isCorrectSetAfter()
    {
        CardFunctions cf = new CardFunctions();

        Card[] PreDeck = new Card[]
                { new Card(), new Card(), new Card(), new Card(), new Card() };

        Card[] PostDeck = cf.ShuffleCards(PreDeck);

        ArrayList decklist = new ArrayList(new Card[5]
                { PostDeck[0], PostDeck[1], PostDeck[2],
                PostDeck[3], PostDeck[4] });

        Assert.Contains(PreDeck[0], decklist);
        Assert.Contains(PreDeck[1], decklist);
        Assert.Contains(PreDeck[2], decklist);
        Assert.Contains(PreDeck[3], decklist);
        Assert.Contains(PreDeck[4], decklist);
    }

Now that we know that none of the cards get lost or duplicated, we need to ensure that shuffling is indeed changing the order. It's important to note that since the shuffling is random, there is a small chance that the order may be shuffled into the same order. The odds of this happening through chance alone are low, but even so, I still include a note on the possibility of false failure by the test.

    [Test]
    // NOTE: There is a very slight chance this could
    //       occasionally produce false "fail" results.
    public void Shuffle_orderChangedAfter()
    {
        CardFunctions cf = new CardFunctions();
        Card[] PreDeck = new Card[]
                { new Card(), new Card(), new Card(), new Card(), new Card() };
        Card[] PostDeck1 = cf.ShuffleCards(PreDeck);
        Card[] PostDeck2 = cf.ShuffleCards(PreDeck);

        Assert.AreEqual(true, (PostDeck1[0] != PostDeck2[0] ||
                               PostDeck1[1] != PostDeck2[1] ||
                               PostDeck1[2] != PostDeck2[2] ||
                               PostDeck1[3] != PostDeck2[3] ||
                               PostDeck1[4] != PostDeck2[4]));
    }

For the final test, I need to ensure that no card ends up in the same position more than statistically likely. As the previous test, when dealing with random values, there is always a small chance that this could produce false fail results. To keep the likelihood of this low, I am using a relatively high number of cycles.

    [Test]
    // NOTE: There is a very slight chance this could
    //       occasionally produce false "fail" results.
    public void Shuffle_outputMatchesProbability()
    {   
        CardFunctions cf = new CardFunctions();
        int[] counter = new int[5];

        Card[] PreDeck = new Card[]
                { new Card(), new Card(), new Card(), new Card(), new Card() };
        
        for (int cycle = 0; cycle < 10000; cycle++)
        {
            Card[] PostDeck = cf.ShuffleCards(PreDeck);

            for (int i = 0; i < PreDeck.Length; i++)
                for (int j = 0; j < PostDeck.Length; j++)
                    if ( PreDeck[i] == PostDeck[j])
                        counter[j] += i;
        }

        foreach (int num in counter)
        {
            Assert.AreEqual(true, (num >= 19000) && (num <= 21000));
        }
    }

If the shuffle function passes all of these tests, I can be reasonably confident that it is randomizing the order of the cards as expected.