chevron-thin-right chevron-thin-left brand cancel-circle search youtube-icon google-plus-icon linkedin-icon facebook-icon twitter-icon toolbox download check linkedin phone twitter-old google-plus facebook profile-male chat calendar profile-male
Ask a question
The Unit Testing Company
Development Resources
Knowledge Base
Welcome to Typemock Community! Here you can ask and receive answers from other community members. If you liked or disliked an answer or thread: react with an up- or downvote.

Related questions

AAA tests pass individually, fail in suite

0 votes
I have a simple class: 

public class MailThread
    {
        public static int GetThreadRootContentId(string inReplyTo)
        {
            if (inReplyTo.IsNullOrEmpty())
            {
                return 0;
            }

            var sqlParamsList = new List<SqlParameter>
                                    {
                                        DataAccess.getSqlParameter("@In_Reply_To", SqlDbType.VarChar, inReplyTo),
                                    };

            var result = DataAccess.RunSPReturnSingleItem("[dbo].[MessageThreads.GetOriginalMessageContentId]", sqlParamsList.ToArray());

            return result == DBNull.Value ? 0 : Convert.ToInt32(result);
        }

        public static bool BelongsToExistingThread(string inReplyTo)
        {
            return inReplyTo.IsNullOrEmpty() ? false : GetThreadRootContentId(inReplyTo) > 0;
        }
    }


And a simple test for the BelongsToExistingThread method: 

       [Test]
        public void BelongsToExistingThreadReturnsTrueWhenRootContentFound()
        {
            Isolate.Fake.StaticMethods<MailThread>();
            Isolate.WhenCalled(() => MailThread.GetThreadRootContentId("")).WillReturn(7);
            Isolate.WhenCalled(() => MailThread.BelongsToExistingThread("22")).CallOriginal();

            Assert.That(MailThread.BelongsToExistingThread("22"), Is.True);

            Isolate.Verify.WasCalledWithAnyArguments(() => MailThread.GetThreadRootContentId(""));
            Isolate.Verify.WasCalledWithAnyArguments(() => MailThread.BelongsToExistingThread("22"));
        }


This method passes when run as an individual test, but fails when run as a suite. I am using the [Isolated] attribute to call Isolate.CleanUp().

Here is the error:

TypeMock.VerifyException:
TypeMock Verification: Unexpected Call to MailThread.BelongsToExistingThread()

What's the problem here?
asked by scalvert
Report User
 (1.2k points)

10 Answers

0 votes
Do you have any other tests in the suite that use Isolator? I have run into a similar problem before - where tests run successfully by themselves or in their own fixture, but fail when run in the full suite - and it always turns out that some test in some other fixture has their mocks set up incorrectly. For example, one of the other tests in some other fixture may be doing a MockAll when it needs to only mock one instance, or maybe the other test fixture isn't clearing/verifying mocks after a test so there are residual expectations hanging out. That's why I continually stress to my team that they should only mock what they absolutely need. If you only need a call to a method to be mocked one time, don't set the mocks up to always mock the call. That sort of thing.

Unfortunately, if that's the case, finding the problem is going to be a challenge. I generally will load up the tests in the NUnit GUI tester and test things by namespace, working my way up the tree until I either get to running every test in the suite or find some sub-section of the suite that causes the test in question to fail. After that... well, honestly, it's "exclude a test, recompile, run tests... exclude a test, recompile, run tests..." It's hard.

The Tracer utility might help you out, too. It can help you see what expectations are getting set up in each test so you can figure out which tests in the suite are setting up expectations that aren't getting met.
answered by tillig
Report User
 (6.7k points)
0 votes
Thanks Travis.

To build further on the Tracer tip: you can activate the tracer and run the test suite in debug, with a breakpoint in the first test you expect to fail before any mocking is done. When you hit the breakpoint you should not see any expectations in the tracer; if you do it means some test did not clear out properly.

Doron
Typemock support
answered by doron
Report User
 (17.2k points)
0 votes
Thanks to you both. I'll dig into the trace and see what i can find.
answered by scalvert
Report User
 (1.2k points)
0 votes
I think I've found the solution to the problem.

I have two tests: 
[Test]
public void Test1()
{
            Isolate.Fake.StaticMethods<ImageContentObject>();
            Isolate.WhenCalled(() => ImageContentObject.SetDefault(1, 1, true)).WillReturn(1);
//...my test code
}

public void Test2() {
            Isolate.Fake.StaticMethods<ImageContentObject>();
            Isolate.WhenCalled(() => ImageContentObject.SetDefault(1, 1, true)).WillReturn(1);
//...my test code

}


If I run the whole test fixture, Test1 passes and Test2 fails. The problem seems to be with 
Isolate.Fake.StaticMethods<ImageContentObject>();
and calling it from Test2, a second time, within the same fixture. If I comment that line out from the second test, both Tests pass.

I have Isolate.CleanUp() at the end of both tests and the [Isolated] attribute at the class definition.

Is this a bug? Or am I just not creating my fake static methods correctly.
answered by jpennal
Report User
 (800 points)
0 votes
I have fine tuned my test case to the specific problem (I hope).

For my two tests: 
        [Test]
        public void Test1()
        {
            Isolate.Fake.StaticMethods<ImageContentObject>();
            Isolate.WhenCalled(() => ImageContentObject.SetDefault(1, 1, true)).WillReturn(-1);

//..test code here

            Isolate.Verify.WasCalledWithAnyArguments(() => ImageContentObject.SetDefault(1, 1, true));
            Isolate.CleanUp();
        }

        [Test]
        public void Test2()
        {
            Isolate.Fake.StaticMethods<ImageContentObject>();
            Isolate.WhenCalled(() => ImageContentObject.SetDefault(1, 1, true)).WillReturn(1);
//..test code here

            Isolate.Verify.WasCalledWithAnyArguments(() => ImageContentObject.SetDefault(1, 1, true));
            Isolate.CleanUp();
        }


When I run the entire fixture, I get a TypeMock.VerifyException in the second test.

If I remove the [Isolated] attribute from the class and comment out the Isolate.Cleanup() methods, all of the tests run in the fixture correctly just as they do individually.

There seems to be some conflict between CleanUp and faking StaticMethods. Is this a bug in the way StaticMethods are mocked in TypeMock?
answered by jpennal
Report User
 (800 points)
0 votes
Hi

It seems like a bug. :(
My guess is that this is because the same static instance of ImageContentObject class is running in both tests.

Thanks for the clear example.
We will update you once it is fixed.
answered by ohad
Report User
 (35.4k points)
0 votes
Thanks for the update (5.1.1 build #63) that you sent me. I tried running 5.1.2 though and I was getting failures again. Is this fix in that version?
answered by jpennal
Report User
 (800 points)
0 votes
No the bug you've reported is part of the next version of Isolator, the path you've received has all of the features/bug fixes of 5.1.2 as well and should be ok to use until the next version is officially released
answered by dhelper
Report User
 (11.9k points)
0 votes
Hello,

I am using typemock 5.3.4 and I have the impression that this is still an issue.

In my code I block log messages in one test in my fixture and this is blocking logging in all my tests. 

Isolate.Fake.StaticMethods<Logger> ();
Isolate.WhenCalled (() => Logger.Log (this, MsgLevel.Error, "")).IgnoreCall ( );


I am using the Isolated tag and I have tried using Isolate.CleanUp (), but my logger blocking seems to be an all or none thing.

thanks in advance
dennis.
answered by sellingerd
Report User
 (5.7k points)
0 votes
Hi Dennis,

It could be that there's something in the static constructor that causes a problem.
Try using: 
Isolate.Fake.StaticConstructor<Logger> ();


at the beginning of each test.

If that doesn't work, we'll probably need a reproduction.
Let me know how it works, and we'll continue from there.
answered by gilz
Report User
 (14.5k points)
...

Products

Help

Company

Community

Contact

Typemock Store (US toll FREE 866–895–4680). Copyright © 2006 Typemock Inc. All rights reserved. Terms of Use.