Alternating pass/fail

Hi,

Hi,
I'm using VS 2002 with TestDriven.NET and .NET Framework SDK 1.0. Additionally, I installed .NET Framework 1.1 (just the framework, not the SDK) especially for TypeMock.

We haven't actually tried this but it should work.

I was just comparing different mock frameworks to decide which one to use, when I found some very strange behavior. When I repeatedly run the same set of tests without changing anything in the source code, the result of the one test that uses TypeMock alternates between pass and fail.

:?: What version of TypeMock are you using?

:arrow: Try using: MockAll

Mock math = MockManager.MockAll(typeof(CMath));

:arrow: and also try using the MockObject

MockObject math = MockManager.MockObject(typeof(CMath));
Calculator calculator = new Calculator((IMath)math.Object);

:arrow: or use MockObject on the Interface

MockObject math = MockManager.MockObject(typeof(IMath));
Calculator calculator = new Calculator((IMath)math.Object);

:arrow: Also please post the failure when you use MockManager.Verify(); WITHOUT the Assert statments

[Test] 
      public void Triple() 
      { 
         // setup mock/dependencies 
         MockManager.Init(); 
         Mock math = MockManager.Mock(typeof(CMath)); 
         Calculator calculator = new Calculator(new CMath()); 
         math.Strict = true; 

         // expectations 
         math.ExpectAndReturn("Multiply", 0).Args(0,3); 
         math.ExpectAndReturn("Multiply", 3).Args(1,3); 
         math.ExpectAndReturn("Multiply", 6).Args(2,3); 
         math.ExpectAndReturn("Multiply", 3000); 
 
         // execution
         calculator.Triple(0); 
         calculator.Triple(1); 
         calculator.Triple(2); 
         calculator.Triple(1000); 
         // verify mock 
         MockManager.Verify(); 
      } 
   } 
} 

:?: What version of TypeMock are you using?

2.3.0

Also please post the failure when you use MockManager.Verify(); WITHOUT the Assert statments

Here it is:

TestCase 'Sandbox.UnitTest.TFTypeMock.Triple' failed: TypeMock.VerifyException : 
TypeMock Verification: Method Sandbox.CMath.Multiply() has 4 more expected calls

   at TypeMock.Mock.Verify()
   at TypeMock.Expectations.Verify()
   at TypeMock.MockManager.Verify()
   c:mydocsisual studio projectssandbox	ftypemock.cs(39,0): at Sandbox.UnitTest.TFTypeMock.Triple()

Using MockAll fixes the problem, using MockObject on the Interface, too.
Using MockObject on the class produces the same behaviour as the original code, and also the same failure messages.

I think I found the real problem:
When I run the test alone, it always passes as it should, only when I run the complete set of tests the alternating occurs. As I said, I was comparing frameworks, so every Test runs on a different one. Commenting out the NMock test completely solves the problem. I'm not sure if this can still be considered a bug, but I think it's worth mentioning somewhere in the documentation that running more than one Mock framework at a time can produce unpredictable results. People considering migrating incrementally will surely appreciate it.

Hi,
This problem is actually to do with the way the TypeMock handles different instances of the mocked type.
When you use Mock, TypeMock waits for the next time a new is encountered and assigns the expectations to that instance.
When using MockAll it assigns the expectation to all instances of the type.
When using MockObject it returns a mocked object. (This had a bug...)

For some reason the new instance is missed.
This could be for a few reasons -
:arrow: No new is called (Singletons for example)
:arrow: An extra new is called (in constructor etc) - this can happen because of compiler optimizations too. For example the new is called before the MockManager.Mock is called.
:arrow: Bug in MockObject

The MockObject had a bug and should work with TypeMock 2.3.2 - Please try it.

While comparing the mocking frameworks, TypeMock's biggest added value is the ability to mock static methods and concrete classes (even those created internally in the tested methods)

Thanks a lot for your explanations.

I just followed your advice and installed TypeMock 2.3.2. Well, at least I tried to. After running TypeMockSetup232.msi and starting Visual Studio, the TypeMock Auto Updater told me there was a new version. I found the installed version identifies itself as 2.3.1. Setup/Repair did not change that, neither did uninstall/reinstall.

After running TypeMockSetup232.msi and starting Visual Studio, the TypeMock Auto Updater told me there was a new version. I found the installed version identifies itself as 2.3.1. Setup/Repair did not change that, neither did uninstall/reinstall.

Hi,
This is our mistake. We will fix it and post the fix, in any case the new version is already installed on your machine.