Can Num3rs Lie?

In a series of posts Oren (Ayende) and Sasha have debated the need to design for performance. Both parties make some very compelling arguments and its real interesting reading material. Since I still need to sort my thoughts on the matter (which will be presented in a future post) here’s just a teaser.

Oren has rose to the challenge, and in a series of experiments have tested the performance difference between the two design approaches. Here’s one of his experiment result

And the results are:

  • IList<T> – 5789 milliseconds
  • List<string>, NullList – 3941 millseconds

Note that this is significantly higher than the previous results, because no we run the results two hundred million times.

Individual method call here costs:

  • IList<T> – 0.0000289489
  • List<string>, NullList – 0.0000197096

We now have a major difference between the two calls. When we had a single impl of the interface, the difference between the two was: 0.00000406 milliseconds.

With multiple implementations, the difference between the two methods is: 0.0000092393 milliseconds. Much higher than before, and still less than a microsecond.

which Oren concludes that although there is a performance difference, the actual time difference even in this close to “worse” case is so small so there is not much point to consider it.

is there?

lets try a different interpretation:

Reading the numbers I can also say that using the List<String> is 31% (0.0000092393/0.0000289489) faster then using the IList<String>.

Question – How much time and money do you think that Intel&AMD will invest in the next couple of years to achieve a CPU which runs 31% faster then current technology? (actually I don’t know either, but I sure wouldn’t mind putting my hands on a small part of it).

Point is, that in cases where performance is an issue (which is not true for all cases), I would most definitely go down the design road which offer a 31% speedup.