Sunday, February 24, 2008

New feature: search tests by name

The upcoming version of TeamCity 3.1 will contain a possibility to search tests with a particular name in build results.

It allows to find out if a particular test has been run in the build. If you have a project with several thousand tests, using a filter is much more convenient than loading the full test list.

Right now, you can see ordinal number of the test, its status, and duration (with possibility to view a graph of the test duration history). All this is described in the corresponding part of the TeamCity UI documentation.

In the future, we plan to add a possibility to view the output of a particular test right from this interface (watch/vote for TW-3553).

Saturday, February 23, 2008

Incidental Field Test: Email Notifications Necessity

Yesterday we noticed an error sending email notification in the logs of our internal TeamCity installation. A quick investigation revealed that the server email sending settings are misconfigured. What was more interesting (and bitter) is that the settings were spoiled 2 weeks ago. Two weeks. All that time nobody was receiving email notifications from our internal TeamCity server! That might seem a disaster, but wait: why we had no complaints about it?

Of course we have other notifiers like Jabber, Windows tray, IDE and RSS, but still email should be the most popular. So why?

My guess is that advanced TeamCity users (who JetBrainers certainly are) do not want to be notified, they want to monitor. Web UI gives you a continuous, real-time feedback on what is going on. There are the whole bunch of additional data like the changes, commits, build running steps, own changes status and more. People may open TeamCity's web in the morning and leave it open for the whole day.

So most people either turned off the notifications at all or just did not notice them missing.

I remember our CruiseControl days when almost the only interface to the server was email. Web UI did not actually produce much more information and after all it was unbearably slow because of the build running on the same machine. Breaking email notifications in those days would mean no CI results - everyone would notice the problem in a couple of hours (or minutes, depending on the length of the builds). But not now. Things change.

Good notifications should not be underestimated. And we do improve notifications and will continue to do so. But who knows, with the tools advances, in a couple of years we may look down on email notifications like we do now on snail mail.

Getting down to earth, what we can do to prevent silly misconfiguration mistakes from going unnoticed? A possible solution is to display a warning to administrators in the web UI if any alike server error occur. Filed as TW-4552.

Thursday, February 21, 2008

500 build hours per day

You might not know but here at Jetbrains we are using our own TeamCity server to build almost all of our projects (including such giants as IntelliJ IDEA and Resharper). This is a part of our company strategy proudly named "eat your own dog food". And it works quite well.

Big projects usually require much time to build and run tests, for example, to compile IntelliJ IDEA sources you will need about 16min on a relatively modern PC. While to run a full test suite you'll need almost two hours. The same picture is in Resharper project: the whole build with compilation and tests takes more than 2 hours to complete. Given such a long builds it is critical to be able to run them in parallel. That is why TeamCity has such feature as build grid from the beginning, otherwise we won't be able to use TeamCity at all.

But how much agents will be enough for us? Well, when we released TeamCity 1.0 in October 2006 we had a pool of about 20 agents:

Things did not change much when we released 2.0 version in April 2007 (except that now we had a so-called "glass" indicating current server load :)

Then as more Jetbrains projects moved to TeamCity we started to realize that we need more power. In December 2007 when TeamCity 3.0 released we had more than 30 agents:

Interestingly even with 33 agents we have almost as many builds waiting in the queue :)

So did we stop? Nope! Thanks to our selfless administrators we increased our pool again:

Now we are able to build more than 700 builds per day (all of them took >500 hours, or about 20 build days) and publish more than 10Gb of artifacts. I think it is serious power. BTW our server is installed on a PC with dual core CPU 3.2GHz / 4Gb RAM of that TeamCity uses 750Mb only.

Do you think so much agents will be enough for us? Will see :)