2012 pre-season testing – Patterns in the Chaos…

Hello guys (and girls). I have a problem. I don’t know how to start this post. So, I think I’ll just get right down to it.

I was a little bit disappointed with my last blog entry (the round-up), because it was too argumentative and not factual enough. The 2012 pre-season testing has, arguably, been one of the most difficult to understand and decipher in the history of the sport. The regulations have apparently closed the gaps between the teams, to a point where it’s difficult to say if Ferrari will be faster than, say, Force India. This ambiguity led many respected journalists to desperate measures, such as Andrew Benson’s decision to take all lap times, from all testing sessions, from both tracks (!), add them up, and produce an average as a means of comparison. I think F1 fans deserve a little bit more than that. So, immediately after I did my last blog entry, I decided to sit down and try to crunch some numbers, trying to find patterns in the chaos (don’t you just love it when the title of the post is mentioned in the post itself?), and see if I can make some sense of what we have seen in testing over the last few weeks. You should, therefore, brace yourselves for a very long post, filled with stats, assumptions and calculations. If you are here just to confirm that Red Bull is fastest or find out what the pecking order is, I suggest you continue reading no further.

So, our testing analysis, first of all, will focus only on the Barcelona tests that took place between 21/02 – 24/02 and 01/03 – 04/03. I think that we must exclude Jerez from the testing picture because (a) it adds a variable for which we simply don’t have enough data to factor in (b) not all teams were present with their 2012 cars and (c) most teams used Jerez as a springboard to get a baseline for their cars, so most laptimes are completely inconclusive. From the various lap times that the teams were performing, it was clear that the teams were following four (4) different testing scenarios:

1. Back to back comparison runs, with heavily fueled cars. I will refer to that in my analysis as “heavy fuel stints”. The stints that I have decided to include in this category and analyze were those that consisted of at least five (5) consecutive lap times, with the first 2-3 laps above the 01:28.000 mark.

2. Back to back comparison runs, usually in shorter stints, with reasonably fueled (but not very heavy cars) – I presume around the half tank mark. I will refer to that in my analysis as “medium fuel stints”. Again, the stints that I have decided to consider as “medium fuel stints” for analysis, were those that consisted of at least five (5) consecutive lap times, with the first 2-3 laps within the 01:24.000 – 01:27.999 bracket.

3. Low fuel runs, usually in very short stints. Some teams (like Red Bull, McLaren and Mercedes) were running a bit more heavy and some were going for more accurate qualifying simulations. I will refer to them in my analysis as “low fuel runs”. In order for a stint to qualify as low fuel and be part of the analysis, there is no minimum limit on the laps / stint, but there is an upper limit of 01:23.999 in terms of lap time.

4. Finally, the last testing scenario that I was able to make out was, of course, race simulations. The term “race simulation” is by itself a bit contentious, since it doesn’t mean the same to every team on the grid. Anyhow, we will see that later on.

To give you an idea, you can see in the picture below (a snapshot from my working file) a typical testing day, and how I have marked the various stints. The low fueled stints are marked orange, the medium fueled stints are marked yellow, the heavy fuel stints are light green and, finally, the race simulations are dark green.

Snapshot of 24/02 Barcelona 2012 pre-season testing

By breaking the testing into these 4 categories, it’s easier to understand how each team compares to each other, and it’s also safer (but definitely not safe) to draw conclusions. We are trying to compare apples to apples and oranges to oranges, and we managed to get some pretty interesting results. I would like to start by examining the low fuel runs, and see what we get:

You can go to any F1 website and it will give you the fastest lap times from testing and an unofficial classification. Do that, and you will find out, for example, that Lotus registered the fastest lap time from all teams with a 01:22.030 and that Mercedes were the slowest with 01:22.932, i.e. 3 tenths off Caterham’s pace. One must, however, take into consideration the amount of fuel that each team is carrying. This is an information that we don’t have. What we do have, however, is the amount of laps that they did during these low fuel runs. By removing the fuel penalty (again, I am using the 0.150 sec / lap figure) that you have to carry in order to complete all the other laps of the stint (apart from the out lap / in lap and the timed lap), we are getting some different results. You can see for example, in the table below, an example, where Red Bull’s fastest lap time was 01:22.662, but fuel corrected on an other stint they were actually able to go faster than that, with a 01:22.157. I don’t think it’s necessary to remind you that Red Bull were still probably carrying quite some fuel onboard, something that we cannot be positively claim for the smaller, midfield teams.

Red Bull low fuel runs, fuel corrected

By applying the above logic to all the low fuel runs of all the teams from all testing sessions, we are getting the results of the following table. I have posted the standard classification to the left (i.e. the fastest lap times as recorded), and next to it I have posted the fastest lap times, after the low fuel stints were fuel corrected.

Fastest laps classification, fuel-corrected

So, there you have it. As you can see, the order changes dramatically. Does this mean that the new order is correct? No, it’s not – but it’s closer to reality. It all has to do with how much fuel was left in the tanks when the cars were returning in the garage, and we don’t have that information available. It’s safe to presume that teams like Sauber and Williams had very little fuel, and those 1:21’s are close enough to a very legitimate qualifying performance. On the other hand, we can safely say that Red Bull and McLaren apparently have a considerable amount of fuel left in the tanks, and that Mercedes are looking good, based on the above table; in fact, they look quite fast. I also assume Caterham must have attempted legitimate qualifying simulations, so the gap they have to Sauber and Williams is not very small. Unless they had at least 6-7 laps of fuel onboard, they seem to not be exactly where they wanted to be. As you understand, these are all big assumptions and we can not base any meaningful conclusions on this table alone, so let’s move to the next part of our analysis: heavy fuel stints.

As I have described to you above, in order to consider heavy fuel stints as part of my analysis they had to be at least 5-laps long and they had to be above a lap time benchmark. In this way, I was able to shed a lot of scattered and inconsistent data; noise, if you prefer. I then worked on the assumption that teams would be testing their cars with comparable (high) fuel loads, and that averaged laptimes between those stints should be a good indication of where we stand. I am afraid I am repeating myself, but I have to stress that it’s only natural that bigger teams will be testing bigger fuel loads and doing all sorts of things to blur the waters, such as doing slow and fast laps in the same stint, slowing down in a specific track sector, not operating DRS or KERS, etc. I have therefore tried not only to produce average laptimes, but I also calculated for each team the average degradation that they suffered in all their heavy stints, from all Barcelona testing sessions and I have excluded the peak times (which can be a result of a driver lifting off, facing traffic, or simply making a mistake).

Surprisingly, Lotus didn’t do any heavy fuel running at all, since they focused on shorter stints, medium fuelled and race simulations. Very surprising. We therefore have no data for them, but we have 9 other teams which are presented in the following figures:

Heavy fuel stints by Red Bull, McLaren, Ferrari and Mercedes

Heavy fuel stints by Force India, Toro Rosso, Sauber, Williams and Caterham

First of all, it’s clear to the naked eye that Ferrari’s runs don’t belong in the first group (top teams) but rather to the 2nd. Visually one can also say that the team that stands out from the 2nd group seems to be Force India, with Caterham being, again, visually slower than the other teams in the group. Let’s take a closer look then at all the numbers involved. In the following table, I have listed the pecking order, as a result of averaging the laptimes that were performed, using the above mentioned criteria. Next to this table, there’s the classification based on the calculated degradation (average), but factoring in all the heavy fuel laps, of all the stints, from all the days of testing and for all teams.

Classification based on (a) heavy fuel running lap times and (b) degradation

As you can see, McLaren was the fastest team of all and, on top of that, enjoyed a very mild degradation of 0.169 seconds per lap. Mercedes were a very close second (less than a tenth), but their degradation was the worst of the lot (0.238 seconds per lap), however such levels of degradation are not cause for concern, especially since we noticed that the Mercedes drivers were pushing more at the start of the stints, taking more life out of the tyres, in comparison to McLaren or Red Bull. McLaren’s fastest lap times are complimented by the fact that they did 11.5 laps per stint, the 2nd biggest tally after Caterham. Mercedes (with 10.8) and Red Bull (with 10.4) were close behind. Red Bull was only 4th fastest, but their degradation was insignificant (easily the best of the lot, at 0.121 seconds), indicating that the drivers weren’t pushing and there’s much more performance to come from this side. Force India’s good results, both in terms of lap times (3rd fastest, 4 tenths off McLaren’s pace) and in terms of degradation support the visual observation that they are the best of the midfield in this comparison. Ferrari’s results are somewhat disappointing. Their average lap times are only good enough for 6th, they suffered the 2nd highest degradation of all (0.227 seconds per lap) and they did the least amount of laps per stint (only 9.4). These results also confirm our visual observations that they should have been graphed with the 2nd group. One last observation is Toro Rosso’s seemingly disappointing performance with regards to lap times. I remain sceptic as to whether this is the real picture or if they have been hiding the car’s pace (more than others, that is), especially considering their decent results in the low fuel runs above.

Having said that, there’s always a degree of uncertainty in those figures; a considerable one, for that matter. Yet I believe that although the above figures and tables don’t represent the truth, they are a good indication; they are, let’s say, a hint of where things stand at the moment. But still, it’s not enough. It’s time to move to the next stage of our analysis, the medium fuel stints. In the following figures, I am doing a graphical representation of the various stints. Before we get there though, a very surprising observation: Red Bull have decided to do absolutely no running on medium fuel levels (apart of course during the race simulations). That’s a completely bizarre decision and it makes us think that, apparently, they are trying to hide the car’s true pace. From all the lap times that I have amassed, it’s clear that Red Bull have only been focusing on very heavy fuel running (as we saw above) and on race simulations, which are also a bit “bizarre”, as we will examine later on.

Medium fuel stints by McLaren, Mercedes, Ferrari and Lotus

Medium fuel stints by Force India, Toro Rosso, Sauber, Williams and Caterham

Before we go into the analysis of the medium fuel stints, it’s worth saying that medium fuel stints are less accurate than heavy fuel stints, because there is more data scatter, so they are considerably less conclusive. The reason is that a medium fuel stint can start at 1:24.100 and end at 1:27.800, or it can start at 1:27.000 and end at 1:30.000, you therefore have more than 6 seconds of data scatter between laptimes. On the contrary, in a heavy fuel stint you start pretty heavy (at 01:28.xxx) and end up in the 1:32’s, so there’s less scope for scatter – it’s much more accurate. On that disclaimer, let’s look at the results:

Classification based on (a) heavy fuel running lap times and (b) degradation (medium stints)

It’s immediately clear that the picture is less intuitive than before. Ferrari remain solidly in the middle of the pack (5th), however they manage more laps / stint than anybody else (9.1). As you can see the degradation figures are higher, and that’s because on a lighter car, the drivers tend to push more. Also, many of the performance tests and setup evaluations are done using half tanks (medium fuel), and that’s when the drivers go for it. McLaren and Mercedes continue to look reasonable, as does Force India. Toro Rosso seem to be going very fast, but they are paying the price in tyre degradation (0.446 seconds per lap), as opposed to Force India who are both fast and consistant. Lotus is behind Ferrari here, and look relatively mediocre. However, it’s important to note that, in general, McLaren, Lotus, Force India and Caterham did must fewer medium fuel laps in comparison to the rest, so the results are a bit skewed. All in all, I think it’s better to leave the medium fuel stints and go to the final part of our analysis, the race simulations.

All teams bar Caterham attempted race simulations in Barcelona, but the way the teams approach a race simulation can vary dramatically. Others choose to stretch the legs of their cars and test the performance whereas others (the most) tend to hide stuff and prefer to test mainly reliability, consistency and procedures, such as pitstops. It makes sense. A race simulation is very easy to be read and analyzed by opponents. You know the fuel loads (you start filled to the brim and work your way through it) so you can calculate the pace, the degradation, the strategies, everything. Which is why most teams prefer to hide as many things as possible during race simulations, and usually do them to get their reliability, systems and consistency checked. I would therefore pay the least importance on the race simulation figures (it’s clear to me that the most important testing scenarios are on that order: (a) heavy fuel stints (b) low fuen stints (c) medium fuel stints and (d) race sims), but for what it’s worth I am posting the race sim results below:

Race simulations

A brief explanation on how I compiled the above numbers: Some teams did a bit less than 66 laps (theoretical Barcelona distance) and some a bit more, so I normalized all race simulations to reflect a 66 laps race, using the average lap times. Also, in the final race time I included the pitstops. As you may recall from our previous blogs, the average time penalty for a pitstop in Barcelona is 20 seconds, so adding all the pitstops together we finally get the total race time. As you can already see, there are some oddities. First, Williams 24/02 race simulation is the 2nd fastest recorded, whereas their 03/03 simulation is the slowest of all. McLaren, despite for their excellent tyre degradation, seem to be lacking in pace, etc. Mercedes’ race simulation is a very good one, and should provide hope to the Merc fans, but I have to stress that these race simulations is the least reliable set of data we have in our disposal.

Which brings us to the conlcusion of this article. As I said, I was disappointed a bit with my previous blog entry, but I now think we have some good data to base our “conclusions”. First, and foremost, all the top teams have excelled in one part of our analysis or the other. Mercedes have been very quick in low fuel runs and also 2nd quickest in heavy fuel running, not to mention their impressive race simulation. McLaren have been the fastest in heavy fuel running and have exhibited admirably low levels of degradation throughout. Red Bull have been relatively fast in heavy stints with the tiniest of degradations under all conditions. Even Force India have shown flashes of brilliance, with good and consistant speed and no degradation issues.

Ferrari though is different. They have been strikingly mediocre, failing to shine in any of the testing scenarios we examined, and particularly the important ones (i.e. heavy and low fuel stints). Their heavy fuel pace is unspectacular and their tyre degradation at the same time is a bit poor. Their quickest lap times in low fuel mode were 2nd to last, and they have never been able to escape the middle position of the tables. If they are sandbagging, they are doing a hell of job. A lot of the people have suggested that the figures aren’t bad for Ferrari and that we are basing our negative reviews on heresay and rumours. The data, though, seem to suggest that Ferrari are behind at the moment, firmly in the midfield.

Red Bull are fast, but are clearly hiding the majority of their car’s true pace – they should be bloody hard to beat in Australia. McLaren are very fast in all situations and their degradation patterns suggest that they are also consistant. Mercedes have clearly done a much better job this year, and they are in a much better position to challenge. It seems they have inherited Ferrari’s position in the top 3 of the sport. Ferrari have fallen in the clutches of Force India, Lotus, and the rest. Lotus’ pace is a bit inconclusive, because they have done no heavy fuel stints and their fuel corrected low stints pace is a questionmark. As for the rest, they all look in very good shape, which is proof of the extremely tight battle that is developing in the midfield.

I, for one, cannot wait for the season to commence, and I hope you will be here with me to share our thoughts and experiences… I hope you liked this series of articles on testing, and do stick around! 😉