Menu
vs 
- Is Eye Tracking worth the trouble?Back in 2017, I embarked on yet another digital adventure, creating another website. The excitement was palpable, each pixel and piece of content meticulously crafted. I was proud and couldn’t wait to show it to people. But there was an underlying doubt, a silent question that lingered: Was it any good? Will people tell me the truth when I ask them?
Whenever you or your team creates a new website or landing page a few questions come straight to mind:
Now, we all know how decisions about these things are generally done. The first one is the good old “best practices” and the second one is just opinion. Nobody really wants to admit that the latter is a major driver, but a lot of professional web designers know what I mean.
There are plenty of user experience (UX) tools on the market today, the list is basically endless. Some of them are online survey questions, watching users interact live with your website, tracking visitor’s cursor, and eye tracking.
I am generally quite skeptical of just asking questions, that’s because most of the time people don’t know why they do or don’t do something. My personal preference is raw behavioral data, so I tended more towards cursor or eye tracking.
Now don’t get me wrong, the other ones have their place, just that more often than not, people put too much faith in the results and forget that the way they ask questions is already highly biased.
When I came across cursor tracking it seemed pretty interesting to me as a method, but then I read that it was supposed to be as good as eye tracking that made my alarm go off. The claim that was thrown around was “the poor man’s eye tracker”. I immediately started searching for actual research papers that investigated this but couldn’t really find any. All I could find were the claims of the cursor tracking companies.
In the end I basically got sidetracked and decided to investigate this myself by comparing cursor and eye tracking to each other, the good old scientific way. You can find the full details of the paper below but here are the basic results.
If cursor position and eye position are related, there should be a correlation. Interestingly, there actually is one of r = 0.22 (SE=0.045, p < 0.001). That is quite interesting, but keep in mind that correlation is not the final answer here, especially when you look at the graphs (Figure 1).
We can clearly see that the correlation is just positional and not movement related, otherwise speed would correlate too, which it doesn’t. Another aspect is that correlations can also occur independent of the distance of the cursor and the eye position.
Meaning that if cursor and eye position are so similar to each other, the average distance between cursor and eye position should be pretty small. Unfortunately, the average distance between the two is quite large and on average 9.39 cm. The next graph (Figure 2) shows the distribution of distances.
Ok, hold on, you might say. The cursor can still be at the same locations the eye was, just not at the same time! I would say, that you are completely right, so let’s look at the pixel coverage independent of time, which you can see on the heatmaps. Also here, it doesn’t look too good. There is only a 2.3% overlap of cursor to eye position.
Now comes the final argument in favor of cursor tracking, its scalability. I often heard that cursor tracking makes up for all of this by simply adding more users that are being tested. So instead of having 10 participants for eye tracking, cursor tracking uses 100 to make up for the lack of detail. But does that work?
To answer that question, I performed an entropy analysis that looked at exactly that. How much information does a new user add for each tool. Let me tell you, it doesn’t look too good for cursor tracking here either. Both tools reach higher levels of entropy the more users are added but they reach an asymptote at some point. Eye tracking tapers off at around 6 bits and cursor tracking at ~3.5 bits, which also means that cursor tracking will never reach the entropy levels of eye tracking. The graph itself is also on a logarithmic scale, which means that eye tracking entails not only twice as much information but way more than that.
Cursor tracking does not constitute “the poor man’s eye tracker”, not even close. On the contrary, it could be costly to interpret cursor data as loose eye tracking data, as it will suggest false optimization points.
Authors: Adjmal Sarwary
Publication date: 2017/10/16
Download link to the paper: Eye trackins vs cursor tracking: is eye tracking worth the trouble?
The PaperCommenting Rules: Being critical is fine, if you are being rude, we’ll delete your stuff. Please do not put your URL in the comment text and please use your PERSONAL name or initials and not your business name, as the latter comes off like spam. Have fun and thanks for your input.
Join a growing community. Every Friday I share the most recent insights from what I have been up to, directly to your inbox.