Don’t blame Feedburner/Google, it’s Netvibes’ fault
A couple days back a tremor was felt in the webs when TheNextWeb showed a video with how to hack your feedburner feed numbers by making a bunch of Netvibes widgets of your feed. Everyone then proceeded to say what a failing this was of Feedburner’s or Google’s. Let me just say this is not Feedburner’s problem - ok, it is Feedburner’s problem in that it is giving them bad numbers, but it isn’t their problem to fix. In this they have two technical options, believe what Netvibes is telling them or stop reporting Netvibes numbers. Obviously they can go to Netvibes and say - you better fix this crap. Towards the bottom I’ll show how easy it is to mess with Feedburner directly and how difficult it’d be for them to solve.
The difficulty with feed reporting
Here’s the situation. Reporting on RSS subscriptions is an approximate business at best. You know how general web metrics are really hard to get? And that’s with all the benefits of browser sophistication and what not? RSS is even more so because you don’t have any sophistication of RSS readers. They just poll and poll and poll without telling you anything about them except an IP Address and a client name and version. No cookies, no javascript, no nothin. So Feedburner has to do some kind of crazy voodoo to somehow cull this mass of faceless information into a coherent set of numbers. That’s why your Feedburner numbers fluctuate on a daily basis.
Now some aggregating sites represent a bunch of people, Google Reader, for example. When you have a Google Reader account and subscribe Reader becomes your client and goes to fetch the feed and because of this under normal circumstances everyone who subscribed to a feed in Google Reader would be hidden because it would come across as a single client whether 1 or 1000 people subscribed. Reader (and I think it was the first to do this, but I could be wrong) cleverly added into the client name that it sends the actual number of subscribers it is representing for this feed.
Hacking Netvibes
So for Reader you can know specifically and not have to estimate how many subscribers there are. Wonderful! Netvibes does this too. It’s very nice and convenient way to get at these numbers and the only way. If they didn’t do this, all them subs would boil down to 1 Reader and 1 Netvibes subscriber. That’s life. So what’s happened with this hack is that it was a way to trick Netvibes into sending to Feedburner a much inflated number of subscribers.
Here’s an example of what the Netvibes UserAgent string looks like in the logs (pulled from my very own logfiles):
Netvibes (http://www.netvibes.com/; 2 subscribers; feedId: 5558025)
You can see - there’s nothing Feedburner can do here. NV needs to fix this loophole and then we can all get back to our normal approximation of subscribers.
Hacking Feedburner
If you don’t believe me… I ran a test yesterday to see if Feedburner was using the client string as one factor to infer number of users. I wrote a few lines of perl to grab my feed with a different client string each time and lo and behold, today my subscriber number is about 100 higher than it was yesterday (checkout the Feedburner chicklet near the top of my rightnav).
#!/usr/bin/perl
use strict;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
foreach my $major ( 0..9 ) {
foreach my $minor ( 0..9 ) {
$ua->agent(”Deasil Tester/$major.$minor”);
$ua->get(’http://feeds.feedburner.com/deasil/QfYJ’);
sleep(int(rand(60)));
}
}
Now that would be something the Feedburner could fix. Of course, exactly how it would magically know that these agents and not others are fake, I’m not sure. If they did it by agent, I could simply change my agent string, if they did it by IP Address, I could reset my router until TimeWarner gave me a new dynamic IP. It’s a hard business they’re in - but that’s why you go to Feedburner, because if you had to figure it all out, you’d do a much worse job. Feedburner has experience, talent and breadth of data points to be about as effective as you’re going to be pulling this info.
In general you have to take Feedburner numbers with a grain of salt, but for most sites it does a great job approximating the number of subscribers you have. Sure it can be artificially inflated and sure if you’re a major site and you start appearing in default lists of feeds in various aggregators it can be wildly wrong, but those are the exceptions rather than the rules. Tomorrow, my sub numbers will fall back to their normal lowly selves and life will go on.
