"Crufty" Web Pages

[ktaylor310]

Years ago when I first went into web design and started to learn about SEO it was widely taught that optimizing your code was important because extraneous code drove your keywords down the page and Google would only search so far down a page - possibly never getting to the keywords. Because of this, I've always been a clean code nut. I know there are many other reasons why clean code is important (page load time, etc.) but I was curious whether bloated code was still an issue with Google.

I was doing a little research and ran across an interview with Adam Lansik, SEO Strategist at Google, wherein he stated:

As I understand, you are talking about huge patches of JavaScript at the top, with the actual content of the page being pushed really far down in the file, and that is something that we really cannot look down upon... But, here is the core problem why we cannot use this in our scoring algorithms currently: There are a ton of very high quality sites, pages and sites from universities, from research institutions, from very well respected ecommerce stores, of which I won't name any, that have really crufty sites, and sites that won't validate. On some of these you can view the source and cry. And, because this is quality content, we really can't use that as an effective signal in search quality. So, you can quote me a saying, I would be thrilled, it would make my day if people would decruft their sites, but it's not going to directly affect their Google ranking.

In the same interview Mr. Lansik states regarding duplicate content:

And, you are right that we do have a particular limit per site on how much we will crawl them each time.

What I'm wondering is, even though the interview indicates that extraneous code doesn't directly effect your ranking, is it conceivable that bloated code could consume more of your Google spider time and possibly prevent some pages from being spidered?

[Sporkman]

If we're talking about so much code that it significantly affects spidering time, then that code should be sequestered into an external file (which would not be loaded by the spider, I'd assume..?). I can't imagine that it would be easy to build & maintain such code-heavy web pages without centrally organizing commonly used code in separate files...

[ktaylor310]

The Google rep was basically referring to the kind of stinky/bloated code generated by content management systems. I just want to be "current thinking". We used to warn our clients about using dynamic urls - that Google couldn't spider them, but that has changed now. Doesn't sound as though bloated code has the same effect on Google as it used to either, though I would never use that as an excuse not to clean it up. So you're saying that the amount of extraneous code necessary to slow down spidering would be huge?

[Sporkman]

Originally Posted by ktaylor310

So you're saying that the amount of extraneous code necessary to slow down spidering would be huge?

Depends on how much code we're talking about, I suppose. For example, the page on my site with the greatest amount of javascript has about 320 lines of code. If, on average, we figure about 60 bytes per line (that's overstating it, though), then that's an extra 19.2 KB. I suppose it depends on what the total size of the page is to get a feel on how much it would relatively slow down the page downloading & processing. As far as the processing, I would assume that the spider would whiz through the javascript faster than the actual html. content, but who knows.

Then again, I'm a light user of javascript (I do most everything on the back end)...

So, in summation: <shrug> I have no more useful information to contribute.

[BarryGrant77]

This happens when someone includes a whole JavaScript Libary, just to open a single new window (1 line).

I always expain my clients and compare it with a newspaper. If you write 100 lines text which can be writen in 1 or 2 lines. Short and significant it should be!

Further it would be also very helpfully for the crawler (like a newspaper reader) if u provide a valid source code. Imagin you read a newspaper with more than 200 misspellings in a single paragraph ...

This is a good sign for quality!

[ktaylor310]

If you write 100 lines text which can be writen in 1 or 2 lines

Yep. I understand that clean code helps load time and code that has errors can stop a site from being fully spidered, but does the amount of code directly effect ranking? That doesn't appear to be the case anymore.

Imagine I'm talking to a client and explaining why clean code is necessary...

- It improves page load time
- prevents possible spidering issues due to linking errors
- validated code can prevent browser problems

But does it still effect your ranking in Google? After reading the interview the only way I can see that ranking could be effected is if the amount of code ate up your allotted spidering time. Not sure if that is possible.

Thanks for all your input "Sporky"! You're a pal

[Sporkman]

Another argument you can make for efficient code is that it's good for the environment (and for your pocketbook, if you run multiple servers): less computation per unit of usage means you can to the same amount of web serving using fewer servers. If everybody put more effort into this, the world would need fewer datacenters, which consume a significant amount of energy.

Originally Posted by ktaylor310

Thanks for all your input "Sporky"! You're a pal

At your service, as always!

[torka]

Originally Posted by ktaylor310

I understand that clean code helps load time and code that has errors can stop a site from being fully spidered, but does the amount of code directly effect ranking? That doesn't appear to be the case anymore.

AFAIK, it was never the case, at least, not with Google.

What you're talking about is basically the old myth of code-to-text ratio. Vanessa Fox, few years ago back when she worked for Google, pretty much debunked that one:

http://videos.webpronews.com/2006/12...ogle-sitemaps/

(This particular video goes into several interesting topics beyond simply code-to-text. Well worth a view all the way through. It's a few years old, but near as I can tell, it's still basically valid information.)

--Torka

[ktaylor310]

Dang! Duped again Thanks for the link Torka. Will definitely check it out. It was probably 10 years ago that I read about coding/Google issues, so highly likely it was bunk.

The question posed to Mr. Lansik was:

Let's talk about a scenario with a webpage that is built in some dynamic fashion and maybe there are two thousand lines of code, for what ultimately is a relatively simple page. And, the unique content to that page, the text and links and things that are really specific to that page are buried seventy percent or eighty percent of the way down in the file. Can this kind of thing hurt your rankings?

I thought he was referring to position of content as opposed to ratio, but since he mentions the word "buried" (which would suggest ratio) and "way down in the file" (which could mean position) I'm really not sure.

Thanks for popping in to help!!

[torka]

Well, that's a second, but related myth -- that somehow content and keywords that are farther down in the code are given less weight by the search engines. Or, conversely, that you have to cluster your target phrases up near the top in order to rank well for them.

And like a lot of myths, this one has a (teensy tiny) grain of truth at its heart.

It is apparently potentially possible to build a page that's just so huge the search engines don't make it all the way to the bottom of the code. But this would have to be enormously huge.

If memory serves, the limit used to be a file size of somewhere around 100 KB. Not lines of code -- file size. And I'm talking about just the size of the basic text file with the HTML code in it; this doesn't count any graphics or media associated with the file. I mean, two thousand lines of code isn't anywhere near what we're talking about here. I'm talking way, way, way beyond that.

This was back in the days of everybody on dial up, when a 100KB file would have taken FOREVER to download. So it was really unlikely that anybody who had a lick of sense was going to post a file that huge in the first place. (Which didn't stop a few people, but remember that part about having "a lick of sense"? I shall say no more. )

Honestly, I haven't done any testing recently to see if that limit is still in place. I can't imagine what sort of crap code you'd have to load a page up with to send the file size over 100KB.

So, anyway, starting with the idea that stuff waaaaay down at the bottom of an impossibly huge file might not get indexed, somehow this transmogrified into the notion that content lower down in the code counts for less when it comes to ranking a page.

Can that kind of thing hurt your rankings? I'd say no. Here's why.

The first thing is to check how the code shows up in the "view source" option of your browser, since that's what the search engine spiders are going to see. If that 2,000 lines of code in the raw file is, for instance, PHP that gets parsed and processed by the server before the page ever makes it to the browser, it may only translate into a few lines of actual code (or none at all, depending on what the script is intended to do).

So the content might not be as far down in the code as a review of the raw, unparsed file might indicate.

Second, even if there are 2,000 lines of code up top when you view source in the browser, most if not all of that will be discarded by Google before they index anything.

Third, if somebody's going to actively optimize a page for a phrase, that phrase should probably be spread throughout the page's content, not sequestered way down at the bottom of the page. It is possible that if a phrase only appears once at the very bottom that the page won't rank well for that phrase. But IMO that would be because the page wasn't well optimized for that phrase, not because the search engines were somehow giving text toward the bottom of the code less weight.

My further less taxes, adjusted for inflation.

--Torka