Google’s Opinion on Crawling Dynamic URLs vs. Static URLs

In my line of work, there has been a widespread belief that Google is unable to crawl “unoptimised” URLs and that ideally all URLs should be in a directory structure. This is totally untrue as mentioned by Google’s blog post.

In fact, Google actually says that they might have problems crawling and ranking your dynamic URL if you try to change it to look like a static URL.

However, this post will give you my opinion on why URL optimisation should be considered and to what extent should we optimise it to.

Dynamic URLs

An example of an “unoptimised” URL usually refers to dynamic pages that contain query string parameters such as https://www.dannyism.com/?p=230 where the variable number 230 refers to the post id. You can usually tell dynamic URLs where there is a question mark (?) in it followed by the equals and ampersand symbol (=, &).

Static URLs

You usually see static URLs end with the file extension depending what web framework the website is using and without the presence of dynamic parameters (as mentioned above). This can be .php, .html, .asp, .jsp and so on. However, it is somewhat popular now to exclude the extensions for aesthetic reasons and to avoid confusion between all the different types of extensions (which can be hard to remember!).

Why Optimise?

In my line of work, I often consult to clients to optimise their URL in manner which displays clearly their site architecture in a directory structure. I do not advise optimising URL for the myths that Google cannot crawl and rank dynamic URLs nor can they handle a lot of parameters.

Take for example https://www.dannyism.com/?p=230. When you go to that link, it actually rewrites the URL to https://www.dannyism.com/2008/10/23/the-8020-seo-rule-in-web-page-optimisation/ which is optimised in a directory structure. It shows clearly my site architecture with the date of the post and also the title. I also believe it helps people remember your URL much easier than seeing a bunch of cryptic dynamic parameters.

Does Optimising Help My Rankings?

I get asked whether URL optimisation is really essential and does it play a significant factor in search engine results page (SERP) rankings? Well, as a matter of fact, it does play a significant factor in SERPs but probably not in the way you’d expect.

I do believe that you do not contribute significantly or perhaps nothing at all towards Google’s algorithm in determining your rank in the SERP by optimising your URL structure. However, I also do believe that if you’re optimising it by putting keywords in your URL, it does help not just ranking wise, but also clickthrough rate.

As you can see above, keywords are actually bolded in the SERP and will stand out more to the search user, thus will improve your chances on increasing traffic and clickthrough rate. You can read more in my earlier post on the 80/20 Rule in Web Page Optimisation which lets you know how you can achieve 80% of the results in web page optimisation with 20% effort.

URL Optimisation Tips

1. Do not optimise your search URL structure from like https://www.dannyism.com/?s=seo&cat=search%20engine%20optimisation to like https://www.dannyism.com/search/seo/category/search-engine-optimisation.

   Although this example has only two parameters, it can get quite messy if your search has several filters and the URL structure can get quite confusing and difficult to manage and understand.

2. Remove all parameters that have nothing to do with the actual content. This is especially true with session ids in URLs which will cause duplicate content issues.
3. Try to keep your URL length as short as possible unless absolutely necessary. A good rule of thumb is not to go beyond 3 or 4 directories in depth. If you look at the SERPs, there is a limit to how much of the URL Google displays.
4. Make sure your keywords are placed towards the start of the URL rather than towards the end.