Google Analytics: Adding True Search Query Terms Using Advanced Filters

In Google Analytics keywords report, you can only segment the keywords by total, paid and non-paid, which pretty much means total, non-cpc and cpc. Also in the keywords report, whenever someone arrives to your website via a cpc medium such as Google Adwords, it will only report the keyword matched in your adgroup keyword list, but not the true search query the visitor typed in.

With a little help of Google Analytics’ custom advanced filters, you’re able to show in your reports what the true search query was and segment it by mediums.

The screenshot above is an example of how the report will look like after the filters are applied. As you can see, within the brackets is the true search query used on search engines and other shopping portals (i.e. Lasoo, Shopping.com, Shopbot). Of course this example doesn’t show any cpc medium because I don’t run any cpc campaigns. Sponsor me and I will :)

This is pretty handy to see on your reports because now you’re able to segment your keywords by medium as well. Of course you could probably use advanced segments to do this but this incorporates true search query and segmentation at the same time.

So the first step I would recommend is to make sure you set up a test profile. You can skip this part and work on your main profile if you like.

To do so, go to your analytics overview dashboard and click on Add Website Profile. Make sure you select the Add a Profile to an existing domain and from the drop down menu, select your website domain.

You can name it test profile or whatever you like as long as you know it’s a test profile. Pretty much this is your hack and abuse profile where you do experiments and tests on it so that it doesn’t affect your main profile’s reports. Once you’re happy it’s working correctly, you can move your filters etc. to your main profile.

Now the next step is to determine your regular expression to extract search query terms from your referral URLs.

There’s an online regex testing tool that I like to use done by gskinner.com. Below is a screen shot of the regex I use to extract the true search query terms on search engines and shopping portals. Of course you can extend this to whatever else you like and to add more search query string parameters within the second bracket.

Once you’re equipped with your working regex, next step is to create custom advanced filters within your Google Analytics profile. In your Google Analytics overview dashboard and click on edit on the chosen profile row. You will see various sections and on the third section is the filters section. Click on Add Filters.

The first custom advanced filter to create is to extract the true search query from the referral URL and set it to the a custom field to be used for the second custom advanced filter. In Field A, I have just used the regular expression I’ve tested above. For Field B, I’ve decided to apply this to cpc, organic and referral mediums. In the output field, I’ve decided how it’s going to look in the reports: search query – campaign medium. Click on Save Changes.

Now create another custom advanced filter. This time I’m going to append the new data (from filter 1) to the campaign term (keyword). Basically for Field A and Field B, I’m just grabbing everything and rewriting the campaign term that appends the new data. Click Save Changes.

Now make sure your filters are in order, that is filter 1 is above filter 2 otherwise it will not work properly. If it’s not in order, you can click on Assign Filter Order.

That’s pretty much it! Give it some time to gather data and you will see in your keywords report the newly added information: true search query terms and the medium.

If you want to read more into Google Analytics’ advanced filter fields, visit Google Analytics help page.

[edit date=13/07/09]

One of my colleagues at First Rate has pointed out that field B in the first filter is not necessary as the variables aren’t even used in the constructor. What will be required is an include filter to only include traffic from a cpc/ppc medium.

You will need to ensure that this filter is placed before the filter that extracts the search query so that the search query extraction filter will only apply to cpc traffic.

[/edit]