Firefox Search Engine Cautions, Recommendations

In Mozilla We Trust

This article was last updated on 11-Dec-2018. See the revision history at the end of this document for a list of changes.

Introduction

The scope of this tutorial is limited primarily to preserving your privacy when using the default Firefox (or derivative thereof) search engine plugins, as well as to present alternative ways to use the major search indexes, such as Google, Bing and Yahoo.

When ‘free’ software isn’t

Many of us probably tend to associate the open-source software community with individuals or small organizations that freely and selflessly give away their work expecting little or nothing in return, however this is not the case for the multi-million dollar Mozilla Foundation. The hundreds of millions of dollars that Mozilla rakes in annually is largely due to its partnership with search engine companies such as Google. These ethically challenged mega-corporations then track your web activities and sell the data they collect to advertisers and governments and who knows who else. I think these kinds of partnerships are clearly at odds with Mozilla’s statement, “Committed to you, your privacy and an open Web“.

So how does Mozilla get paid by the mega-giants like Google? Simple: Any time you use the default search engine plugins that are packaged with the browser, parameters similar to these are added to your search query:

client=firefox-a
name="appid" value="ffd"
name="hspart" value="mozilla"

These parameters tell the search engine that you are using a Firefox/Mozilla product and that’s all it takes for Mozilla to rake in the dough. If you do not wish to participate in these affiliate schemes and/or value your privacy, read on.

Types of search engines

The two basic types of search engines are meta search engines and search indexes and it is important to understand the difference. Google, Yahoo and Bing for example, use software robots known as “crawlers” to discover and index web content. In other words these companies actively seek out updated and fresh content to store in their databases so that it’s ready for you to find. Meta search engines do not typically index the web and instead rely primarily on third parties like Google, Yahoo and Bing to provide their search results and therefore when you use these so-called “alternative” search engines, such as DuckDuckGo, Startpage, Searx, etc., you are still subject to the content manipulation and censorship employed by the big corporations. While alternative meta search engines do indeed make a great deal of sense from a privacy perspective since one can avoid being tracked by the big indexes directly, they are not true alternatives as they are often described, but more like search engine proxies which simply provide their own interface in order to display the search results as provided by third parties. These alternative search engines are also subject to local laws, such as secret surveillance requests issued by a government.

Indexing the web and storing the massive amount of data that results in a centralized way such as Google does, is an incredibly expensive proposition which requires a massive amount of infrastructure and this is why the much smaller meta search companies like DuckDuckGo, Startpage and others rely heavily upon corporations like Google. There is an even better solution however, one which both respects your privacy and is censorship resistant and these are distributed search indexes. Imagine a free, open-source, decentralized search engine where the search index is distributed among millions of personal computers like yours, each storing a piece of the whole. This is what the developers behind YaCy have done with their peer-to-peer search engine and i think it’s a great way to move forward and avoid corporate censorship and attacks on personal privacy.

Privacy-centric search engines

For a list of search engines that focus on privacy, see the article, Alternative Search Engines That Respect Your Privacy.

Adding search engines to Firefox

Possibly the easiest way to mitigate risks to your anonymity posed by using the default Firefox search engines is to simply disable all of them and use alternatives. One of my favorite choices is the open source and highly configurable Searx meta search engine which you can host on your own server if you like, or you can use any one of a number of instances hosted by other folks. Like DuckDuckGo, Startpage and others, Searx is not an index and so it does not crawl the web seeking out and storing content like Google does. The primary difference between Searx and most of the other meta search engines however, is that Searx is capable of pulling results from many other indexes, including decentralized peer-to-peer indexes such as YaCy, as well as centralized indexes like Google Yahoo, Bing and many others, any of which you can enable or disable using the excellent preferences interface that Searx provides.

One easy way to add Searx to Firefox is to locate a hosted instance which you like and which is preferably close to you geographically, and then from the Firefox search bar, simply click the “add” menu item. While searx.me is the original instance of Searx as provided by the developers, it is best not to use it because it can become overloaded or disabled. The Searx developers cannot afford to have too many people using their particular instance without your help and so they will disable it at times in order to promote the many other third party instances. That said, a potential pitfall of using a third party Searx instance is that the server may be logging traffic, such as IP addresses, countries of origin, etc., so you’ll have to decide whether you want to trust them.

Most other search engines can be added to Firefox in the same way as described above, but there are other methods also. The Mycroft Project hosts tens of thousands of preconfigured search engine plugins for a variety of web browsers and any one can be added by simply clicking on its name, the top 100 of which are listed here. They also have a form for writing your own search plugins. Although it is not possible to review the code from the main listing of search plugins, you can use their submission form to do so by mousing over the plugin name to reveal its numeric ID and then filling in that ID in their submission form page.

Another easy way to add a custom search engine to Firefox is with the Add custom search engine add-on by Tom Schuster. This add-on allows more control over the above methods, including the ability to define the icon path or base64 code (a binary-to-text encoding scheme that encodes the image in text form). A great on-line resource for converting an icon to base64 code is the Base64 Encoder utility which can accept the icon URL or file you upload.

Yet another way to add search engines is with the mozlz4-edit Firefox add-on by ‘serj_kzv’. This slick add-on allows you to edit the search.json.mozlz4 search plugin file directly from within Firefox. It is in this file that Firefox stores the code for all of the search engine plugins. The add-on works for both the newer compressed version of the file with the *.mozlz4 extension, as well as the older, uncompressed version (search.json). Regardless of how you add search plugins, the mozlz4-edit add-on is a handy tool to have for editing the search.json.mozlz4 file because you can use it to decompress, edit, sanitize, recompress and then save it, overwriting the old one. See the Sanitizing the default search engine plugins section below before you do this though.

Sanitizing the default search engine plugins

If you would rather avoid the hassle of sanitizing the default Firefox search engine plugins, see the Pre-sanitized search plugins section below.

Sanitizing manually

If you choose to use the default search engine plugins provided by Mozilla, you may want to sanitize them in order to circumvent some risks to your privacy, however you should be aware that doing so will not prevent tracking or other privacy risks when using the default plugins. If you are going to use the default search engines anyway, then you should also use something like the Neat URL add-on which at least strips the tracking parameters from the links returned by the search engine. You should also disable JavaScript for the search engine web page if possible.

The first thing to be aware of before you start hacking is that you will need to create copies of the default search plugins and sanitize the copies, else Firefox will will delete all your search plugins and restore only the default ones. It will do the same if there’s a syntax error as well, so be careful. If you don’t want to see or use the default ones, just disable them in the search preferences of Firefox.

Before we begin, backup your existing search.json.mozlz4 file if you have added any search plugins to Firefox.

To edit the search.json.mozlz4 file you first need to decompress it. There’s at least a few utilities available that will do this, but i would suggest using the mozlz4-edit Firefox add-on by ‘serj_kzv’ since it is very easy to use and it provides a basic code editor with syntax highlighting. Simply click the ‘mozlz4-edit’ toolbar button to load the add-on. Next, click the ‘Open file’ button and navigate to your Firefox profile folder and select the search.json.mozlz4 file. In the following example we will sanitize the Google search plugin which should give you an idea of what to look for if you decide to sanitize the other default search plugins. As of Firefox version 62, here’s what the default code for the Google search plugin looks like, though without the lengthy base64 icon string which i removed for brevity:

{
    "_name": "Google",
    "_shortName": "google-2018",
    "_loadPath": "jar:[app]/omni.ja!/google-2018.xml",
    "description": "Google Search",
    "__searchForm": null,
    "_iconURL": "[base64 icon code removed]",
    "_metaData": {
        "order": 5
    },
    "_urls": [
        {
            "template": "https://www.google.com/complete/search?client=firefox&q={searchTerms}",
            "rels": [],
            "resultDomain": "www.google.com",
            "type": "application/x-suggestions+json",
            "params": []
        },
        {
            "template": "https://www.google.com/search",
            "rels": [
                "searchform"
            ],
            "resultDomain": "www.google.com",
            "params": [
                {
                    "name": "q",
                    "value": "{searchTerms}"
                },
                {
                    "name": "ie",
                    "value": "utf-8"
                },
                {
                    "name": "oe",
                    "value": "utf-8"
                },
                {
                    "name": "client",
                    "value": "firefox-b-1-ab",
                    "purpose": "keyword"
                },
                {
                    "name": "client",
                    "value": "firefox-b-1",
                    "purpose": "searchbar"
                }
            ]
        }
    ],
    "queryCharset": "UTF-8"
},

In the above code you will notice the string firefox is used several times. This is one of the ways that Google knows you’re using Firefox and also it is how Mozilla gets paid when you use the Google search plugin, though it may not be the only way. To sanitize the code, we simply want to remove any mention of firefox, but we first need to create a copy of the plugin, else Firefox will restore the default plugins as previously mentioned. To create a copy, highlight the entire Google block of code beginning with the opening bracket ( { ) and ending with the closing bracket and comma ( }, ). Note that you must eliminate the comma if you paste the copy as the last one in the "engines": section. You will also need to add a comma after the closing bracket for the plugin code block above your copy if that code was the last one in the "engines": section. In this case the default Google plugin code was not the last search plugin, however we will add our copy after the last plugin code block, so we will need to add that comma to the closing bracket of the plugin code block above ours.

After removing the parameters which identify Firefox as our browser, here’s what our sanitized copy of the Google plugin looks like:

{
    "_name": "[s] Google",
    "_shortName": "google-2018",
    "_loadPath": "jar:[app]/omni.ja!/google-2018.xml",
    "description": "Google Search",
    "__searchForm": null,
    "_iconURL": "[base64 icon code removed]",
    "_metaData": {
        "order": 5
    },
    "_urls": [
        {
            "template": "https://www.google.com/complete/search?q={searchTerms}",
            "rels": [],
            "resultDomain": "www.google.com",
            "type": "application/x-suggestions+json",
            "params": []
        },
        {
            "template": "https://www.google.com/search",
            "rels": [
                "searchform"
            ],
            "resultDomain": "www.google.com",
            "params": [
                {
                    "name": "q",
                    "value": "{searchTerms}"
                },
                {
                    "name": "ie",
                    "value": "utf-8"
                },
                {
                    "name": "oe",
                    "value": "utf-8"
                }
            ]
        }
    ],
    "queryCharset": "UTF-8"
}

You can simply copy the above code and paste it as the last search plugin as described earlier, just be careful to add a comma to the last closing bracket of the search plugin above it as also described.

Here are the changes we made:

This…

    "_name": "Google",

became this…

    "_name": "[s] Google",

There’s two reasons for the above change, 1) you can’t have two search plugins with the same name and 2) prefixing Google with the [s] let’s us know that this is the sanitized version of the Google search plugin.

Next, this…

"template": "https://www.google.com/complete/search?client=firefox&q={searchTerms}",

became this, where we removed the client parameter…

"template": "https://www.google.com/complete/search?q={searchTerms}",

and this…

                },
                {
                    "name": "client",
                    "value": "firefox-b-1-ab",
                    "purpose": "keyword"
                },
                {
                    "name": "client",
                    "value": "firefox-b-1",
                    "purpose": "searchbar"
                },

was removed entirely to become this…

                }

Notice that we needed to remove the comma after the last closing }of the parameter code block since it is now the last block of code in the "params": section.

And finally, the last closing bracket for the Google plugin code block which looked like this…

},

had the trailing comma removed since we pasted the new Google plugin code block at the end of the "engine": section.

Sanitizing the remaining search plugins is accomplished in a similar way as above; you want to look for and remove any instances of ‘firefox’, or ‘mozilla’, or sometimes just ‘moz’ or ‘ff’. Once you’ve sanitized the default search plugins, just use the ‘mozlz4-edit’ add-on to save your changes as a ‘mozlz4’ file, overwriting your existing search.json.mozlz4 file.

Download pre-sanitized search plugins

If you do not want to sanitize the default search engine plugins yourself, you can download my pre-sanitized copy which contains a search.json.mozlz4 file that should work for Firefox version 57 and up (“up” meaning until the next time the M@M’s (Morons@Mozilla) decide to break everything again). The download contains the default engines which come with Firefox version 62, plus the sanitized versions of them, plus all of the engines i personally use. All in all there’s over 40 search engine plugins which you can edit or disable as you see fit. Many are disabled since i only use them occasionally, so be sure to adjust as necessary in your Firefox preferences.

Download: ff-search-plugs.zip

Install: Backup your existing search.json.mozlz4 file, then extract the archive and copy search.json.mozlz4 file to your Firefox profile directory and restart Firefox.

Sanitizing the prefs.js search engine preferences

Another item you should check is whether prefs.js in your Firefox profile directory contains any browser.search.param preferences. To sanitize these search engine preferences, open the about:config URL in your browser and enter browser.search.param in the search field. If none are found, great, but at the time i originally wrote this article there were two preferences found; browser.search.param.yahoo-fr and browser.search.param.yahoo-fr-ja. The default values may be different in your case, but in mine they were data:text/plain,browser.search.param.yahoo-fr=linuxmint and an empty string, respectively. What you need to do is create a custom user.js file to store your modified preferences if you don’t already have one, then copy the following code to your user.js:

user_pref("browser.search.param.yahoo-fr", ""); // sanitize Yahoo
user_pref("browser.search.param.yahoo-fr-ja", ""); // sanitize Yahoo

Removing the ‘Follow On Search’ system add-on

Mozilla packages some system add-ons with Firefox, installs them without your permission and doesn’t provide the user with any convenient means to remove or disable them. These system add-ons aren’t listed in the Extensions section of the Preferences UI. One of these system add-ons is called “Follow-on Search” (also see the Mozilla GitHub repository page) and it was used to collect data about the way people use the Google, Bing and Yahoo search engines.

The Follow On Search add-on, which has the file name followonsearch@mozilla.com.xpi, is located at /usr/lib/firefox/browser/features on Linux and \Program Files (x86)\Firefox\browser\features or \Program Files\Firefox\browser\features on Windows. To remove these system add-ons and stop them from being reinstalled when Firefox is updated, see the ‘System add-ons‘ section of the Firefox Configuration Guide for Privacy Freaks and Performance Buffs.

We’ve only scratched the surface…

Sanitizing the default Firefox search engine plugins is a good start, but there is much more to do if you’re interested in circumventing the risks to your digital privacy that are inherent in any popular web browser. For further information see my article, Firefox Configuration Guide for Privacy Freaks and Performance Buffs. You may find it helpful even if you aren’t using Firefox.

Resources

Special mention goes to ‘Thorin-Oakenpants’ (aka ‘Pants’) as well as the ‘ghacks’ crew and their GitHub repository where they host an excellent privacy and security centric user.js for Firefox and its derivatives, as well as an extensive Wiki which is full of valuable information.

Document revision history

Click to expand...

15-Sep-2017

  • first publish

16-Sep-2017

  • added this change log
  • corrected an error in the pre-sanitized Wikipedia search plugin and re-uploaded sanitized_search_plugs.zip
  • added information as suggested by ‘Pants’ in his comment below, particularly details and resources regarding the followonsearch@mozilla.com.xpi system add-on in a new section titled “Removing the ‘Follow On Search’ system add-on
  • added Hulbee and MetaGer to the search engine list
  • added a “Decentralized” column to the search engine table
  • added resource: 5 Best Search Engines That Respect Your Privacy – BestVPN.com
  • misc. cleanup and edits

17-Sep-2017

  • corrected typo in metager URL
  • added “Requires JS / Cookies” column in search engine table
  • changed links for search engines in table to point to company/about page and added links to point to search page
  • added link to the ‘lite’ version of DDG
  • added a link to the uBO filters to block Startpage/Ixquick tracking images
  • misc. minor edits

18-Sep-2017

  • added “Client Required” column to search engine table
  • corrected some info regarding the search engines in the table
  • minor misc. edits

24-Sep-2017

  • added a link to the Duck Duck Go: Illusion of Privacy article
  • added findx to the search engine list
  • minor edits

27-Sep-2017

  • added Qwant to the search engine table

29-Sep-2017

  • misc. edits and added info, nothing really important

3-Oct-2017

  • very minor edits

23-Oct-2017

  • moved the list of alternative search engines to it own page
  • minor edits

5-Dec-2017

  • minor change to the section ‘Sanitizing the default search engine plugins’ thanks to commenter ‘nohamelin’ – more changes coming shortly thanks to this persons comments

23-Dec-2017

  • updated search plugin import/export instructions as per the very helpful comment left by ‘nohamelin’, the developer of the XML Search Engines Exporter/Importer add-on in which he made available Scratchpad scripts that work with FF v57+
  • corrected an error in the pre-sanitized search engine archive, added Startpage and re-uploaded a new archive
  • misc. minor edits

28-Jan-2018

  • polishing

2-Oct-2018

  • major changes, additions and deletions

3-Oct-2018

  • fixed corrupted download files
  • added info about Add custom search engine add-on
  • added better instructions for installing the search plugin file, search.json.mozlz4
  • minor edits

21-Oct-2018

  • rewrote the section on manually sanitizing search plugins
  • various minor edits

15-Nov-2018

  • updated the search.json.mozlz4 file
  • spelling corrections

27-Nov-2018

  • updated the search.json.mozlz4 file
  • minor edits

11-Dec-2018

  • referred to my Firefox configuration guide for info on removing system add-ons

13 thoughts on “Firefox Search Engine Cautions, Recommendations”

  1. I realize that this page is somewhat dated, but Linux Mint limits default installed search addons in Firefox to those who have supported their distro’s development, but grudgingly gives a link to a somewhat larger set. I tried to use Scratchpad to install the sanitized search engines downloaded from the included link, but Scratchpad looks for HTML files and doesn’t recognize the sanitized search addons, even if the zip file is unpacked. Maybe I am doing something wrong, but I followed the instructions to the letter, and tried several times. Maybe it doesn’t work in Linux?

    1. I realize that this page is somewhat dated

      actually i updated it today and yesterday

      you don’t have to install the search.json.mozlz4 file – just unpack the .zip and copy the file to your profile directory – if you downloaded the file yesterday, download it again – i changed them

      let me know if you have further problems and, in the mean time, i’ll update the guide with better instructions

  2. (XML Search Engines Exporter/Importer developer here)

    Nice text. I want to add that the list of search plugins included by default in Fx is highly dependent of the build installed: that given list is for the en-US build, I think, and it changes for other locales, according to regional deals done by Mozilla, between other things.

    Now, given that no WebExtensions add-ons interacting with your search engines can be written yet, that leaves us for Fx57+:

    * First, you can get easily the original XML opensearch definition of all the engines included by default navigating to the “resource://search-plugins/” uri. From here, you can inspect them and save them to disk for manual sanitizing.
    * I adapted from the add-on some scripts to import/export engines via Scratchpad; it’s a bit awkward to use, but it should help:
    https://gist.github.com/nohamelin/8e2e1b50dc7d97044992ae981487c6ec
    https://gist.github.com/nohamelin/6af8907ca2dd90a9c870629c396c9521

    Also, be aware that Firefox dropped recently the support for adding engines from the searchplugins/ folder after deleting search.jzon.mozlz4; it will aply since Fx58:
    https://bugzilla.mozilla.org/show_bug.cgi?id=1405670

      1. I got the message, that it couldn’t be removed as the file/ filecatalog did not exist. english is not my first language, so i’ve translated the message. tech terms may not be correct. I’m no savvy myself, so I’m only familiar with the most common terminal commands. I do attend a local community. they might know about alternatives. yet, I’m the only one pro privacy. off subject: this is a great blog. very user friendly and in depth. thank you @pants for linking at ghacks

  3. Excellent article 12bytes. Really well done mate :)

    Some items for thought:
    – users should note some engines will require stripping tracking of search *results* (eg utms on google)
    – Firefox has a system add-on called Follow On Search, you should kill that (see ghacks user.js)
    – XML Search Engines Exporter/Importer is not Web Extension (yet?) so for FF57+ users, I guess we can use a portable legacy FF to create the file(s)?

    1. Also remember that in a lot of cases it is better in the first place to use a site specific search engine as well. For example, I have added an iTunes search engine so no-one but iTunes knows I search for Taylor Swift .. #GoTayTay :)

    2. thanks for your input Pants! much appreciated – i updated the article to address this stuff and added a section “Removing the ‘Follow On Search’ system add-on”

      Q: do you know how these system add-ons are loaded – are they loaded dynamically each time FF starts like other add-ons, or are they actually installed/cached some place? i’m wanting to be sure that deleting them is sufficient

      1. System add-ons are listed under about:support>”Firefox Features” – they act just like normal extensions AFAIK (except no disable/uninstall options). Delete the xpi and restart FF and you’re good to go.

        System add-ons are packaged with each Firefox application update (full updates at least). I use portable FF, so I am not sure about installed version behavior re dot releases. If I try to update and I have deleted some system add-on xpi files, the update fails and FF prompts for a full package instead (although the last update from 55.0->55.0.3 didn’t? Can’t remember! Dot releases may vary.). Anyway, I keep an eye on my system add-ons directory, and I only do updates now by downloading the portableapps.com packages (yes they include all the system add-ons too – both 32+64bit app/dirs)

        One of the reasons system add-ons exist, is so that patches/changes can be pushed without an app update. But the update check and update settings can have an effect. See: https://github.com/ghacksuserjs/ghacks-user.js/issues/172 . The ghacks user.js checks for app updates but lets you decide when to apply them – so until this bug is resolved, you won’t get system add-ons re-added or updated without your knowledge.

        You could also try 0505: “extensions.systemAddon.update.url” – if the system doesn’t know where to go, what can it do :) The only reason this is inactive in the js, is because this mechanism was initially designed to PUSH fixes – I think it was spurred on by a critical vulnerability in pdfjs a few years ago (Yes, pdfjs is a system add-on, but seems to be a special case compared to how/where it is stored)

        1. thanks for the clarification

          in my case, with Linux, no system add-ons were displayed in about:support and, actually, there wasn’t even a section called ‘Firefox Features’ – the ‘follow-on search’ add-on wasn’t present either, though others were

Leave a Reply