Advanced FireWatir – cheat sheet

by Matt 20. October 2009 18:18

OK, so the last post was a bit heavy. Er, detailed. The upshot is that FireWatir, a Ruby library for automating Firefox via the JSSh (JavaScript Shell) extension can get access to the same APIs used by JavaScript Firefox extensions, allowing for some very low level automation of Firefox. The example in that post showed how to send arbitrary extra headers when navigating to a page:

# Essentially a copy of goto(uri) but can pass headers through the request
# Pass a list of headers, e.g. [ "X-Forwarded-For: 10.15.142.22", "cheese: toast" ]
# (Note that we should also be able to post data through this mozilla method)
def goto_with_headers(url, headers)
  #set_defaults()
  get_window_number()
  set_browser_document()
  h = ""
  headers.each {|value| h += "#{value}\\r\\n" }
  # Load the given url.
  jssh_command = "var headers ="
jssh_command += "Components.classes['@mozilla.org/io/string-input-stream;1']"
jssh_command += ".createInstance(Components.interfaces.nsIStringInputStream);
" jssh_command += " headers.setData(\"#{h}\", #{h.length});" # first null is referrer, second is postData jssh_command += " #{BROWSER_VAR}.webNavigation.loadURI(\"#{url}\", 0, null,”
jssh_command += "null, headers);
" $jssh_socket.send("#{jssh_command}\n", 0) read_socket() wait() end

I thought it might be useful to show another example, without having to explain the background. If you’re interested in details, read the last post.

Right. Let’s automate Firefox’s preferences. Again, searching for what extensions do, they use the nsIPrefBranch interface to get and set preferences. We’ll do the same.

Instead of using the createInstance method to get the interface, we’re going to use getService (presumably because the preferences service object is already in memory, and we want to get a handle on that, rather than a new instance). In Javascript:

var prefs = Components.classes[\"@mozilla.org/preferences-service;1\"]
    .getService(Components.interfaces.nsIPrefBranch);"
prefs.setBoolPref("app.update.enabled", true);

And the Ruby helper method from my version of FireWatir::Firefox (note the escaped quotes):

def set_bool_preference(key, value)
  jssh_command = "var prefs = Components"
jssh_command += ".classes[\"@mozilla.org/preferences-service;1\"]
" jssh_command += ".getService(Components.interfaces.nsIPrefBranch);" jssh_command += " prefs.setBoolPref(\"#{key}\", #{value});" $jssh_socket.send("#{jssh_command}\n", 0) read_socket() end

Other preferences are stored as strings and as comma separated string values:

def set_string_preference(key, value)
  jssh_command = "var prefs = Components"
jssh_command += ".classes[\"@mozilla.org/preferences-service;1\"]
" jssh_command += ".getService(Components.interfaces.nsIPrefBranch);" jssh_command += " prefs.setCharPref(\"#{key}\", \"#{value}\");" $jssh_socket.send("#{jssh_command}\n", 0) read_socket() end def add_string_preference(key, value) jssh_command = "var prefs = Components"
jssh_command += ".classes[\"@mozilla.org/preferences-service;1\"]
" jssh_command += ".getService(Components.interfaces.nsIPrefBranch);" jssh_command += " var value = prefs.getCharPref(\"#{key}\");" jssh_command += " if (value === '') { value = \"#{value}\"; } else {" jssh_command += " var values = value.split(\",\");" jssh_command += " if (values.indexOf(\"#{value}\") === -1) {"
jssh_command += " values.push(\"#{value}\");"
jssh_command += " }
" jssh_command += " value = values.join(\",\");" jssh_command += " }" jssh_command += " prefs.setCharPref(\"#{key}\", value);" $jssh_socket.send("#{jssh_command}\n", 0) read_socket() end

These can be used to make sure that your test environment’s version of Firefox is correctly setup before using. And if you’re wondering what to use for “key”, then take a browse through Firefox’s about:config (then Google it to see what it does). Here’s what I’m using at work:

browser.set_bool_preference("browser.download.manager.showWhenStarting", false)
# Convert "/" to "\". Double escaped because we're putting it into a JS string
browser.set_string_preference("browser.download.dir",
default_download_dir.gsub(/\//, '\\\\\\')) browser.add_string_preference("browser.helperApps.neverAsk.saveToDisk",
"application/zip") browser.add_string_preference("browser.helperApps.neverAsk.saveToDisk",
"application/postscript") browser.set_bool_preference("app.update.enabled", false) browser.set_bool_preference("browser.search.update", false) browser.set_bool_preference("extensions.update.enabled", false) browser.set_bool_preference("browser.sessionstore.resume_from_crash", false) browser.add_string_preference("network.automatic-ntlm-auth.trusted-uris",
"http://xxx.example.com") browser.add_string_preference("network.automatic-ntlm-auth.trusted-uris",
"http://yyy.examplecom") browser.add_string_preference("network.automatic-ntlm-auth.trusted-uris",
"http://zzz.example.com")

Most useful keys?

  • network.automatic-ntlm-auth.trusted-uris. With this, I can add a domain to automatically get NTLM authentication, so I know my tests won’t get stuck at an authentication prompt.
  • browser.helperApps.neverAsk.saveToDisk. I didn’t even know this one existed, but it’s great. This is a comma separated list of mime types that will always get saved to disk, so again, I know that I won’t get stuck at the confirm download prompt. (This is my favourite – previously, we’ve had to set these up by hand by actually downloading an instance of the file. I’m very happy to be automating this.)

Handy, eh?

One more post to go – let’s dive a bit deeper.

Tags: , , ,

Advanced FireWatir

by Matt 20. October 2009 17:41

There’s no delicate way to put this, so I’m just going to have to go ahead and say it; at work, we write automated acceptances tests in a BDD style, using cucumber and Ruby, and using FireWatir to automate Firefox.

Nothing controversial there, it’s just not the most thrilling of opening sentences.

This all started with the need to send a custom HTTP header to a web page we were testing. Now, we could write that using Ruby’s Net::HTTP module, but that would require also writing stuff to manage logins and cookies, and frankly, I’d rather let Firefox handle all that. It just needs to be convinced to send that extra header.

Now, FireWatir has a very interesting implementation. It uses an extension for Firefox called JSSh - a JavaScript Shell (which seems rather tricky to download – try the FireWatir download site). This extension starts listening on a socket, to which you can send raw JavaScript, which JSSh evaluates and returns the output. It exposes a fairly simple API that allows you to e.g. enumerate open windows, navigate to web pages, and access the DOM. FireWatir exercises this quite extensively.

Firefox really likes JavaScript. After all, most extensions are written in JavaScript.

And that right there is the key.

Firefox extensions are written in JavaScript, and they can do *anything* with the browser. So, I just have to figure out how an extension would add a header to a request. One quick Google later, and I’ve found nsIWebNavigation.

Firefox is written to be astonishingly componentised. It is implement on a platform called XPCOM (cross-platform COM, very much like Microsoft’s COM) that exposes a huge amount of functionality. All of which is publicly accessible to those JavaScript extensions.

And nsIWebNavigation is an XPCOM interface all about navigating the web browser, and one method it exposes is loadURI(url, flags, referrer, postData, headers). This method allows me to add custom headers to a navigation request – exactly what I’m looking for. I now have two small problems – how do I get my hands on an instance of nsIWebNavigation, and how do I marshal up the data into the headers parameter?

The first question was answered through a bit of trial and error. If you telnet into JSSh (telnet localhost 9997) you can start investigating your surroundings. You’ve got an interactive shell from which you can run commands, such as getWindow(). But if you type getWindow without the parentheses, you get the JS listing of the function definition. One such example lead me to discover that the higher-level API that JSSh exposes for navigation simply calls into the webNavigation property of the browser variable. And this property is an instance of nsIWebNavigation.

The second problem was solved by Google. I need an instance of nsIInputStream to represent the headers. Turns out I can create an instance of nsIStringInputStream, set the data from a JS string and pass that to loadURI. All of which gives us the following JavaScript:

var headers = Components.classes['@mozilla.org/io/string-input-stream;1']
    .createInstance(Components.interfaces.nsIStringInputStream);
headers.setData("X-Forwarded-For: 10.15.142.22\r\n", 31);
#{BROWSER_VAR}.webNavigation.loadURI("http://...", 0, null, null, headers);

The first line looks like some heavy magic. I won’t confess to knowing exactly what’s going on here, but it’s safe to assume that it’s creating an instance of a named XPCOM class and returning the nsIStringInputStream interface implemented by that instance. The second line sets the data into the object, pushing in a string (the loadURI docs state that each header must be separated by a carriage return/line feed pair) and the length of the string. The headers variable is then passed to loadURI, and ta-da! the browser is now navigated to the given URL, and the custom header is sent.

Of course, that’s just the JavaScript. We need to be able to use this from Ruby. I opened up the Firewatir::Firefox class and added:

# Essentially a copy of goto(uri) but can pass headers through the request
# Pass a list of headers, e.g. [ "X-Forwarded-For: 10.15.142.22", "cheese: toast" ]
# (Note that we should also be able to post data through this mozilla method)
def goto_with_headers(url, headers)
  #set_defaults()
  get_window_number()
  set_browser_document()
  h = ""
  headers.each {|value| h += "#{value}\\r\\n" }
  # Load the given url.
  jssh_command = "var headers = Components"
jssh_command += ".classes['@mozilla.org/io/string-input-stream;1']"
jssh_command += ".createInstance(Components.interfaces.nsIStringInputStream);
" jssh_command += " headers.setData(\"#{h}\", #{h.length});" # first null is referrer, second is postData jssh_command += " #{BROWSER_VAR}.webNavigation.loadURI(\"#{url}\", 0, null,"
jssh_command += " null, headers);
" $jssh_socket.send("#{jssh_command}\n", 0) read_socket() wait() end

Note that the carriage return/line feed escape characters and the quotes in setData have been escaped – we’re writing Ruby that is going to be writing JavaScript. And as the comment says – call it with an array of headers that you’d like to pass to the server.

So there we have it. FireWatir, via JSSh, has a much larger API available to it, thanks to XPCOM. This opens the door to some very interesting possibilities.

I’m aware this is quite an exposition heavy post, so I’ll do an executive summary with a few more examples (would automating Firefox preferences be useful?) and then we’ll get really advanced.

Tags: , , ,

Rel=Me

Month List

RecentComments

Comment RSS