Advanced FireWatir

by Matt 20. October 2009 17:41

There’s no delicate way to put this, so I’m just going to have to go ahead and say it; at work, we write automated acceptances tests in a BDD style, using cucumber and Ruby, and using FireWatir to automate Firefox.

Nothing controversial there, it’s just not the most thrilling of opening sentences.

This all started with the need to send a custom HTTP header to a web page we were testing. Now, we could write that using Ruby’s Net::HTTP module, but that would require also writing stuff to manage logins and cookies, and frankly, I’d rather let Firefox handle all that. It just needs to be convinced to send that extra header.

Now, FireWatir has a very interesting implementation. It uses an extension for Firefox called JSSh - a JavaScript Shell (which seems rather tricky to download – try the FireWatir download site). This extension starts listening on a socket, to which you can send raw JavaScript, which JSSh evaluates and returns the output. It exposes a fairly simple API that allows you to e.g. enumerate open windows, navigate to web pages, and access the DOM. FireWatir exercises this quite extensively.

Firefox really likes JavaScript. After all, most extensions are written in JavaScript.

And that right there is the key.

Firefox extensions are written in JavaScript, and they can do *anything* with the browser. So, I just have to figure out how an extension would add a header to a request. One quick Google later, and I’ve found nsIWebNavigation.

Firefox is written to be astonishingly componentised. It is implement on a platform called XPCOM (cross-platform COM, very much like Microsoft’s COM) that exposes a huge amount of functionality. All of which is publicly accessible to those JavaScript extensions.

And nsIWebNavigation is an XPCOM interface all about navigating the web browser, and one method it exposes is loadURI(url, flags, referrer, postData, headers). This method allows me to add custom headers to a navigation request – exactly what I’m looking for. I now have two small problems – how do I get my hands on an instance of nsIWebNavigation, and how do I marshal up the data into the headers parameter?

The first question was answered through a bit of trial and error. If you telnet into JSSh (telnet localhost 9997) you can start investigating your surroundings. You’ve got an interactive shell from which you can run commands, such as getWindow(). But if you type getWindow without the parentheses, you get the JS listing of the function definition. One such example lead me to discover that the higher-level API that JSSh exposes for navigation simply calls into the webNavigation property of the browser variable. And this property is an instance of nsIWebNavigation.

The second problem was solved by Google. I need an instance of nsIInputStream to represent the headers. Turns out I can create an instance of nsIStringInputStream, set the data from a JS string and pass that to loadURI. All of which gives us the following JavaScript:

var headers = Components.classes[';1']
headers.setData("X-Forwarded-For:\r\n", 31);
#{BROWSER_VAR}.webNavigation.loadURI("http://...", 0, null, null, headers);

The first line looks like some heavy magic. I won’t confess to knowing exactly what’s going on here, but it’s safe to assume that it’s creating an instance of a named XPCOM class and returning the nsIStringInputStream interface implemented by that instance. The second line sets the data into the object, pushing in a string (the loadURI docs state that each header must be separated by a carriage return/line feed pair) and the length of the string. The headers variable is then passed to loadURI, and ta-da! the browser is now navigated to the given URL, and the custom header is sent.

Of course, that’s just the JavaScript. We need to be able to use this from Ruby. I opened up the Firewatir::Firefox class and added:

# Essentially a copy of goto(uri) but can pass headers through the request
# Pass a list of headers, e.g. [ "X-Forwarded-For:", "cheese: toast" ]
# (Note that we should also be able to post data through this mozilla method)
def goto_with_headers(url, headers)
  h = ""
  headers.each {|value| h += "#{value}\\r\\n" }
  # Load the given url.
  jssh_command = "var headers = Components"
jssh_command += ".classes[';1']"
jssh_command += ".createInstance(Components.interfaces.nsIStringInputStream);
" jssh_command += " headers.setData(\"#{h}\", #{h.length});" # first null is referrer, second is postData jssh_command += " #{BROWSER_VAR}.webNavigation.loadURI(\"#{url}\", 0, null,"
jssh_command += " null, headers);
" $jssh_socket.send("#{jssh_command}\n", 0) read_socket() wait() end

Note that the carriage return/line feed escape characters and the quotes in setData have been escaped – we’re writing Ruby that is going to be writing JavaScript. And as the comment says – call it with an array of headers that you’d like to pass to the server.

So there we have it. FireWatir, via JSSh, has a much larger API available to it, thanks to XPCOM. This opens the door to some very interesting possibilities.

I’m aware this is quite an exposition heavy post, so I’ll do an executive summary with a few more examples (would automating Firefox preferences be useful?) and then we’ll get really advanced.

Tags: , , ,

Comments (13) -

Martina Klamm
Martina Klamm
2/8/2011 3:46:06 AM #

Lese eueren Blog schon länger und wollte mal eine Lob aussprechen. Einfach Spitz macht weiter so! Lustig Grüsse Eure Michela


Sashwat Sharma
Sashwat Sharma
7/18/2011 7:58:19 PM #

will such http headers give problems while loading on IE6 or the older versions of Safari?


suv reviews
suv reviews
7/20/2011 10:25:25 PM #

Hi,what an excellent article this is,I found it on bing and I like it very much,I agree with what you have said, lots of things will be learned form your site,but I still have some questions with the last part,can you explain it for me ?I will appreciate your answer,and I will be back again!


7/23/2011 1:41:11 PM #

Nice article. Although I'm not that familiar with the topic your discussing, but I guess I learned something from your article. Come to think of many informative topics in the future. Good luck!


iphone os4
iphone os4
7/23/2011 8:44:32 PM #

Una pagina sarà dedicata agli accessori, una alle giacche e ai giubbotti. Troverai le indicazioni per lo spaccio o negozio Moncler più vicino a casa tua e tutte le offerte più vantaggiose di questo prestigioso marchio.


ipad features
ipad features
7/24/2011 4:50:09 AM #

hi!,I like your writing so so much! proportion we communicate extra about your post on AOL? I require a specialist in this area to unravel my problem. May be that's you! Taking a look forward to see you.


Dream Marriage
Dream Marriage
8/23/2011 11:47:14 PM #

You made a great point right there. I made a research on the topic and found most people will agree with your article.


9/8/2011 3:19:32 PM #

You got some nice lay-out out there. You also have a very nice content. No wonder you have so many visitors because you express your ideas effectively. Nice blog lay-out + nice blog content = Effective blog post! This blog is just perfect. Keep up the good work!


soul food suitland md
soul food suitland md
10/13/2011 8:10:12 AM #

Hello there! This is kind of off topic but I need some advice from an established blog. Is it hard to set up your own blog? I'm not very techincal but I can figure things out pretty fast. I'm thinking about creating my own but I'm not sure where to start. Do you have any tips or suggestions?  Appreciate it


text message marketing baltimore
text message marketing baltimore
11/14/2011 11:02:45 PM #

Hiya! I know this is kinda off topic nevertheless I'd figured I'd ask. Would you be interested in exchanging links or maybe guest writing a blog post or vice-versa? My website addresses a lot of the same subjects as yours and I think we could greatly benefit from each other. If you're interested feel free to shoot me an email. I look forward to hearing from you! Fantastic blog by the way!


photographe mariage paris
photographe mariage paris United States
6/8/2015 2:13:12 AM #

Thanks for this excellent article. One more thing to mention is that many digital cameras arrive equipped with some sort of zoom lens so that more or less of your scene to be included simply by 'zooming' in and out. These kinds of changes in focus length usually are reflected in the viewfinder and on huge display screen right at the back of this camera.


philix United States
11/19/2015 11:59:23 AM #

Good Web Site.


Philix United States
11/21/2015 3:01:13 AM #

wonderful web site.


Pingbacks and trackbacks (1)+

Add comment

  • Comment
  • Preview


Month List