Poxy proxies

by Matt 26. January 2007 11:09

This is one area of .net software that just feels like a step backwards. It's all so 1997. Every single piece of .net software I've got that needs to talk to the internet requires me to set up the proxy.

Every win32 program I use just works.

Now, I can't really blame the developers of these programs (too much). Microsoft didn't make things easy.

Win32 developers used the wininet libraries. These are the libraries that support Internet Explorer, and so have built in support for Internet Explorer's proxy settings, including auto detection and .pac scripts.

.net is not built on top of wininet. Not quite sure why. My best guess is that (amongst other things) most uses of http web requests in .net were expected to be web services, rather than web pages, so wininet's caching was an unnecessary overhead. Whatever the reason, the support for proxy settings isn't exactly as transparent as wininet.

When you created a web request in .net 1.1, the framework would create a proxy for you, based on the values in the config file. You could explicitly define all the properties, or you could use the useSystemDefault attribute to use Internet Explorer's proxy settings.

There were several problems with this approach. IIRC, if the configuration section wasn't in your app.config, you didn't get a proxy - at all (I could be wrong on this, so don't quote me). Secondly, if the IE settings changed, this proxy class (which was cached, by the way) didn't update, so you were stuffed until you restarted your app. Thirdly, automatic proxy settings, based on .pac files, were ignored. And finally, and more importantly for me, no credentials were assigned.

Yep. I'm behind an authenticating proxy. Sucks, doesn't it?

To fix this, you had to explicitly assign credentials to the proxy. If the developer thought about proxies at all, they'd usually forget about this bit. In fact, if the developer did think about proxies, then you were usually in more trouble. They would be more explicit about everything, and leave you nothing to override in the config files. They'd go so far as to expect usernames and passwords to be hard coded in config.

.net 2 improved things a lot. Because I've just spent a good portion of this morning tracing through this looking for a bug I'm going to spell out what happens:

When you create an instance of HttpWebRequest, a default proxy is created for you, based on config. By far your best bet is to accept this, do nothing else and move on.

There's plenty you can do with the config. You can specify everything explicitly, or you can set useSystemDefault=false and get an empty proxy (not too much use. It always treats a url as though it bypasses the proxy, but it does allow you to set credentials). You can even pass in a type that gets spun up via reflection for complete flexibility. Or, you can get a proxy based on IE's settings (useSystemDefault=true).

This last one is quite impressive. It now understands automatically detecting settings and .pac files. It also (finally) responds to changes. And it even respects IE's policies and gets the settings per user or per machine. Another thing to note is that the web proxy is created in the context of the original process, not that of any impersonated users.

The best bit is that it also assigns default credentials to WebProxy.Credentials if you set useDefaultCredentials="true". These default credentials are, effectively, your credentials, and are enough to get you past an authenticating proxy. Hooray!

The slight downside is that this value isn't set by default - I guess for security, you don't want people sending credentials without cause. You can always set it in app.config or machine.config.

That's a lot of flexibility, and all from config. Alternatively, you can set the proxy in code. There are some caveats, so the moral is - don't do this. Don't set a proxy explicitly. Just leave it alone and accept the defaults. They should be flexible enough for the majority of situations.

But if you must set them in code, let's have a look at those caveats. What are your options?

First, the GlobalProxySelection class (originally seen in .net 1.1) allows you to get an empty proxy or returns WebRequest.DefaultWebProxy.

WebRequest.DefaultWebProxy will return either the nice config based class, or a static instance of a proxy class that you set in code - that's right, you can override the config based class. If you do, make sure you get it right.

WebRequest.GetSystemWebProxy() will return the same proxy you'd get if you'd specified useSystemDefault=true in config. However, it short circuits the config file, and it doesn't get credentials. Use this only if you have to (you know, like someone has a gun to your head).

WebProxy.GetDefaultProxy() is deprecated. In .net 1.1 it used to get the IE proxy settings (freshly, each time, which works around the stale settings problem). In .net 2, it's been rerouted to get the proxy settings from IE, but it doesn't touch config and it doesn't get credentials. Please don't use this one.

If you want to set credentials on a proxy, you can specify an explicit set by passing username, domain and password into a new instance of NetworkCredential. Or you can use CredentialCache.DefaultCredentials (which will get you your own credentials). Then set the Credentials property on the WebProxy you've created/retrieved.

A big gotcha is setting WebRequest.UseDefaultCredentials or WebRequest.Credentials. If you've got a proxy assigned to the WebRequest (and by default, you will have) these fields are not used.

A somewhat sneakier gotcha is that the bypass list is processed slightly differently. If you have a bypass list of e.g. *.sticklebackplastic.com, you're saying that you want to bypass the proxy for all subdomains of sticklebackplastic.com. Unfortunately, if you're trying to hit a different port e.g. my.sticklebackplastic.com:8443, this will not match, even though it does in IE. You'll need to add a rule of *.sticklebackplastic.com:8443. This was the big bug that I was chasing today (well, that and the fact that no credentials were set in the config file).

So, there we have it. Short answer is - don't set your own proxy in code and set useDefaultCredentials="true" in the config if you need it.

Tags:

Comments (22) -

Tita
Tita
3/29/2007 7:40:01 AM #

RE: Poxy proxies

Nice article - helped me very much! But one question - can I take the proxy settings including user name and password from code (I can't use config file)

Reply

Matt Ellis
3/29/2007 4:41:48 PM #

RE: Poxy proxies

Hi. You can set the proxy settings in code. I've just re-read this post, and it wasn't clear in a few places, so I've updated it a little.

You've got quite a few options of getting a proxy from code. They're pretty much all listed above. Once you've got one (empty, based on a config file or just a new instance of WebProxy) you can change it by setting any property you see fit. To set the credentials, set the Credentials property. This will probably be an instance of NetworkCredential, or CredentialCache.DefaultCredentials.

Hope this helps!
Matt

Reply

sam
sam
7/5/2007 6:36:35 AM #

RE: Poxy proxies

proper english would help.

Reply

gilad
gilad
5/14/2008 11:16:59 AM #

RE: Poxy proxies

This really saved me. though, wasted 4 hours on this. why didn't MS set the damn thing to true in the first place.

Reply

Janie Koka
Janie Koka
3/24/2011 12:37:40 PM #

The great truth about this  blog article is, that it spoke to me greatly. Thank you for sharing your thoughts and concerns.

Reply

Houston Chiropractor
Houston Chiropractor
3/24/2011 2:57:33 PM #

The solid truth about this post is, that it spoke to me greatly. Thank you for sharing your thoughts and concerns.

Reply

Sam Thompson
Sam Thompson
7/13/2011 9:19:54 AM #

15. Just want to say your article is as surprising. The clearness in your post is just excellent and i can assume you're an expert on this subject. Well with your permission allow me to grab your feed to keep updated with forthcoming post. Thanks a million and please carry on the gratifying work.

Reply

suv reviews
suv reviews
7/20/2011 10:36:34 PM #

hi!,I like your writing so so much! proportion we communicate extra about your post on AOL? I require a specialist in this area to unravel my problem. May be that's you! Taking a look forward to see you.

Reply

best hybrid cars 2011
best hybrid cars 2011
7/22/2011 3:48:55 AM #

hi!,I like your writing so so much! proportion we communicate extra about your post on AOL? I require a specialist in this area to unravel my problem. May be that's you! Taking a look forward to see you.

Reply

best ipad games
best ipad games
7/24/2011 4:50:40 AM #

it ti permette di orientarti con semplicità nel pulviscolare niverso di Moncler.Troverai le indicazioni per lo spaccio o negozio Moncler più vicino a casa tua felpe moncler, pantaloni e abbigliamento sportivo.

Reply

LAURENCE  Debora
LAURENCE Debora
7/27/2011 10:48:22 AM #

Bienvenue sur annuaires-gratuit.com.Des fonctionnalités inédites, nombreuses annuaires.

Reply

Encinitas chiropractors
Encinitas chiropractors
8/13/2011 8:54:17 AM #

Super, informative post. Thanks for taking the time to share this.

Reply

Cincinnati Bengals Live Stream
Cincinnati Bengals Live Stream
8/28/2011 2:47:26 PM #

Hey i think your blog is good. i think it on Yahoo i think i will came back one day.  click this link  <A href="www.bengalslivestreaming.com">cincinnati bengals live stream </A>

Reply

Stephane
Stephane
10/22/2011 10:02:43 AM #

Hello

Interresting post here may come back soon
continue updating your blog

Reply

Stephane
Stephane
10/23/2011 11:58:30 AM #

Hi

Interresting post here may come back soon
keep update

Reply

Btissam
Btissam
10/23/2011 7:28:37 PM #

Hi

Nice, post here
keep update

Reply

Robert
Robert
10/23/2011 8:47:03 PM #

hi,

Nice, post here
continue updating your blog

Reply

Lisa
Lisa
10/23/2011 10:54:32 PM #

hi,

Interresting post here may come back soon
Keep like this

Reply

Eric
Eric
10/24/2011 12:23:02 AM #

Hi

interresting, post here will came back
Keep like this

Reply

Lisa
Lisa
10/24/2011 12:24:06 AM #

hello there,

Nice, post here will came back
continue updating your blog

Reply

iPhone games reviews
iPhone games reviews
11/17/2011 5:55:23 AM #

Good job  , Awesome Post !!!

Reply

Andrew Webb
Andrew Webb
11/17/2011 11:13:33 PM #

This is a great, great, great post.  The greatest ever on .NET, imo.  Thanks for your insight into this problem.  The solution is so simple, but after all these years the solution is also very little known.  And the implications of this are profound, in a bad way.  The amount of wasted time is staggering.

I have created an issue/idea with Microsoft to fix this in .NET going forward:-
visualstudio.uservoice.com/.../2397357-fix-it-so-that-net-apps-can-access-http-thru-auth

Please vote it up if you agree.

Thanks.

Reply

Pingbacks and trackbacks (2)+

Add comment

biuquote
  • Comment
  • Preview
Loading

Month List

RecentComments

Comment RSS