Using HttpWebRequest to perform HTTPS post fails with strange error message

Posted on August 13th, 2008 in .NET, Vista, Windows Server 2008 by Tom

Recently, after upgrading a server to Server 2008, some developers (ok, about 15 developers and BAs) began complaining that a post to a 3rd party vendor was no longer functioning. One of the devs whipped up a winform app to test from the server and locally from his workstation. From his Windows XP workstation, it was fine. From the Server 2008 box (and from my Vista laptop) it failed to connect with:

The underlying connection was closed: An unexpected error occurred on a send.

Descriptive.

A full stack trace revealed:

System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
— End of inner exception stack trace —
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Threading.ExecutionContext.runTryCode(Object userData)
at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Write(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.ConnectStream.WriteHeaders(Boolean async)
The underlying connection was closed: An unexpected error occurred on a send.
at System.Net.HttpWebRequest.GetRequestStream()

Keep in mind that this worked fine on XP and 2003. Vista and 2008 always threw that exception…without exception. The code was just doing a basic XML post to an HTTPS service with authentication enabled. 


ASCIIEncoding ascii = new ASCIIEncoding();
string requestToSend = body;
byte[] data = ascii.GetBytes(requestToSend);
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(destination);

webRequest.Credentials = new NetworkCredential("User", "Pass");
webRequest.Method = "POST";
webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2";
webRequest.ContentType = "text/xml";
webRequest.ContentLength = data.Length;
webRequest.KeepAlive = false;                                              

//Throws an exception HERE
Stream outStream = webRequest.GetRequestStream();
outStream.Write(data, 0, data.Length);
outStream.Close();

From that stack trace, I could see that the remote server was closing the connection…but, I had no idea why. Something in how it made the request was different than XP or 2003.

ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;

As it turns out, the defult behavior in Vista and Server 2008 is to use TLS first for secure connections. If the server doesn’t support TLS, it’s supposed to negotiate with the client to use SSL3. In this case, the remote server wasn’t negotiating at all…It was just dropping the connection.

http://blogs.msdn.com/wndp/archive/2006/04/12/tls_enabled_by_default.aspx

Long story short:

If you upgrade to Server 2008 or Vista, and your HTTPS XML POSTs are failing due to some strange error, try to force SSL3.

How to Remove/Delete Link Scanner from AVG-Free

Posted on June 17th, 2008 in Miscellaneous Software, Uncategorized by Tom

I like AVG Free. It’s quiet, not a resource hog, and FREE. I don’t like the Link Scanner in 8.0. Removing it caused AVG to tell me that it had issues all of the time. So, here’s how to remove Link Scanner from AVG Free, and not have it complain about an issue.

Quick answer: Run the installer with these flags:
/REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch

Detailed Answer:
This works even if AVG is already installed (in my experience).

1) Open a command prompt (Start->Run->CMD.exe is my preferred method).
2) Navigate to the directory where you’ve downloaded AVG to. Let’s say it’s on your desktop.
3) Type CD DESKTOP. This assumes that your default directory is your profile directory.
If not, you’ll need to type cd \Users\Tom\Desktop (Vista) or cd \Documents and Settings\Tom (XP)
4) Type (or copy/paste): avg_free_stf_*.exe /REMOVE_FEATURE fea_AVG_SafeSurf /REMOVE_FEATURE fea_AVG_SafeSearch
5) Run through the installer (choose repair/reinstall when it asks).

When it’s done, Link Scanner should be gone.

Source: HWDot

All is well now.

How to export your IIS7 config from one server and import into another

Posted on May 27th, 2008 in IIS7, Microsoft, Windows Server 2008 by Tom

I had copied the IIS7 config files from an already-configured server to a new server I was building. The two servers were going to be load balanced (non-NLB). After overwriting the config files on the new server with those that I had exported from the old server, I discovered that my app pools kept crashing. I attempted to reset the domain account credentials on the app pools, but found myself getting:

hresult:0×80090005, Message: Failed to commit configuration. Bad Data.

Built-in account works, domain accounts did not. Given that I’m much too lazy to re-configure all of the application pool IDs, I began looking for a way to gracefully import settings from the other server. Turns out, there isn’t an “import” button, so to speak. I did find that using “Shared configuration” I could essentially accomplish an import.

In IIS manager, you need to export the config from the already-configured server. In IIS manager, click the Server node, and go to Shared Configuration under Management.

IIS Shared Configuration

Click “Export Configuration”

Export Configuration

Enter the path you’d like to export the config to, and set an encryption key password:

Export Configuration Settings

Copy administration.config, applicationHost.config, and configEncKey.key to your new server to a temp location.

On the new server, go back to the “Shared Configuration” section and check “Enable shared configuration.” Enter the location in physical path and click “Apply.” It should prompt for the encryption password that you had set. Enter it, and reset IIS.

Import IIS Config

After resetting IIS, go back to Shared Configuration and uncheck “Enable shared configuration.” Click apply. You should see this:

Shared Configuration Warning

Click YES and it will import all of the settings from your Shared Config into the local config on your new IIS server.

At this point, all you should need to do is change your server-specific site bindings, and it should be good to go.

Follow up - BluRay Wins! Who cares?

Posted on February 21st, 2008 in Uncategorized by Tom

Saw this link in an RSS feed today. Seems like I’m on the right track :)

http://news.yahoo.com/s/nf/20080220/bs_nf/58454;_ylt=Ai7J7Du6zVCp6.KMVWrPaIcjtBAF

BluRay Wins! Who cares?

Posted on February 20th, 2008 in Home Electronics by Tom


It finally happened. HD-DVD has had its last few coffin nails nail-gunned into place. But what does it mean? More BluRay sales? Maybe for the next year or so, but with cable operators on the verge of delivering 100mbit+ to your door, does it matter?

The future of SD and HD content isn’t via BluRay, HD-DVD, or whatever else comes out. It’s going to be delivered via on-demand services like Netflix Watch-Now, Comcast On-Demand, and other services. Apple is heading the right way with Apple TV, but they’ve still got it wrong. Nobody wants ANOTHER box for the TV. They want to use what’s there (ie, their cable box, Tivo, Media Center PC, etc). There was a mini-debate going back and forth about this with my team via email, and I pointed out that it’s just going to take one piece of hardware for this “breakthrough.” You could almost compare it to the iPod. Apple had a good, easy to use, all-in-one (iTunes + iPod) solution for the user. They’re close with AppleTV, but not quite. HTPCs just aren’t as main stream…but, there is a box that millions of people have that does have internet access, and can have features added. The XBox 360.

After going back and forth this morning about the future of HD content, and me saying “All it will take is one consumer friendly device when the time is right,” I see this article at MSNBC come across my RSS feeds. Maybe this explains why MS didn’t fight harder for HD-DVD…they knew that it didn’t really matter in the long run.

After seeing the Netflix plug-in for Media Center by Anthony Parks (available here, I almost signed up for a Netflix account. I just hate renting DVDs. I never want to drive to the video store, shell out $5, worry about returning it on time, watching it, etc. When I’m in the mood for a movie, I usually want to watch it then. Throw some effort into it, and I’ll go play video games instead. It’s easier. If, from my Vista Media Center, I can navigate to my Netflix Watch-Now menu, pick a movie, and start watching, I’d definitely watch more movies. Hell, I’d probably have watched all of those movies that seemingly everybody on the planet has seen except for me by now (Indiana Jones for starters).

There are rumor floating around about Microsoft adding BluRay to the XBox 360 now that HD-DVD is dead. The fact that it was never integrated into the XBox is just evidence that they were never totally committed to the format. Now that it’s dead, we’ll probably see BluRay, which only makes sense because currently the best BluRay player out there is the PS3…why lose customers to another console because they just needed a BRD player? Then again, if MS would have integrated HD-DVD into the XBox in the first place (well, with the Elite), HD-DVD would have won the war…Guess we’ll never know now.

Importing product descriptions into ZenCart using PowerShell, Excel, and phpmyadmin - Part 2

Posted on January 17th, 2008 in E-Commerce, PowerShell, Zen Cart by Tom

If you followed the last post, this one is a continuation.

So, you’ve imported all of your products into the products table with the last script. Next, you need to add the information about the products to products_to_categories, so the categories are properly mapped, and to products_descriptions, so you have a name, etc.

 You’ll need the products_id of the FIRST product you inserted with the last script.

Check out this PowerShell script: More…

Importing product catalogs into ZenCart using PowerShell, Excel, and phpmyadmin - Part 1

Posted on January 17th, 2008 in PowerShell by Tom

I’ve been busy tonight working on ChaosCollectibles, trying to get all of the products added. Since the first thing I did was grab all of the images, I was done “scanning.” Next, I needed to add them all to the site.

Dave had spent about 5 hours in total adding around 100 cards. We still had 140 to go, and let’s face it…I’m lazy. Well, not lazy. I just prefer scripting and saving time where I can. It’s like the old saying around work: “Work Smart.”

 Since I had a standard naming convention for the image files, I was able to use those as a basis for my import. My hosting provider give me SQL access through PHPMyAdmin, so I can run SQL scripts…I just needed some SQL queries and stuff to run.

Product information is stored in 3 tables in the database for ZenCart. These tables are “products”, “products_description”, and “product_to_categories.” Products has a TON of columns. The others have 5 and 2, respectively. Now, as I was saying, I had a standard image naming convention. That convention was CTDOP_cardname.jpg.

 Based on that, I could do some image manipulation, and fill out some basic data I needed for the product table. The products has a column called products_id that is auto-incremented. This column is referenced by the other two tables. To fill out the products table with the card-specific information, I needed the following:

Product name, image location, date added, model name, and a few other fields that I could have made static, but chose to use variables for. Since I used that handy naming convention above, I was able to manipulate the names of every image file into something I could insert into the database to keep with the other conventions. For example, my model name convention is CTDOPcardname (no underscores). My image location was…the name of the file. That’s easy enough. Then I had to format some dates, and output my SQL statement. So, I’m tired of blabbing. Here it is.

Edit: OK, I was having issues with my code formatting. I really need a new, code-friendly template. I’m open to suggestions.

Here’s the script: Add-Product.ps1 - PowerShell script to generate ZenCart products

So, you’ll be left with a long list of SQL statements. But, we’ve still got the other two tables. Import that list anyway. Make a note of the FIRST products_id that gets imported, after the import completes.

The next step involves checking that table you just imported all of those products into. Since products_id auto-increments, you need to see where you started, and where you finished and make a note of those numbers.

I’m going to end this one right here and add a second post. This is getting too long, I think. Check here.

www.chaoscollectibles.com and PowerShell system.net.webclient Script

Posted on January 12th, 2008 in PowerShell, Vista by Tom

I finally did it. I got the e-commerce bug and started my own shop. I’m working with Dave K on it, and we’re progressing quite well with a lot of mentoring from Bill and all of his XekoShop experience.

So, PhishThis will contain a lot of posts related to how I’m making like easier for myself as far as batch watermarking, getting images, etc.

Here’s one:

I need to scan all of my cards for Chaotic. But they have them all online and could save me some trouble. However, I don’t feel like right-click-saving 250 times. Enter, you guessed it, PowerShell.

I need to create a web client to connect to the site.
$wc = new-object system.net.webclient
Now, the site requires a login. You might say “CRAP!” However, the webclient supports passing credentials, as long as they’re typed as system.net.networkcredential

So, I can login to the site, from PowerShell with:
$wc.credentials = new-object system.net.networkcredential("username", "password")

Now, there were around 250 images, so I construct a simple for loop that uses $wc.downloadfile(”source”, “location”) and we end up with:

$wc = new-object system.net.webclient
$wc.credentials = new-object system.net.networkcredential("username", "password")
for ($i = 0; $i -lt 250;$i++)
{
  $wc.downloadfile("www.foobar.com/getimages.aspx?ImageID=$($i)", "C:\images\$($i).jpg")
}

This script saved me HOURS of scanning and stuff…and now, even if the card is out of stock, I still have the image. That makes sense in my head.  

After, I set my view for that folder to EXTRA LARGE icons. This shows a readable preview of the jpeg. I renamed the first one, and instead of hitting enter after each one, I hit tab, and it went to the next file to rename it. I didn’t know you could do that, and tried on a whim. Saved me many keystrokes.

Next, I’m going to need to add them all to Zen Cart. Since I’m lazy a script kiddy, I’m going to write a script that will look at all of the product images in a folder, and write my SQL queries based on them. Hopefully I’ll even get it to do all of my formatting, too. We’ll see. Look for that this weekend sometime.

System Center Operations Manager 2007 and BizTalk 2006 SQL Server Warning

Posted on January 2nd, 2008 in SQL Server by Tom

Well, finally a post after the chaos that was the holidays. I was on vacation for 12 days and accomplished…nothing. Except for a lot of Counterstrike. And Team Fortress 2.

If you’re running SCOM 2007 and BizTalk 2006 (and have the SQL agent jobs properly configured) you may notice a warning in SCOM that says there are long running jobs. Upon inspection, you’ll discover that a single job (ManageRefCountLog) has been running since the SQL Agent last started (or the minute immediately following). I ignored this for quite some time, leaving the alert ACK’d the whole time because I couldn’t figure out why the job was running forever…that, and everything was working fine. I finally got fed up with it today and did a little digging. The error looks like this:

There are long running jobs on SQL instance MSSQLSERVER on computer SERVER.FOO.BAR.COM. This may indicate an issue with one or more jobs.

The ManageRefCountLog job has only one step that runs a stored proc on BizTalkMsgBoxDB called bts_ManageMessageRefCountLog. If you look at the store procedure itself, you’ll see:

WHILE (1 = 1)
BEGIN

So, the job is supposed to run forever…or until 1 != 1. Whichever comes first.I never would have noticed this job running forever, if not for SCOM2007 + the SQL 2005 management pack. Now I just need to add an exception. I’ll do that tomorrow…

PowerShell Script for Remote Event Log Viewing

Posted on December 13th, 2007 in Microsoft, PowerShell, Server Management by Tom

I had an issue today where I needed to find the frequency of an error on some of my VMs. It seems like I get VMSCSI errors at the same time each which (which probably means high SAN activity, but I’m trying to nail everything down). Either way, I needed to check all of my event logs for EventID 11 and 15. It’s slow…and by slow I mean it took about 30 minutes to scan 10 or so VMs…but it works, and I was able to get a decent idea of the times I’m seeing these errors.

$servers = .\getservernames.ps1 Tom 
 
foreach ($server in $servers) 
{ 
     if ((get-wmiobject -computer $server win32_computersystem).manufacturer -eq "VMware, Inc.") 
     { 
         get-wmiobject -query 
            "select * from Win32_NTLogEvent where LogFile = 'System' AND EventCode = 11 
            OR EventCode = 15" | 
            foreach { add-content c:\temp\$server.log "$_.timegenerated - $_.eventcode" }
     }
}

Aaaand, it’ll return logs for each server, with a time stamp, the event ID, and nothing more. Quick, dirty, but took me 5 minutes and got the info I needed…30 minutes later :p There’s probably a better way. I’ll have to see what I can come up with.

Next Page »