2008-10-30

Is this file open?

Coding problem of the day: I have a service (written in C# .Net) that monitors a folder for incoming files, and when a file appears, it needs to process it. At the moment, it is using a FileSystemWatcher object. I don't know if I'll continue to use that or not (it was written by someone else before I got here), as it doesn't guarantee any sequence of events, nor does it help if files exist before the service is launched. But that's beside the point.

The problem I discovered is, if the file is coming from another computer over a slow link (e.g. an FTP or other slow network transfer), the FileSystemWatcher will raise its Created event as soon as the file appears, but the file is not yet ready. (I simulated this by writing another program that slowly writes a very large file to the target directory, using a loop and a Thread.Sleep.)

The solution, like a lot of other things in programming, is a little convoluted, but it seems to work for the time being. The gist of it is, in the Created event handler, I first call a function that tries to open the file for exclusive read access. If the file is still open, this will fail.

Testing for this failure is the hard part. The exception that gets thrown is a fairly generic IOException, and while a "file in use" is one condition for which I want to stop and wait, there are other conditions that I would quite definitely not want to wait to magically resolve themselves. The MSDN doc on IOException lists several derived classes that, for example, I would rather treat as critical errors immediately, like PathTooLongException, DirectoryNotFoundException, FileNotFoundException…. If I waited on those, I have a feeling my code would be waiting a very long time.

So, here's my "is this file locked" function:

using System.IO;
using System.Threading;

/// <summary>
/// Makes sure a file is closed before attempting to use it
/// </summary>
/// <param name="fullFilePath"></param>
public void WaitForFileClose(string fullFilePath) {
 while (FileIsLocked(fullFilePath)) {
  Thread.Sleep(new TimeSpan(0, 0, 15));
 }
}

/// <summary>
/// Determines if a file is still locked by attempting to open it for unshared (exclusive) read access.
/// If an IO Exception occurs that includes the text "another process" in the message (i.e. "in use by
/// another process"), the file is assumed to be locked.  Any other exceptions are rethrown.
/// </summary>
/// <param name="fullFilePath"></param>
/// <returns>
/// true if the specific "another process" exception was found trying to open the file, false
/// if no error occurred.
/// </returns>
/// <remarks>
/// May not work on systems in other languages.  There is no specific "file locked" exception to
/// test for, and there are other exceptions that derive from IOException (like FileNotFoundException)
/// that should not be waited on.
/// </remarks>
private bool FileIsLocked(string fullFilePath) {
 try {
  using (FileStream fs = new FileStream(fullFilePath, FileMode.Open, FileAccess.Read, FileShare.None)) {
   fs.Close();
   return false;
  }
 } catch (IOException ioex) {
  if (ioex.Message.Contains("another process"))
   return true;
  else
   throw;
 } catch {
  throw;
 }
}

As I note in the comments, I'm concerned this might not work on other locales, since I'm specifically looking in the exception's message text for the string "another process". Unfortunately, I don't have a better way to determine what IOException got thrown. I set a breakpoint and tested it, and (at least in .Net 2.0 on Windows XP SP3), it was indeed throwing a base System.IO.IOException with that text in the message.

I'm open to any better ideas, though….

2008-10-28

The DMCA, 10 years later

The EFF is "celebrating" the 10th anniversary of the Digital Millennium Copyright Act with a report of the unintended consequences its use and abuse has had on research, trade, and civil liberties. It's a worthy read, even if they do leave out one of my favorites — that Sharpies were suddenly circumvention devices because they could be used to defeat CD copy protection*. :D

*Ok, no legal action was ever taken against Sharpies, or even web sites for describing the process as far as I know; but the fact that this joke popped up around the internet more than once just shows how ridiculous the whole thing is, and how easily it was recognized as such back in 2002.)

2008-10-27

If the MPAA Did Handbags

Continuing the theme I seem to have picked up lately, in linking to random articles I find interesting, was this one I saw via Techdirt. It's from the "Pure Purse Passion" site BagBunch.com, and it's an article that claims to, in its own words, "[show] the bad ethics, hypocrisy and stupidity of the companies behind our TV Shows, our documentaries, our movies, and our music." In short, what would happen if the handbag industry created an "HIAA" and adopted the same practices as the MPAA and RIAA? clicky

Granted, the analogy isn't perfect (and they admit as much in their opening statements). For starters, music these days as a digital good can be copied infinitely, with zero degradation, for essentially zero cost; whereas a handbag as a physical good would still require materials and labor.

Despite the flaws, I think it's a worthwhile article. By applying the *AAs' "logic" to a physical (and non-technical) context, perhaps it'll help make more people more aware of what is going on, instead of just being iTunes sheep who unknowingly lock themselves into whatever draconian DRM rules Apple throws at them.

2008-10-23

Biden's Bungles: A Blatant Bias

Wow. When the New York Post starts to report on media bias (and even Dan Rather is commenting about it), you know there are problems.

2008-10-21

Would the Last Honest Reporter Please Turn On the Lights?

Hear, hear!

How dare you obey the rules of the road

According to the Colorado Driver Handbook, which I just downloaded from the DMV website for confirmation, section 10.1a, if a traffic signal is malfunctioning or not operating, the intersection should be treated as a four-way stop. If the lights are on but flashing yellow, it is a warning of hazard, to slow and proceed with caution (but not "treat as a four-way stop").

For some reason, people in Colorado don't seem to understand these. When they're not completely ignoring them, they get these rules completely backwards.

Last week, I was driving home in the middle of the day to shuttle my wife to the auto shop, as we were down to one car while the minivan was having some maintenance (an $800 repair of the climate control system — ouch), when I noticed a stoplight was completely out. The road I was on was a six-lane divided road, and the intersection was with a small, two-lane road. A pickup was waiting on the cross street.

I slowed to a stop. A truck in the lane next to me blowed through the intersection, and I honked at him — which earned me an irritated honk from the truck behind me. I waited patiently as three more cars blew through the opposite direction before the pickup finally got his turn to go through the intersection, which, as you'll note in my first paragraph above, is legally a four-way stop at this point.

The next street light is also in a "non-standard" mode, but this one is blinking yellow. (The street I'm on has reduced to a two-lane road by this point.) Wouldn't you know it, traffic coming the other way is actually stopping at this blinking yellow light. I slow a bit, to make sure the car waiting on the cross street at his blinking red light doesn't try to pull out in front of me, and cautiously proceed through the intersection.

After I pick up my wife, we head back on the same road. I notice people are now treating the dead light like a four-way stop, but I believe this is due to the police car that has stopped beside the intersection, as a cop is donning an orange vest to direct traffic.

Slow down, so I can cut you off!

My friend, the "Top Hat Rabbit", just made a blog post that reminded me of a driving incident. This happened many years ago, but I still remember it clearly.

I was driving home from work one day, and to avoid traffic, I cut through a neighborhood, taking a route I had taken many times before. At one point, I came around a curve, where a side street joins the main road I was on. A couple guys in a little red convertible were at the stop sign. The driver was talking to his passenger, not paying any attention, and started to pull out in front of me as I came by. To avoid getting hit, I hit the gas and swerve around him. My sunroof was open, so I heard him yell, "Slow down, a**hole!"

Of course, he couldn't let this go. He had to prove to his friend how big a man he was. Up at the street light exiting the neighborhood, I waited to turn left, and he pulled up beside me in the right turn lane, arm and middle finger extended. When I refused to so much as glance in his direction, he tried to get my attention. "Hey a**hole!" he called. Still no reaction. (Nice language to be shouting in a neighborhood, I thought.)

He inched his car forward, I suppose so he could get a look at my front license plate, which I guess he wrote down, because he then started waving a piece of paper in the air, calling, "A**hole!!!"

The light turned green. I finally looked over at him, blew him a kiss, and drove off.

Hope I didn't make his boyfriend jealous.

2008-10-07

SQL 2005 getting too smart?

We had a bug with one of our clients that had just upgraded from SQL Server 2000 to 2005. In the process, we were deploying some new application features to them, and one was crashing. This same feature was working just fine for other clients (and in-house, of course), so we were puzzled.

The problem is actually from something I used to wish SQL Server was smart enough to figure out on its own for a long time. Namely, when you write a SQL statement such as this:

Select U.UnitId, U.UnitName, UT.UnitTypeName
From Unit U 
Inner Join UnitType UT On U.UnitTypeId = UT.UnitTypeId
Order By UnitTypeId

you will get an error to the tune of "Ambiguous column name 'UnitTypeId'", because (in this case) you reference it in the Order By clause but don't indicate which source table you want, since the same column name exists in both tables referenced in the From clause.

Logically, it doesn't matter which one, because you mandate in the From clause that both UnitTypeIds are equal, but SQL syntax dictates you must specify which one you want.

Despite seeing this many times, it's still not automatic that I'll specify the table in my Select and Order By clauses, so I still see this error a lot.

I hadn't seen it recently, but suddenly it popped up on this one client's database. Sure enough, I hadn't specified the table in the Order By clause again. Yet it was working fine on other clients' databases, and working fine internally.

Our QA guy managed to find the "SQL Compatibility Mode" option on the database. Because they had upgraded this database from a SQL 2000 database, it was still in "SQL 2000" mode. Because of this, it was revealing the error in my SQL that was going through undetected on servers running in native SQL 2005 mode. He flipped the switch to "SQL 2005", and it let my bug go through.

Needless to say, my local database is now in the more restrictive "SQL 2000" mode, so hopefully I can catch more bugs before they're revealed by environment. It should also help if we ever decide to migrate to other database servers, such as MySQL.

Impact of video on bandwidth

Continuing the saga of bandwidth, I had an interesting data point come up this weekend. My church has a general conference twice a year, where we have an opportunity to hear from the leadership of the church. This is streamed live over the internet in fairly decent quality video (in my completely unscientific opinion, it looked just as good as any standard TV show might look on a 12" laptop computer screen).

The conference spans the entire weekend, with two 2-hour sessions broadcast each on Saturday and Sunday. At the end of the second Sunday session, the thought suddenly occurred to me; I wondered how much bandwidth I was using up.

Fortunately, this is something I can check without too much difficulty. A quick call to vnstat reveals the following statistics:

DayInOutTotal
Saturday4.68 GB206.82 MB4.88 GB
Sunday3.98 GB123.33 MB4.10 GB

Saturday will of course include some extra bytes for online gaming, which doesn't happen on Sunday.

Interesting thing about those stats: if that happened every day, watching four hours of streaming video, I still wouldn't hit the cap. I'd come far short of it, in fact.

2008-10-01

Bandwidth - the Baseline

September is now over, so I now have my baseline for monitoring my bandwidth, to see how I fare against the bandwidth cap. Total data used for the month of September: 22.36GB down, 5.44GB up, 27.80GB total.

September was a bit of an odd month, though. For about a week and a half, I was without an Xbox 360; and for almost the whole month, my wife was using my laptop while we tried to get hers repaired (the wireless network card was fried). However, even with those odd variables, there are some worthwhile data points.

First off, there is only one day in the entire month of September that shows a total over 2GB (2.24GB used on 1 Sep) — every other day is under 2GB, and only 5 days total are over a gig and a half, with an additional 7 days between the 1G and 1.5GB marks.

There is a definite dip in usage around the time my 360 died, so its effects are noticeable. There's also a spike on the 13th that would represent when I got my new 360 and proceeded to play the heck out of it all day. Another sweep up starts on the 23th, which could represent when I and three others decided to make a speed run at a Halo campaign level for a competition that Friday night (we practiced every night that week). The extra boost starting on the 25th may come from my wife finally getting her laptop back from HP and having to download updates and reinstall software.

Even with the up-ticks, there's still not a lot of bandwidth being used. If you take the heaviest of the days that should represent things getting "back to normal", the 25th, and multiplied that by the entire month, it's only 50GB, or about a fifth of the cap.

But how could that change? If I wanted to ditch the $40+/month "digital voice" service from Comcast and shop around for other VOIP options, I'd be looking at an increase in usage just for using my phone. If I wanted to take advantage of the upcoming Netflix integration with the Xbox 360, that would cause a huge increase in bandwidth, depending on how many movies I tried to watch. For that matter, the "New Xbox Experience" is creating a new paradigm for interacting with friends (i.e. they're ripping off "Miis"); will this have an impact on how much bandwidth is used just sitting in the dashboard, while my friends' avatars are displayed on my console?

I think I've come to the conclusion that it's safe not to worry yet, but it still makes me nervous about trying new and potentially high-bandwidth-eating applications or services. And I still think that's just the way Comcast likes it.