Entity Framework 6, SQL, and nullable strings

I ran into an issue that appears to be caused by Microsoft attempting to protect me from myself. Although, truth be told, it wouldn't have been an issue if things were a little better designed.

Imagine, if you will, a SQL Server database with a table of transactions. One of the fields on this table is a CorrelationId. It's a text field that is populated by a different system to tie transactions together (for example, two sides of a transfer from one customer to another). This field always gets populated on new transactions; the uncorrelated ones will just be the only one with a given CorrelationId. However, this system is not new; it was converted to replace an older system that did not have a defined CorrelationId. So, although the five million or so transactions created by this system have a CorrelationId, there are 12 million "legacy" records that have a CorrelationId of NULL.

So, say, for a given transaction, you want to find all correlated transactions. In SQL Server, you might use a simple query like this:

FROM dbo.TransactionTable
WHERE CorrelationId =
(SELECT CorrelationId FROM dbo.TransactionTable WHERE Id = @TransactionId)

And this would work, for the most part (except for legacy records, since SQL will fail to match on the NULL value — but we can ignore this for now). If you took this query into SQL Management Studio and looked at the execution plan, you'd see a nice thin line from the index seek on the CorrelationId, showing that it found and processed a tiny number of matching records, resulting in a very quick response.

However, if you were trying to do this programmatically from a C# application using Entity Framework 6, you might write some code like:

var query = from txn in transactionTable.Entities
where txn.Id == transactionId
join txn2 in transactionTable.Entities on txn.CorrelationId equals txn2.CorrelationId
select txn2;

The problem is, in C# code, null values are equal to another; while in SQL, "null" is considered "unknown", and doesn't equal itself. (The theory is, you can't know if one "null", or unknown value, equals another "null"; so equality tests between "null" and "null" are false.) Instead of leaving it up to the programmer to explicitly code for this condition, Entity Framework "helpfully" writes the join clause that it gives to SQL server in this manner:

JOIN [dbo].[TransactionTable] AS [Extent2] ON (
([Extent1].[CorrelationId] = [Extent2].[CorrelationId])
(([Extent1].[CorrelationId] IS NULL) AND ([Extent2].[CorrelationId] IS NULL))

The extra check for IS NULL on both sides has two unfortunate side effects in this case:

  1. If the transaction is one of the legacy records, it will return a positive match on all 12 million other legacy records with a null CorrelationId.
  2. If the transaction has a CorrelationId, because of the IS NULL, SQL Server will investigate the 12 million null values in the CorrelationId index, resulting in a big fat line from the index seek in the execution plan, and a return time of a couple seconds or more.

The really annoying part is that there doesn't appear to be a way to stop this. Even if you explicitly add a check for a not-equal-to-null on your target table, Entity Framework still wraps the equality test with checks for IS NULL. The result is almost comical. For instance, adding txn2.CorrelationId != null either in the join statement or as a where clause, results in this (with contradictory statements highlighted):

[Extent2].[CorrelationId] IS NOT NULL
([Extent1].[CorrelationId] = [Extent2].[CorrelationId])
(([Extent1].[CorrelationId] IS NULL) AND ([Extent2].[CorrelationId] IS NULL))

Even trying to break up the work into two statements didn't help. This code:

var corrId = from txn in transactionTable.Entities where txn.Id == transactionId select txn.CorrelationId;
var txns = from txn in transactionTable.Entities where txn.CorrelationId == corrId select txn;

Resulted in this SQL:

WHERE ([Extent1].[CorrelationId] = @p__linq__0)
(([Extent1].[CorrelationId] IS NULL) AND (@p__linq__0 IS NULL))

Granted, this is a really bad situation to be in to begin with. Indexes on text fields tend to perform poorly, and having such a huge number of null values in the index is likewise unhelpful. A better design would be to rip the text field off into another table, or somehow otherwise convert it into an integer that would be easier to index (something we've had to do in other tables on this very same project, where we've had more control of the data).

I'm willing to bet that Microsoft's translation goes completely unnoticed in over 99% of the cases where it occurs. And, if I had the time to make a design change (with all of the necessary changes to all points that hit this table, some of which I don't have direct control over), it could have been resolved without fighting Entity Framework. Even just populating all of the legacy transactions' CorrelationId with random, unique garbage would've solved the problem (though with a lot of wasted storage space that would've made the infrastructure team cry).

In the end, it was solved by creating stored procedures in the datbase to do correlated transaction lookups (where the behavior could be controlled and expected), and having C# code exectue those directly (bypassing EF6) to get transaction IDs. Standard Linq queries would then use those IDs, instead of trying to search the CorrelationId.

This whole exercise was prompted by a script that I had to run to get a bunch of data from a decent number of transactions. It took nearly eleven hours to complete, finishing close to 1am after I started it. If I had time to go through this debugging and implement the fix, it turns out I could've gotten it done in about a third of the time.


The Church and the BSA

The news broke that the Church of Jesus Christ of Latter-day Saints has officially announced that it will no longer sponsor Boy Scouts of America troops. As a member of that church who currently serves in a calling as a Cub Scout den leader, and as a father of four boys — one of which just earned his Eagle Scout, and one who likely will within the next year — and being married to someone who currently volunteers as a Roundtable Commissioner for the BSA district, I have some thoughts.

First off, this has been a long time coming. The Church has members all across the world, and the Boy Scouts is an American institution. While Church leaders have often noted that the BSA is, essentially, the Young Men's organization within the Church, this has been completely unavailable outside of this country. We've heard rumors for a long time of the Church looking to institute something that would be available to members everywhere.

There are certain benefits to dumping the BSA from the Young Men's program. I've heard from numerous sources how the youth budget is often seen as unfair, with the Young Men getting far more allocation than the Young Women. However, the main reason for this is that the Church pays much of the registration fees, awards, and so forth required by the BSA — which is something that the Young Women do not have to deal with. The BSA also has many rules, regulations, and so forth that can be difficult for a ward full of unpaid volunteers to navigate. (One of the responsibilities my wife has is to help new scout troops and packs get through some of this — and the ones requiring help are not limited to LDS Church-sponsored troops.)

The move could also allow more freedom in Young Men's groups in their activities. Without having to follow strict rules and a proscribed list of activities and merit badges that boys are supposed to earn, it could be more easily tailored to the needs and desires of the people in the program (kids and leaders alike).

On the other hand, there are some things that would be a great loss. The structure and required activities are sometimes a benefit. When I asked my wife about her experience growing up in Activity Days (the program for girls the age of Cub Scout boys), she expressed her disappointment that it was little more than a cooking class. Her leaders were skilled in the kitchen and enjoyed doing that, so, without any real incentive to do anything else, that's what they did, week after week. Not being a very outdoorsy person myself, if I didn't have the Cub Scout requirements to drive me, I would likewise have very little incentive or direction to do some of the things that the boys in my care would enjoy (or should learn).

With the lack of BSA sponsorship, access to some of the campgrounds and resources that the Church currently enjoys will be lost. So although there will be more money to go around, some activities will cost more.

The Boy Scouts of America is a national institution, recognized by people regardless of their religious affiliation. Anecdotally, having an Eagle Scout rank is something that employers see as an asset in potential employees. Certainly, a Church program would be less likely to carry the same weight, even if its requirements were at all similar to those of the Eagle Scout.

There is very little commonly known about the new youth program-to-be. The official separation — and, I presume, the implementation of the new program — is scheduled for the beginning of 2020. Time will tell how many of the benefits and losses will be offset by the new benefits (and drawbacks) of the new program. My belief in an inspired leadership gives me great hope that many of these things will be addressed, and that the new program (which, incidentally, is to replace the current programs for boys and girls) will be successful.

One common refrain in the news is that the entire LDS membership of the Boy Scouts is going to go away. While that's most certainly an exaggeration, there is some truth to that. Without the encouragement inside of the Church to join Scouting, it's very likely that fewer boys will feel inclined to join. Also, something I've heard from many parents and have experienced myself, is that there seems to be very little time to fit in all of the extracurricular activities that all of your kids are involved in. With the Church bringing in its own program, boys that want to do Scouts will now have two activities to balance (Scouting and whatever the new program is) — and there's no guarantee that meeting times won't conflict. Also, although religion has been an integral part of the Boy Scouts (belief in God is a requirement, even if the manner of worship is completely open), I have heard various reports of non-LDS-sponsored packs and troops using Sunday as an activity day, which is something that most members of the Church try to avoid. Additionally, an LDS-sponsored pack and troop typically takes care of paying the dues for its members. Joining another troop will mean paying those membership costs out of the families' own pockets. (I have not had much personal experience in this, so I do not know how willing and how much other troops can assist with dues and fees for families that may need the help.) So even though the option to join Scouting is not out of the question, it will be more difficult for LDS youth to make that commitment. The BSA membership is probably going to take a significant hit. While it might not be 100% of LDS youth, it wouldn't surprise me to see it be very high.

The timing of the announcements has certainly been interesting. Several publications have noted that the announcement of the LDS/BSA split came within a week of the BSA announcing that they would change their program name from "Boy Scouts" to "Scouts" (as part of their move to include girls in the program), leading many to speculate that the name change was part of the cause. I won't deny that it certainly looks that way, but I suspect the reality is much more complicated.

The announcement of the split consisted of simultaneous press releases from both the Church and the BSA. While it may not be outside of the realm of possibility, I have a hard time believing that a coordinated PR move happens that quickly, which leads me to believe that both sides knew about this for a long time. Now, it's certainly possible also that the Church knew of the BSA's plans ahead of time as well, so there might yet be some merit to the idea that the Church made plans to leave because of the direction the BSA was moving. It's also possible that the BSA knew the Church was planning on striking out on their own with their own youth program, and the decision to include girls was made to try to mitigate the potential loss of LDS boys from their ranks. In other words, the cause-and-effect might be reversed.

Still, from the perspective of public perception (or at least the narrative that some press are driving), it does look like the Church is abandoning the Scouts because of the changes in the Scouts. While it may sound conspiratorial, I have to wonder if it wasn't timed to make the Church look bad, so everyone can point and laugh at the church, "Look at how those backwards Mormons run from simply having to include girls in their program!" On the other hand, if the timing were reversed, it may appear that the BSA would be the ones to appear reactionary, and the story would be, "Look at how the BSA is so desperate for members now, that they're changing their name to include girls!" The timing from the BSA could have been more defensive than offensive.

All told, I'm kind of looking forward to the change. While Scouting has had some great benefits for my boys and the other boys in the Church, I won't deny the administrative side has been a stress to deal with; and I'm hoping the new program will bring some positive change. I don't know if my own younger boys will continue in Scouting (my oldest is already an Eagle and moving into his adult life; my second oldest will be aging-out at nearly the same time the split becomes official), but it will be a choice we will prayerfully consider in the months to come.


Forget this "Free Healthcare" business!

I had one of those shower moments where I started replaying conversations and debates I've ever had or witnessed. For some reason, my mind had settled on the idea of universal healthcare (not something I've argued much on either side, but definitely witnessed a lot). Proponents often describe this as "free healthcare", which leads to opponents arguing that "it's not free" since it's paid for by taxes. I've even seen one argument that you'd have a hard time convincing a doctor to use his skill and many years of medical school learning and training for no cost.

"Yeah," said the voice in my head, "just like police and firemen should expect to be paid for their service."

And that's when it occurred to me. The proposal shouldn't be "free healthcare"; it should be "make healthcare a public service". Because that's really the truth. No one's really suggesting that anything be "free". They're suggesting that the costs be covered by society as a whole (i.e., government, paid through taxes), rather than by the individual using the service at that point in time.

While I can understand the appeal of calling it "free", I think proponents do the discussion a great disservice by using that word. It implies, at best, a fundamental misunderstanding of economics, and, at worst, a lie covering it up (since both sides know that health care costs actual money, that it's not really "free" at all).

Do I think a mere change in word choice will clear up the whole discussion? Absolutely not. There are still plenty of points to argue — quality of care, the ability of government to manage, and the actual cost for the public, just to name a few — I do think it would at least let us get past the part where we argue about "free" being "free" or "not free".


Pass a recordset to C# by way of XML

UPDATE: I have to discourage using this trick. For reasons I do not yet know, it doesn't seem to work with a large dataset. I do not know the exact point at which it fails, I just know that it does. I noticed that a significant number of values that should have been updated with text, actually got updated with nulls. As much as I would love to investigate this and try to see what is wrong and whether it's a failure in C#, SQL, or some combination, unfortunately, it's more important that my work actually get done; so I've had to abandon the XML route entirely.

Original post follows.

It's been a while since I've posted, well, anything. But I learned of a neat trick that I thought I'd post.

I'm currently working on a program that is converting data from two different sources into a single database. A lot of it is just done with carefully crafted SQL statements, but there are a few steps where I have to take data from one source and use some C# code to do some kind of processing before storing it in the target database. Since the data set is on the order of millions of rows, processing these records one at a time can be prohibitively time-consuming. And, since I have limited access to the SQL Server itself, using SQL CLR isn't a great option. (I probably could get the access if I needed to, but it will be an additional step to have to remember and configure when this goes to production, and the fewer moving parts I create for myself, the better.)

One of the tricks I've implemented is to use multi-threading to let the different steps run simultaneously — one thread extracts the records and puts them into a ConcurrentQueue<>, another thread processes that and puts the results into another queue, and a third thread updates the records in the database.

I've been trying to come up with ways to do the update in batches. There are ways to create a stored procedure that will take a table parameter, and ways to call that stored procedure by binding the parameter to an equivalent DataSet, but I didn't like the idea of creating a DataSet object just to pass the records in. It just seemed too "heavy" to me. (Though it might've been faster than calling a command object in a loop for records one-by-one.)

Another option was to create a VALUES table and build the command text dynamically. But, since I was working with strings, I didn't like the idea of building dynamic SQL and having to escape quotes or any other special characters that might cause SQL to choke. (Not to mention it's just bad practice, even if my code is unlikely to be used as a SQL injection vector.)

So, I came up with the idea of passing in values as an XML document. By building the XML with Linq-to-XML C# code, all necessary character escapes would be performed automatically. I could pass in as many values at once as I felt comfortable with, and let SQL do the work in a batch instead of one at a time.

To give some context to this code, I am taking email addresses that were encrypted in the source database, and converting them to their decrypted values in the target database. At this point, my queue consists of objects that have two properties: EncryptedEmail and DecryptedEmail. Earlier in my conversion work, I've simply copied the encrypted strings over into the Email field of the table, so all this method has to do is update the table and changing the Email field to its decrypted value.

var recordsToUpdate = GetRecordsToUpdateBatch(250); //Retrieves up to 250 records off of the queue at a time
if (recordsToUpdate.Any()) {

var xdoc = new XDocument(
new XElement("emails",
recordsToUpdate.Select(r => new XElement("email", new XAttribute("encrypted", r.EncryptedEmail), new XAttribute("decrypted", r.DecryptedEmail)))

using (var connection = new SqlConnection(GlobalSettings.DatabaseConnectionString)) {
await connection.OpenAsync();
using (var command = connection.CreateCommand()) {
command.CommandText = @"
WITH emails AS (
--Convert the XML document into a table that SQL can use normally SELECT Tbl.email.value('@encrypted','varchar(256)') AS Encrypted, Tbl.email.value('@decrypted','varchar(256)') AS Decrypted
FROM @emails.nodes('/emails/email') AS Tbl(email)
) UPDATE cust SET cust.Email = emails.Decrypted
FROM dbo.Customer cust
INNER JOIN emails ON cust.Email = emails.Encrypted;
command.CommandType = System.Data.CommandType.Text;
var param = command.CreateParameter();
param.ParameterName = "@emails";
param.SqlDbType = System.Data.SqlDbType.Xml;
param.Value = new SqlXml(xdoc.CreateReader());

await command.ExecuteNonQueryAsync();

await Task.Run(() => Thread.Sleep(1));


The server exploded.

As a warning, because I know the boys will tell you as soon as you walk in the door, the internet is down.

As I was debating whether or not I should leave work or try to get one more thing done first, I got a text from my wife saying the internet was down at the house. It happens from time to time. Usually, I just have to reboot something, and usually, that thing is the wireless router that most everything connects to. It's a Rosewill, which I bought mostly because my trusty Linksys WRT-54G was having a hard time keeping up with all the devices we kept adding to the mix. The Rosewill supports wireless N and has a much better signal range, but maybe once every week or so, things just go a little "wonky" and it has to be rebooted. Just this past weekend, in fact, it ended up "jamming" my home network completely, sending so much traffic (that wasn't actually going anywhere) that none of my machines could hear each other. I didn't immediately know it was the router at the time, but when I went to the basement to check the servers and all the networking gear, that's when I saw the rapid flashing on the switch connected to the Rosewill router.

I didn't expect it to be a big deal this time, either, but I got a couple more texts with some more details. Of course, my wife had tried rebooting the wireless router already. (Even the kids know that, sometimes, you just have to go over to it, pull the power plug, wait a few seconds, and plug it back in.) When that didn't help, she went down to the basement herself, and she heard some high-pitched beeping from what she described as a small box with blue lights on it. She turned it off, waited a bit, and tried turning it on again; and when it started screaming at her immediately, she just turned it back off.

From her description, I knew the device in question was the UPS. It seemed strange that the UPS would be beeping like that, unless the power was off and it was running out of batteries or something. It's a common story in tech support circles to get a call from someone who claims their computer doesn't work, and only after troubleshooting for a while does the clueless user say something like, "Well, I can't quite see, because the power is out and it's dark in here." I didn't believe my wife would fail to mention a power outage, though, so I figured it must be something else. The UPS going bad, perhaps? A tripped circuit breaker that cut the power to that outlet?

I got home and went downstairs to check things out. It was very quiet, which seemed like a bad sign. Two servers — the email server, and the main server that does just about everything else — are plugged into the UPS, but a third — the media server, which stores all our DVDs for easy access — is plugged straight into the outlet. If the UPS were bad, the media server should still have power. At this point, I'm thinking it's a tripped circuit breaker.

I go out to the power box to check the breakers, but none are tripped. I locate the one going to the servers and flip it off and on, just in case; then I head back inside. At this point, I'm getting a little concerned. We ran that power line ourselves; did we do something really wrong in the process? It's been fine for a few years, though. If it did go bad, what can I do to get power to that corner of the basement while we figure things out?

Back in the basement, the outlet still has no power. I noticed, though, that we installed a GFI outlet. Maybe that's what tripped. I pushed the red "reset" button, and as soon as it clicked, the media server hummed to life. Ok, that's what's keeping things down. Now, I'll bring things back up and see if it trips again, and then figure out what's causing the problem. I turned on the UPS, which gave only the slightest of beeps. The email server gave a soft beep as it got power, and then….

A little background on the main server. This thing runs pretty much everything. It is the only thing connected to the cable modem on one network card, and another network card connects to the switches that distribute internet traffic to the rest of the house. It was a machine I built several years ago, picking out the parts and assembling them myself. I didn't look for anything special in the case, but the one I happened to find on sale had some interesting LED lights on the fans and a clear side window, so you can see everything inside. I didn't even know about these features of the case when I bought it; I was just looking for something that would hold all the parts together for a decent price. Over the years, the server has been carefully configured to do everything I need it to. It has a web server, which is mostly used by my wife for her web design work. It has a minimal email server, which does some preliminary filtering before passing email on to my "real" email server inside the network. It does the firewall and routing, with some hand-crafted iptables scripts to make sure bits go where they're supposed to. It has a DNS server, which is configured to give easy access to important devices on the network by name, plus has the bonus of having a few hundred known advertising sites redirected to the address as a convenient, network-wide ad block. (Fun fact: I tried to do the same with porn sites, but when I got a list of known sites and fed them into my DNS server, it promptly crashed. There were just way too many to filter out wholesale.) It also has a large file store with an FTP server used internally to back up, share, and keep files we want to hang on to.

Anyway, as the main server got its turn to power up, there was a series of three or four very loud POPs, accompanied by a bright flash that could be clearly seen through the case's clear side panel. Accompanying the popping noise, I shouted something that I don't quite remember. And then everything went quiet again as the GFI switch once again tripped and cut the power. A thin tendril of blue smoke leaked out of the power supply fan of the main server, and the smell of fried electrical parts hung in the air.

I went upstairs and told my wife the bad news. The server just exploded.

My wife helped me get the server unplugged (mostly because, even with the power cut, I was still a little terrified to touch the thing after what I had just seen), and I took it upstairs where I had more light and began taking it apart. There were a few cobwebs and a lot of dust inside, but no obvious sign of what blew up. Unfortunately, there's no real easy way to tell what may be good and what may be dangerous. Unwilling to risk frying any more components than necessary, I resigned myself to having to buy a new machine and rebuild.

I can only hope at this point that the hard drives are ok. The server contained four in total — two smaller ones that held most of the OS, and two larger ones that made up the file share, each pair in a RAID-1 array. But without access to the internet, downloading the appropriate installation media would be tricky. Not that I had a replacement server handy, anyway. First things first, find a replacement.

I took a quick trip to the nearest electronics-type store, that being Best Buy. I knew it was probably a long shot going in there, and, unfortunately, I was right. Plenty of laptops and costly consumer desktop systems, but nothing that would be good for a server. I wasn't willing to overspend on a system that wasn't suited for the task.

My next bet was Micro Center, which was a half hour away. Unfortunately, that, too, was a wasted trip. Pre-assembled systems were limited to the desktop and laptop variety. They do have a large array of components for building machines from parts, but, being perfectly honest with myself, I was not in a frame of mind to start piecing one together in a hurry. If I'm going to build something, I want to take the time to research, and really put together what I want for the best value. But I need a server, and quick. I figured Amazon is probably going to be my best bet.

In the parking lot of the Micro Center, I double-checked Amazon's site. (I had looked before I left the house, but I didn't commit to anything as I wanted to at least try to buy something from a local store that I could take home and start working on that night.) I found a couple possibilities, but my biggest issue was trying to find the internal specs on the machines. This mini-tower server looks like a good deal, but does it have the internal space and ports for four full-sized SATA hard drives? I don't know if it was because I was trying to use the mobile website, or if their site was really lacking that information, but I found it really hard to find. (I probably would have found more details on NewEgg, but I was wanting to take advantage of Amazon's better prices and faster shipping.) I found one that actually included a mention of "space for 6 drives" in the description, placed the order, and elected to pay extra for one-day shipping.

On my way home, I started to go over my options. I wouldn't be able to restore the web and file server until the new machine arrives, but what could I get up and running now? I had that old Linksys wireless router, which I had installed DD-WRT firmware on — meaning it is something that is very configurable and something I could really tweak. That, I figured, could take the duty of routing and firewalling for the internal network, and we would at least have internet access again. Email might be a bigger problem, though. Sure, the email server was alive, but the way I had it configured, I depended on the main server to filter email first. Maybe some of the security settings I had applied in the not-too-distant-past would allow me to grant it more direct access to the internet without becoming an open relay for spam mail. But that could be a secondary task.

I got home and set to work, hooking up the Linksys router in the place of the main server. I had some issues getting it configured, since my prior tinkering with the device (when it was just a toy to play with) had left it in a weird state. I ultimately had to reset it to its default state and rebuild it from there. DD-WRT has a very convenient web-based interface, though, and it took me much less time than I expected to get things to a working state. The thing that slowed me down the most was the fact that devices on the network still remembered their configuration from the main server, and didn't immediately update to point to the Linksys router when I brought it online.

With that accomplished, I figured I'd try setting up email. I did have a few small issues configuring the network, but again it came down to having to just reboot the server a couple times to force it to update its network configuration. I had some issues from there trying to get some external email server testing programs to talk to my email server, and that slowed me down a bit. It turned out that the email server didn't take too kindly to being forcibly rebooted, and the email services just plain hadn't started up. (It's amazing how much better things can work if the expected program is actually running.) I forwarded the secure email ports through the firewall easily enough, but I wasn't too sure about opening up the unsecured email port required to let outside email come in. It turned out much better than I expected. The security settings I had enabled recently were working perfectly. I ran a couple different open relay tests against my web server (which is something I always, always do when I tinker with the email server — last thing I want to do is to get shut down because my email server is sending out everyone else's spam mail), and it passed perfectly.

So, now I'm back up and running with internet access and email. The major items are taken care of, so everything else from here can get rebuilt on a much less rushed timeline. (Still want to do it quickly, but it doesn't have to be done yesterday.)

Time to count the blessings and see what I learned.

The biggest blessing is that nothing burned down. The GFI outlet tripped, but the UPS at that point should have still been providing power. Near as I can figure, it also detected something was wrong and cut power, then beeped as an alarm. When my wife turned it off and back on, it must have been able to still detect the problem and not try powering on the server. I'm not sure what changed when I got to it later, but when I tried turning things on and it started making loud boomy noises, the GFI tripped again and the UPS just shut itself off immediately. If it hadn't, there could have been much more damage done, and possibly an electrical fire as well. (I'm still keeping my fingers crossed that the hard drives aren't fried.)

We're up and running. Email and internet are the most important things we have to keep going, especially with one child doing homeschool and taking lessons over the internet. I pay for a backup email server that, when our server is down, will receive and hold our email in a queue until our server comes back online; so we haven't lost any email.

With the Linksys router doing the routing and firewall duties, I can rebuild the main server and just keep it behind the firewall itself, without having to configure it for routing as well. Whenever the server has an issue in the future, I won't have to bring down the whole network. Plus, keeping the file share off of the computer exposed to the internet is a better setup anyway.

I should probably look into offsite backups. While I can hope that the hard drives didn't get fried, if it turns out that they did, I could be up a very smelly creek in a barbed wire canoe without a paddle.


When there's no will, there's no way

My relationship with my father can be described as "strained", at best. Growing up, I don't remember him being around much. He worked hard to make sure we never really wanted for anything; and though it meant we did have a very comfortable lifestyle, it did mean we didn't spend a lot of time together.

I've never been very good at communicating. I think there are many reasons for this, but I don't want to get into them for fear of being accused of trying to place blame. Even now, as a fully-grown adult, I get very anxious when I think about making a simple telephone call. I also tend to be forgetful and more than just a little lazy. My ideal vacation consists of one where I never leave the house. Not that I don't enjoy the occasional family trip once in a while, but I don't really ever fully relax until I can get back home.

My father has never accepted these aspects of me, though. While it could be considered noble to want me to be a better person, he's always taken my failings as a personal affront to him. If I don't call, it's because I don't respect him, and I'm a horrible, ungrateful little brat because of that. And he takes that as license to treat me as such. Which makes me more likely to act that way, and the cycle spirals downward.

My biggest issue, though, is that I am always the one to shoulder the blame, and the punishment. And, sometimes, that punishment borders on revenge.

Many years ago, we were having issues with email spam. An email address we had set up was getting hundreds of messages a day. We decided to change our email address to try to hide from some of these electronic assaults. I think that might have been the time when we set up our own email server, so that we could create new email accounts at-will and give a new email address to every website with which we were inclined to register. This is a huge benefit in that it means, when we start getting spam, we can identify exactly whose email list was hacked or sold, and we can terminate a single address without affecting our communication with any other person or business.

Anyway, we sent out notices about our email address change, and after a couple months, deactivated the account. I am not certain if we failed to send such a notice to my father, or if he failed to read it and update his address book (though I'm leaning heavily towards the latter, for reasons I will explain later), but eventually, he noticed that emails he was sending were going unanswered, and eventually started to bounce.

I'm not sure how long it was before he eventually got in contact with us, and we told him (or reminded him) of our email address change. I'm not sure how we failed to communicate this, because apparently what he heard was: "We changed our email address and didn't tell you so we wouldn't have to hear from you." And, being so affronted, he then stopped sending birthday or Christmas cards for a while.

This was annoying for two reasons. One, it was clearly a misunderstanding. Somehow our change of address didn't "take". Although we were reasonably sure we told everyone, we accepted responsibility, apologized, and made sure he had our correct email. A reasonable response would be to accept that it happened, it's been resolved, and move on. But his response was to be angry that it happened, accuse us of doing it on purpose, and to punish us by not sending Christmas cards anymore. Two, he went beyond us. Not only did he stop sending my wife and me anything, but he stopped sending birthday cards to his grandkids, too. He was punishing them for what he perceived as our "attack" on him.

This was a very long time ago, so I don't remember exactly when things might have turned around. It might have been when my son was admitted to the hospital a few years back, and I decided to call him and let him know. But in any case, we had been back to sending cards and gifts for birthdays and Christmas again.

But moving forward to a few years ago. He and his wife came to visit around Thanksgiving time, as they decided to take a vacation to the Rocky Mountains and would stop by for a couple days. They stayed for a day, and we chatted for a while in the living room, but he wasn't feeling well and ended up spending most of the time back at his hotel to get some rest.

The next month, he sent a Christmas card. Since he handed us a check when he came to visit, I didn't expect there to be anything inside. Which I'm not complaining about. In any case — as with all Christmas cards and checks we get from relatives that time of year — I didn't open it right away and put it on the tree, for opening on Christmas day. We had also sent him a package, with some hand-made quilted wall hangings that Karen had been busy making for relatives all that previous month.

Christmas day came, we opened our gifts, and I opened the card to find a rather sizeable check inside. We were pretty thankful for this. We knew he was planning on having a big family gathering at his house that next summer, and we were planning on going, but we were a little concerned about the logistics of making such a trip with four kids on a budget. (I have not done as well for my family as he did for his when it comes to finances. We've always had "enough", and in many cases even some "extra", but I don't know that I would ever consider us "affluent". Though maybe that's because my only real comparison is with how much my own father provided us.) I put the check aside, with the intent to deposit it later, and, as is typical for me, didn't think about it for a bit.

A couple weeks had gone by, and the checks were still on my "to-do" board, since I had forgotten to grab them on my way out to work every day since. (Again, typical me. If you ever send me a check, and you need it cashed right away, please call me and I'll make it happen. Because it's not unusual for a check to be sitting in my "out box" for a month before I remember it's there. I've had our church's clerk come to me and ask me to deposit a reimbursement check for things we've bought for the scout troop on more than one occasion.) So I receive this email:

We sent you a Christmas card with an extra check to maybe make your Christmas a little happier. We didn't hear from you for Christmas or New Year's which is not unusual as you hardly ever call. But I am concerned about the check being lost in the mail and I am going to put a stop payment on it. If you did receive the card and still intend to deposit the check, please let me know before Monday.

Ok, oops. I hadn't deposited the check yet. Need to rectify that. I'd better reply.

Yes, we sure did, just haven't remembered to take the checks to the bank yet. Thanks so much! We're going to be putting that away for our trip out there this summer. Looking forward to it.

How was your trip up to Harvard? I heard there was a big storm that hit New England around that time.

Oh, but he won't have any of that "cordiality" or "small talk".

Unbelievable. You would think I would not have to solicit a thank you, or a Merry Christmas.

And you'd think he would know by now that I'm just not that good at replying. But the sound of his emails imply that we made zero communication at all. Notice how there's no mention of the gift we sent him (that Karen had been stressing over making the prior few weeks), which did include a "Merry Christmas" card enclosed. I don't know whether to be concerned or angry — concerned that our package didn't get to him, or angry that he got it, didn't say anything about it, and tries to lay the guilt trip on me for not saying anything. I'll try to keep things cordial by assuming the first, and copy his wife just in case she got it but forgot to tell him.

Well, now I'm a little concerned. Did you not get our gift to you? Karen spent a lot of time and effort designing and making it. We sent it to the Alabama address around the second week of December. Did it arrive? Vail, have you seen the present we sent?

Enough already, I'm not going to play your game. Yes we got the Christmas present you sent and thank you. And, If I recall correctly, you said thank you when I gave you the Christmas card when we stopped to see you in November. I guess I thought that a check for [amount redacted] would be something a little extra special and something you might pick up the phone and call and express your appreciation, or call on Christmas, or on New Years. I know I don't call, I stopped several years ago because I felt communication with you was a one-way street.

Maybe our feelings toward each other are mutual. Maybe you think I am rude, inconsiderate, and ungrateful for anything you do. That is the way I feel about you and it just isn't worth it to me to try anymore. Please take the check and put it to good use for your family. You don't need to think I sent it to you so you could come visit us.

Yes, see, I forgot the rules of the "game". The rules that say I'm totally at fault for everything, and how dare I accuse him of not taking some of the responsibility himself. He claims to want to talk to me, but he'll be damned to be bothered to actually pick up the phone himself and call, because he doesn't feel that I don't call him enough. Which, yes, I don't, because that's just me. Yet he takes it as some personal slight to him, and because I don't live my life according to his rules, he can't be bothered to try.

I will give him credit for the masterful way he phrased this, though. He managed to call me rude, inconsiderate, and ungrateful, but worded it in a way that put the blame squarely on me for thinking that's what he thinks of me.

But, at least he did address my concern — our hand-made Christmas gift did, in fact, arrive; so it was just his own hypocrisy at work here. "How dare you not thank me for my gift! I don't have to thank you for yours, because you wouldn't appreciate it anyway!" But now I'm starting to wonder about this upcoming trip. Do I really want my kids to be exposed by this kind of treatment? Maybe he doesn't want me there, anyway. Of course, he's unlikely to admit to any such thing, instead making it so that my failure to attend is my decision and fault anyway. Perhaps he would answer a direct question.

No game was being played. Your previous message seemed to imply that you received absolutely nothing from us with regards to Christmas, and we were concerned that what we did send didn't make it. But I’m glad to hear that it did arrive.

You have always made your feelings for me abundantly clear. You can rest assured that none of this was any revelation.

We were legitimately looking forward to using the money you sent for coming out to visit, but the end of your message makes it sound like you don't want us to do that. We are willing to come, but it is your home, so I will leave the decision entirely up to you. Do you, or do you not, want us to come and visit this summer?

Don't bother.

Your actions have been abundantly clear since you've been a teanager and I don't know that there will ever be anything I can do to change your feelings toward me, either. Just like my father told me a few years ago,he did for me and I did for you what I thought was best for you.

Have a good life. I'll stay out of it.

I'll be there if you ever appreciate or respect me as a father, all be it not a perfect one. But I am not going to continuing doing things that are not acknowledged or appreciated.

This will be my last communication.

"Don't bother." As close to a direct answer as I could expect.

The cry to appreciate or respect him as he is, is, once again, hypocritical and insulting, when he obviously has no respect for the kind of person I am. You know, "all be it not a perfect one." And I really hope I don't harbor resentment for my own kids the way they act as teenagers the way he does for me. I'm sure he didn't act out at all when he was a kid, either.

I replied to make it clear that I refuse to accept total responsibility.

As you wish. If you ever decide to change your mind and treat me with the level of respect you seem to expect from me, you know how to reach me.

I did deposit the check, only to get a call from my bank a few days later to inform me that he did stop payment on it. I guess he really didn't want me to "take the check and put it to good use for your family", though I'm more inclined to think it was his reply to my last email (which he couldn't actually write since he already said it was his "last communication"). He had to get the last word in somehow.

A couple years later, we took a trip to visit family. When we got to my father's sister's house, we got a lot of admonishments as to how horribly we were treating my father, and how I should call him. Apparently, he has spent a lot of time telling his family what a horrible son he has and how I'm exiling him from our and his grandkids' life. Up until this point, I have not discussed our relationship with anyone in the family — I did not feel it was anyone else's business and didn't feel the need to burden anyone else with our "issues". Besides, I don't communicate much, anyway. My father, on the other hand, obviously did not feel the same. We had to spend a lot of time explaining our side of things while we were there.

I can only assume that some of our conversations with the family got back to him (one of our biggest complaints being how he seemed to punish the grandchildren for our disagreements or misunderstandings, when he severed all communication), because he actually sent us a card for the next occasion. We sent him one back, and, later, he mailed us a thank-you note that he had tried to email to us. Remember when I suspected that he just didn't update his address book when we changed our email? Yeah, the thank-you email had bounced back from an email service we hadn't used in well over a decade now. Apparently, if he wants to chide or berate me, he has no problem using the correct email address; but when it comes to sending something nice, suddenly he can't use the right address?

We continued to send cards back and forth. (Though not gifts; if he's not going to acknowledge the time and effort we spend into making and sending gifts, it's not worth sending. See, I did learn something from him after all.) We also made sure to send a thank-you card for every gift he sent us and the kids, since that seemed to be very important to him. But it was very obvious he had no interest in talking to us, just the grandkids, when birthday cards would come in for each of the kids and our days were conspicuously skipped. We had almost missed Father's Day, so we sent a quick e-card and, respecting his apparent wishes, "signed" it from the grandkids.

Apparently, the absence of my and Karen's names on this email was the last straw, as this email came in addressed to Karen:

Please inform my son that I have an appointment with my estate attorney in a few days with the purpose being to take him out of my will. I don't know what I did that was so bad that has resulted in his lack of any respect for me. I do believe I was not half as hard on him as my father was on me and I do believe had I not been hard on him he might not have been what he is today.

I intend to send birthday cards and gifts through the end of this year. After that, it just isn't worth it to me to make any more effort.

TL;DR: How dare you snub me after I snubbed you! I quit!

So, I suppose, that's it. Because he makes efforts to punish me for not being the kind of person he demands me to be, and because I don't kowtow and give him the love and adoration and respect he demands while he treats me like dirt, I'm not worth his "effort".

Some of my reaction may seem harsh with the limited discussion here, but this is just a small sampling of my interactions with him. And most of this has occurred after I have gotten tired of the beating down, bullying, and lack of respect as a person that I've endured for many years. He was a very toxic influence on my emotional well-being. I've tried to maintain the lines of communication so as not to cut him off from his grandchildren. But I have felt no guilt for declining to go above the minimal effort required when nothing is ever good enough.


Why does Disney hate families?

Our Netflix queue recently delivered the Disney movie The Princess and the Frog. It's an adaptation of the old frog prince fairy tale, and all and all it's pretty good. John Lasseter, who proved himself at Pixar, is an excellent storyteller, and Disney is benefiting greatly from his influence. (When I first saw Bolt, my first thought was, "This was about as good as a Pixar movie," and it didn't surprise me to see Lasseter's name in the credits.)

When the movie starts, you see a young girl, Tiana, with her mom and dad. Then, we flash forward, and the girl has grown up. She talks to a picture of her dad in a military uniform, and soon we see her mother in person. They make references to the father being "gone", but they don't immediately reveal his status. My wife and I wondered to each other, "Is he away at war, or has he died?" When, later, we finally hear the words "dad" and "died" used together, our response was, "Of course he's dead; it's a Disney movie."

Disney movies seem to have this "thing" about having a broken family. It's like they hate having a complete, traditional family. Seriously. Let's review. These are animated movies, where a main or important character and their family is present.

  • Snow White, Cinderella — wicked stepmothers
  • The Little Mermaid — Ariel's mother not mentioned; Prince Eric's parents never appear (being a prince, one would assume a king and queen, but they don't even show for his own wedding)
  • Beauty and the Beast — Belle's mother not mentioned
  • Aladdin — Jasmine's mother has died
  • The Lion King — father dies
  • The Emperor's New Groove — parents presumed dead for him to be a young emperor (mitigating factor: Emperor Kuzco gets his "new groove" from traditional family man Pacha)
  • Lilo & Stitch — parents have died
  • Brother Bear — Kenai kills Koda's mother, who appears to be his only living parent
  • Chicken Little — mother has died
  • Meet the Robinsons — orphan (mitigating factor: adopted into traditional – albeit weird – family)
  • Bolt — Only Penny's mother is seen; no father mentioned (and, Penny's character in the TV show within the movie only has a father; no mother is mentioned)
  • The Princess and the Frog — dad dies

Note that Pixar isn't much better:

  • Toy Story — Andy's mother is there, but there's never even a mention of a father
  • Finding Nemo — Mother is killed in the opening scene
  • Ratatouille — Remy clashes with his father, no mother mentioned
  • Up — Russell's home life is uncertain; we know his father doesn't spend time with him much, and there's a woman who lives with them that's not his mother (nanny? stepmom?)

Snow White, Cinderella, and Beauty and the Beast probably aren't fair to mention, considering they come from fairy tales and stories that have the situation already established. I'll concede that in some cases, the family situation is part of the story — The Lion King, The Emperor's New Groove, Brother Bear, and Meet the Robinsons wouldn't have been the same story by a long shot if they were in traditional families that stayed whole throughout the movie. Finding Nemo might've worked (a father flipping out over a son being kidnapped, even if he has a wife and other kids safe at home, isn't much of a leap), but seeing his wife and all his other kids die before hatching gave him a reason for being neurotic that you could relate to.

But the rest? Could Lilo have been a troubled kid even with loving parents? Absolutely. Could Tiana have still been trying to build the restaurant for her dad if he was alive at home, just old and retired and unable to work for himself? Yes. Would Toy Story have been any less of a story if Dad was at least around to help with the move, or the birthday party, or to open presents at Christmas? Could Jasmine have had a mother who shared in her father's concern for her future?

So, what's the deal? Is Disney just trying to make their stories accessible to even those with broken families? Is it just easier to not have to write an extra character into the story, or cheaper to not hire another voice actor or animator? Or do they really have something against a mother, a father, and a child together that they avoid the situation as much as possible?

Meeting Your Heroes

I often wondered how I would react if I met someone famous, like an actor. I figured that such people would often be hounded by fans who see them as (and expect them to be) something different than they are, who only "know" them because of the persona they put forward in front of an audience — a personality that, often, is really a product of not just their talents, but also those of the writers and directors that work together to build the show that the audience sees. If I were such a person, I reason, it would be much more refreshing to have someone meet me as a person, instead of as some fictional character that I was pretending to be. So, if I were to meet a famous actor, I would try to treat them as a person, so they wouldn't see me as just some starry-eyed fanboy out of thousands who just fawn over something that doesn't truly exist. Maybe they would appreciate not having to act for a few minutes; and, at best, maybe I could make a friend with a real individual (even if only for a moment). So, that's what I would do if I met a famous actor: meet them like any other real person.

I think it's safe to say that, when meeting Rob Paulsen (the voice behind the real Yakko Warner) at Denver Comic Con in the summer of 2015, that I pretty much failed to live up to that plan.

I wasn't sure I was going to go to Comic Con. My son and his friend were going, and though I knew the voices behind the Animaniacs were going to be there, I wasn't really feeling up to it. I'm not sure I would qualify it as a clinical depression, as I wasn't having any life-terminating thoughts. (I've had those before, so by comparison, this is "just feeling sad".) But the thought of being in large crowds with lots of people didn't sound too appealing. My wife, though, strongly encouraged me to go. I'm not sure if it was for my sake, or for the sake of being near my son (instead of sending him off to be downtown all day with just his friend), or both, but we ended up getting two tickets so that I could go.

I didn't take a ton of cash with me, so I knew that I wouldn't be bringing home a lot of autographs or memorabilia. But I did take my copy of the CD "Yakko's World" — of my Animaniacs CDs, that seemed to be the most appropriate to have Mr. Paulsen sign. After waiting in what seemed like an endless line, I finally got to meet him. My hands shook as I brought out my CD, and when he asked me my name, I'm not sure how I managed not to squeak like a little girl. As he signed, I told him how I used to take this music along on my occasional post-college road trips, how teaching myself to sing things like Yakko's World didn't just make the trip more fun, but helped me to keep my brain engaged and stay awake. I also mentioned how I was looking forward to the Animaniacs reunion sing-a-long later that day, so that I could actually sing along with him.

One thing I noticed in that very brief encounter, is that he seemed genuinely happy to meet people. Some celebrities, when you see them interacting with fans, you sometimes get the feeling that they would rather be somewhere else. They may not overtly show it, but you find they aren't really engaged in the moment. But I didn't get that impression from Mr. Paulsen at all. He really seemed to enjoy meeting each person that came up to him, throwing out one-liners in the character of whatever show his visitor was a fan of.

After meeting Rob Paulsen, I noted that there wasn't a ton of time before the Animaniacs reunion show, probably not enough time to wait in line to talk to the rest of the cast present that day (Jess Harnell, Tress MacNeille, and Maurice LaMarche were all there). But I noticed a shorter line decorated with Animaniacs pictures nearby, leading to one Randy Rogel. It wasn't a name I immediately recognized, but one I should have known. Mr. Rogel was the man behind many of the songs to which I would sing along on those road trips. He had available his song charts, which I eagerly grabbed, and he graciously signed. He also told me that he had written a new verse for Yakko's World, incorporating the changes in countries in the years since he first wrote the song. He was planning on performing it in the Animainacs sing-a-long that afternoon, something I told him I would look forward to hearing.

The sing-a-long was a real treat. Rob Paulsen and Randy Rogel performed some of the original songs, and the other actors would join on stage for some of them as well. One of the things I remember most is when Mr. Paulsen spotted someone in the audience dressed in a full Pinky costume. He said something in Pinky's voice, laughed, "narfed", and then said in a sing-song voice, "I get paid for doing this," as he did a little skip. The impression that I got was that this was a man who primarily loves what he does, has fun, and is sometimes amazed that it's also his job.

After the show, I went to get in line to meet the rest of the cast. Lines were already forming by the time I got there, so I decided to start with the shortest one (in a very relative sense, since they were all pretty long), and that was Jess Harnell's line. Eventually the cast made their way back upstairs as well, to cheers from the crowd. The rest took their places at their tables, but Mr. Paulsen did something different in his line; he went down the line, shook hands, and said hi to everyone, before going back up to his table.

I met Jess Harnell and Tress MacNeille. Mr. Harnell was an easy person to talk to, and was happy to hear about me being able to teach my kids things like Wakko's America. Ms. MacNeille was a little harder to read. I didn't find her rude or anything, but she definitely did not seem as engaged with her fans as her "siblings" were.

I then got in Mr. Paulsen's line again, just to say how much fun it was to sing along. I might've said something else, but I honestly couldn't remember. I just knew I wanted to meet him one more time.

After that, I went back to see Mr. Rogel. I noticed that the mysterious new verse of Yakko's World didn't make an appearance in the sing-a-long, so I wanted to ask him about it. He pulled out his own copy of his charts, which had the new verse in it, and actually sang it for me. Awesome! I got to hear a new verse to a song written by the man himself! I wish I could've recorded it, but getting video was one of the things that cost money (I don't fault them for that; I understand that impressions and paid exposure is how their industry works), and, as I said, I didn't bring much.

I had yet to talk to Maurice LaMarche, so I got in his line. As I was waiting, I kept looking over towards Mr. Paulsen (since they were next to each other). That's when I noticed something about myself: my cheeks were starting to hurt, because I was grinning so much. I was enjoying this.

Waiting for Mr. LaMarche, I noticed the pictures of his characters on his poster, and I was surprised to see the king from Frozen. When I met him and told him this, and how it impressed me when I'm unable to recognize a voice actor from a given character because they can change so much, he did a few more voices for me that I didn't recognize as him. (Sadly, I didn't think until it was too late to tell him, "Egad, Brain, you astound me!").

After this, I decided I had to talk to Rob Paulsen one last time. I'm not sure if he recognized that it was my third time to visit him, and hopefully he didn't start to see me as a creepy fanboy stalker (even if I was starting to feel like one). But I told him, honestly, that I hadn't felt like smiling in a while, and here I was, smiling so much it started to hurt, and I wanted to thank him. He gave me a genuine look of concern, came around the table, and gave me a big hug. That completely made my day.

They say you shouldn't ever actually meet your heroes, because you'll end up disappointed when you find out they're just human. Though I don't know that I ever thought of Rob Paulsen as my "hero" rather than as someone whose talents I enjoyed listening to, I have to say that the person I met was truly a kind and happy person; and in that, was definitely not disappointed.

I was able to meet some other people on a more human level. I met Garrett Wang of Star Trek: Voyager fame. I asked him about what he's been doing since Voyager, and we talked a bit about life goals (how he had been travelling, considering starting a family, that kind of thing). I also got to meet Nichelle Nichols, who had all the personality of a spunky old grandmother. She ribbed me a little for not giving her money, and then I got to shake her hand. So, as far as not going total fanboy meeting famous actors, I guess I didn't completely fail.