Our spot in Japan

We’re 99% sure that we’re going to take the rental condo that is the same building as Chiho’s parents. It’s a different floor plan than theirs and is slightly bigger. The apartment will be 100 square meters (or about 1,000 sq. ft.). That’s less than half the size of the Mukilteo house which is why we’re trying to sell so much of it. Here's the floor plan:

Standalone image: http://www.sbworks.com/images/apartment.jpg. I color coded and labeled the floor plan if you follow the link. The rooms that are "missing" doors actually have sliding doors.

It’s on the first floor of a fairly tall (14 story) building. It faces the ocean side of the complex but that’s not as good as it might sound. There’s a busy road right in front of the condo there. On the dining room & living room side of the house, there is a small yard, probably 30’ x 20’ that is fenced. Then there is a small berm with landscaping on it between our little yard and the street. The noise will probably be quite noticeable when the windows are open but we talked to the current renters and they say that the rooms are very quiet with the windows shut. Since the apartment has air conditioning, that will probably be workable. For the size of the apartment, there is a fair amount of storage. The kitchen is a little small (quite small in comparison to American kitchens) and that will take some real effort to get used to.

Right now, the company that owns and rents the condo is in the middle of refurbishing it after the previous renters moved out. I should be able to see the inside of it in a week or two and I’ll make sure to post pictures.

One month in...

I’ve now been living in Japan for one month and I’m still enjoying myself. Where we’re going to live, Makuhari (幕張), does feel like it’s a long way from downtown but I think I can handle that. The commute, if I really hurry and hit the trains perfectly, is 75 minutes (one hour and fifteen minutes) and that’s quite a bit longer than my old trip to downtown. However, I do break up the morning commute with a stop for coffee and Japanese practice.

One thing that I am really enjoying is that there has been almost no weekend work and everyone in the office says that weekend work is extremely rare. At ISOutsource, evening and weekend work was billed at a higher rate and, since you got a percentage, it was pretty common to do it. I guess I never noticed how much energy that evening and weekend work took. My weekends in Japan end up as quite relaxing and recharging.

As another plus, the office is a business-casual office so I don’t have to wear a suit and tie. That was the one thing from my Seattle lifestyle that I figured I would have to give up. Since the Japanese summer can be pretty brutal, I’m glad that short sleeve shirts are okay and suit jackets can be left in the closet.

I guess I’m still in the early part of “cultural adjustment”, though. I’ll have to write again in a couple of months to be sure.

EMC SAN - Nice hardware but...

At the new office, we have several really big EMC fiber channel SANs. We have it configured in a fairly complicated, high-availability way. Each server has two HBA controller cards and each HBA is connected to redundant storage controllers over Cisco fiber channel switch. This installation is expensive but the performance is excellent. The hardware is top notch and it's automated failover features are really solid. The EMC hardware engineers seem to to have their head screwed on straight.

But... but... but... What moron wrote their software?

To run an EMC SAN, you need to configure settings on the storage controller, use a client side software to mount the volumes, and then manage the backups with another software suite. That doesn't sound to bad, right?

The storage manager software is a Java based website that using LDAP authentication (yea... you don't need another password). To use the SAN, you create RAID groups by selecting the disks that should be included, then you define LUNs inside the RAID group, then assign the RAID group to a storage group, and then assign the storage group to a client. Then you turn clockwise three times, throw salt over shoulder, and (if all goes well) the LUN is available to the client.

I've actually simplified the steps, believe it or not. At the end of this, you get a LUN that acts like a disk to the server side operating system. It has a really useful name like LUN138 but you can assign something more human readable like "ExchangeServer01-StorageGroup1" or whatever. That will make your SAN Management easier (you would assume).

Now, let's more the client side. On the HBA fiber card, there is a world wide name (similar to the MAC address) and there is a way to assign a human-readable name like ExhchangeServer01-HBA1 instead of long hexadecimal string (which should make things easier, you assume). Once you have everything set there you flip back the the SAN manager to register the HBA card. And, of course, you can't read the human-readable name. All you get is the world wide name and you have to flip back and forth to figure out which 20+ character hexadecimal number belongs to which card.

After you get registered on the SAN, you load software on the client to mount the volumes. This client side software does not use LDAP. Instead, it uses a locally controlled password. Once you find out the password and load the software, it scans the storage groups available to the client and displays the LUN "ExchangeServer01-StorageGroup1"...... Well, no - it displays this long 20+ character hexadecimal number that you've never seen before. If you're only mounting one ore two that's not a big deal, but if you are putting in five or six that can be kind of annoying. To make it worse, these LUNs show up in Computer Management as Disk1, Disk2, Disk3, etc. and there is no other useful information as to which disk is which LUN.

So here's a scenario: you create four LUNS for four Exchange storage groups on one Enterprise server and you would like to mount the LUNs in a particular pattern. To make the LUNs match a pattern, you will need to flip between three different windows comparing hexadecimal strings to decide which disk in the computer management window belongs to which LUN. Wouldn't it be nice if the EMC software could read the EMC created human readable tags in the EMC storage system?

We haven't even gotten to the backup system. If your are rolling out a couple of servers with 5 or 6 mounted LUNs per server, you head is already hurting so make sure you take a long break before starting the next step.

Coffee would probably be a good idea.

EMC Replication manager uses cluster-by-cluster snapshotting to create a backup very quickly. You can snapshot a dozen 200 gigabyte LUNs simultaneously and it will take about a hour / hour and a half. Pretty slick, I suppose but (again) the software experience leaves something to be desired.

But, before you frustrate yourself with Replication Manager, you have to dive back into the storage manager. To make a replica, you need a backup LUN that has an identical cluster count and cluster size. And, of course, there is no "Create new LUN with these settings" or "Make replication LUN" option so you have to do it manually. If you are one cluster off, the replication will fail so you better write and type carefully. Go ahead, make a human readable name for the LUN - it won't do any good but it will make you feel better.

After you put the LUN in the storage group assigned to the Replication manager, you can then move to the Replication Manager software. You'll need another password for this, too. Another local password, of course. Oh, and you might as well make a service account with domain administrator access now, you'll need it later. Also, don't ever (never, ever) change the password on that account.

So, you logon the Replication manager, add the new LUNs to a storage group (no, not the storage manager storage group, a replication storage group) , assign it at name (no, it won't pick up the other name you already gave it), create an application group that defines the source LUN and give it a name (no, of course it won't pick up the name you already gave it), and then create a job (and, yes, you need to give the job a name). After you jump through all these hoops, it will create a Windows Scheduled Task to run the replication job.

Yep, you read that right. This expensive, complicated, high end software uses the incredibly unreliable Windows task service.

Believe it or not, it gets worse. The replication service uses a high-numbered TCP port (up around 65000) to manage the service. Ports in that range are not reserved and get dynamically assigned by a variety of programs on temporary basis. Since these high numbered ports are a free-for-all, most programmers make allowances conflicts. You saw this coming, didn't you - EMC's programmers did not make allowances for conflict. For example, an MMC console open in one session of one server that gathers information from another server (Exchange System Manager, for example) and that MMC decided to use the EMC replication manager port. This will make the snapshot fail completely. To top it off, failed snapshots are not written over so if you have some sort of overwrite pattern going, that pattern will probably fail the next time it's run, too.

And, you'll love EMC's fix - make a registry edit on every single server that blocks other programs from using that port. They aren't even offering us a script to make the change.

Are all of the EMC programs written by their summer interns or something? It's amazingly bad. These guys need to read The Inmates are Running the Asylum or Joel on Software to learn how to program useful stuff....

Finally sold the PT Cruiser

My brother-in-law Clay managed to sell the PT Cruiser for me just yesterday and got a good price. That took way longer than I thought it would considering how popular PT Cruisers are in general. I figured that would only take a couple of weeks but it took almost two months to sell.

I hope it's easier to sell the Jetta...

Noise pollution in Tokyo

On thing I've noticed in the Tokyo area, there seems to be a LOT of unnecessary noise. There are pre-recorded warnings, beeping trucks, and even silly recorded bird song at rail stations. Every escalator has a "please watch your step" warning and many trucks have "Be careful, this truck is turning right(or left)" messages that play when they turn on their turn signal. I don't remember as much of it when I was in Japan in the '90s. It must have come later.

Furniture for sale (2)

We've reposted the items we didn't sell

• Daipason 6ft baby grand - $5900
  
• Angel Table - $135
• Vanity Table with Mirror - $150
• White wood bench with storage - $135
• Blue china hutch - $135
• Very large black mirror - $200
• Magnavox 27" TV - $40
• Antique-finished white cabinet - $60
• Aluminum outdoor furniture set (tables, four padded chairs, and umbrella) - $175
• 48" round, center-post table - $500

We have a bunch of other stuff, too...

Lost some weight

I managed to loose four kilos (nine pounds) since I arrived in Japan. It's kind of surprising since Chiho's parents seem intent on stuffing me full of food every day. I think it's all of the walking you just have to do in Tokyo.

... of course, I still need to loose another fifteen kilos or so to get even vaguely close to the weight I was when I got out of the Navy. Wish me luck...

Packout date: April 17th

We've picked the pack-out date for the move. The moving company will show up on April 17th to pack all of our stuff and then load it on to the containers on the 18th. Chiho will need some help the week before but probably more the weekend after. We'll have lots of things that we aren't bringing that will need to be organized and taken care of. Volunteers welcome...

Train delayed this morning

On my way into work this morning the train was delayed about 15 minutes because a big tarp from a construction site blew onto the tracks. Since the express trains are about 10 minutes apart and there are locals scattered in between, that 15 minute delay is going to cause a lot of pain for the train company.

I bet there are some irritated JR officials that are going to have a harsh word with the construction company this morning.

Japanese class at work

There is a nice benefit at the new office - free Japanese classes. There are five levels of the classes and each of them is one hour/twice a week. I'm taking an fairly advanced class (4th level out of 5) and it is really tough. The teach thought I was too advanced for the level below so I'm taking the next one up.

I hope I can keep up...

外人は日本語を話すできない?

I've been finding an interesting gai-jin (foreigner) effect today. It seems that Japanese people automatically assume that foreigners don't speak much Japanese. At restaurants or stores, when I start to speak in Japanese, they seem to have a lot of trouble understanding me, even though I am quite sure that I'm pronoucing things correctly. It seems that they are expecting me speak English or something.

Forging a global team

Now that I've been working for a couple of weeks, it's interesting to see the group dynamics of a global team. Since the key players in my group are on Tokyo time, Paris time, and East Coast US time, the communication problems are interesting. We currently have a weekly telephone meeting on Wednesdays and the only time that works is 9:30pm Tokyo which ends up late in the afternoon in Paris, and early in the morning in America. That's kind of annoying for us, especially since these staff meeting run about two and half hours.

But the time zones aren't really that bad. We seem to have the usual collection of highly-opinionated techie types that gather in any large IT group, though. There seems to be "territories" of responsibilities (i.e. "my data center is mine", "all the MOM servers are his problem", etc.). That was a bit of problem at ISOutsource, I suppose, but it seemed to be pretty minor. Unfortunately, it seems to be interfering with smooth operations. We have enough people in enough places to do a true follow-the-sun model (I can work on things at 10am that is the middle of the night in Paris) but we can't pull that off if people don't want others working on "their" servers.

It will be interesting to see how things will work out. The managers have been kind of disconnected from things because they have been working on a big reorganization. That process is pretty much done so we'll see what changes in the next couple of weeks.

VMWare limitations

The new office uses VMWare's ESX infrastructure for the vast majority of their servers and there are a lot of cool features about the product. We have a fiber channel SAN at the data center and all four VMWare ESX host computers can view and see the same volumes. This allows you to move the VMs on the fly from host to host if one of the VMs needs more processor power or something. you can use the same trick, called VMotion, for maintenance. You can slide the VMs off of one host, patch and reboot, and the move them back without any downtime.

However, I am firmly convinced that there are a couple of limitations:

• Domain Controllers do not work well as virtual machines. All DCs should be actual servers.
• SQL 2000, SQL 2005, and other databases should only be run on actual servers.

The company a whole has also decided that all back-end Exchange servers should be actual servers instead of virtual ones. I haven't experimented with that to decide if I agree.

We have two Microsoft MOM 2005 database servers and a couple of other databases for web-apps, all of which are virtual servers. They have been nothing but trouble. We have been adjusting virtual CPU and RAM settings, changing the ESX host settings, and have been trying every trick that VMWare recommends to us and they all run like crap. At low loads, everything is fine but as soon as the SQL load increases, the performance gets very bad, very quickly.

So, save yourself some pain and make sure every DC and every database server is running on real hardware.

No crackberries at the office

At ISOutsource, we all had Blackberries as our primary e-mail tool and that always-on, always-ready access to e-mail got really addicting. My new company does not use Blackberries but they do support Windows Mobile devices. I was curious about why they only did Windows Mobile and it turns out the Blackberry support in Asia is pretty limited. The low-pixel count screens that the Blackberry uses (which give it such a good battery life) do such a horrible job of displaying Japanese, Chinese, and Korean characters that they just aren't popular. Also, the company wants to only have one set of infrastructure.

But Windows Mobile hasn't caught on much in Japan and was only really just available last year. The various cell phone providers don't seem very interested in them and the phones are kind of expensive. There doesn't seem to be an equivalent to the Blackjack that is available in the states. The phone they gave me is from a hardware manufacturer called HTC. It's a slider style where the keyboard slides out the side. I snagged this picture from the HTC site:

It works fairly well but there is one irritation that you can't tell from the photo. The brick is amost 3/4" of inch thick. So the face plate is quit a bit narrower than a blackberry but that thickness makes it really awkward to carry around. for all intents and purposes, you can't put it in a coat pocket. The belt clip/pouch thing that comes with it is pretty bad, too - almost unusable. I leave it by my laptop bag, instead, but that doesn't work for the weekend.

The screen is really good, though. I've installed Microsoft Reader and copied over a bunch of free e-books from the Baen Free Library and read them on the way to work. That nice, bright screen comes with a price, though. I rarely can go more than 24 hours on a battery charge. I do have a spare battery but that is something else to carry.

Access to e-mail is pretty smooth, though. The screen is wide enough to read and write a reasonable e-mail. The full access to the company e-mail is pretty nice, of course, so that satisfies my crackberry addiction.

What I actually do

Both Chiho and Chiho's mom have been asking for some sort of short description of what it is I actually do at work. When someone asks them "so, what kind of job did Matt take in Tokyo" they can't actually answer. All technology workers have probably come across this problem because it doesn't distill into a single word like "lawyer" or "architect". I suppose if I was a programmer, I could come a lot closer to a single word description.

My group is called the Collaboration Engineering Group (CEG) and we make sure that e-mail, chat, and SharePoint systems are up and running. What I do is broken into these parts:

• Respond to problems that people are having with e-mail, chat, or SharePoint
• Maintain, backup, test, and patch the 30+ servers that run those systems
• Plan for improvements and major changes in these systems

It is probably a 20%/30%/50% split at the moment between those parts.

I thought I would be able to write more often...

I am definately not posting enough on this thing. I'll try to get better at posting but I'm still not completely adjusted to the fact that I don't get home from work until 7:30 or 8:00pm and basically go immediately to sleep. It's odd how long it takes for your body and mind to get back in rythm.

Chiho's flying out to Japan today and will be staying for about ten days. We should be able to finalize the apartment arrangements on this trip and she's already scheduled a couple of evenings out with old friends. There is also a national holiday in Japan next Thursday so we can goof off somewhere then, too. She annouced to the office that she's leaving the company on April 4th and we've picked April 17th as the day to start the official pack-out. Still lots of stuff to deal with before the pack out. After the pack out, too. I have a feeling that we'll still have things left after the final move that we'll need help dealing with.

We haven't got much traffic on selling the house so it's out on the rental market now. We're using the RPM/RPA guys and if you know anyone who wants to rent, pass around this website address: http://www.rentseattle.com/RentDetail.aspx?ID=1009

I'll try to write more in the morning about the job and other stuff but I'm starting to zone out for now...

Furniture for sale

Prior to leaving Seattle, we have a bunch of stuff to sell. Most of it is posted on Craig's List and I'll be putting the rest up when I get pictures:

• Angel table $135
• Small angel side table $15
• Small vanity table with mirror $150
• White storage Bench $135
• Blue china hutch $135
• Blue, tall, skinny chest of drawer $175
• Very large black mirror $200
• Dark wood hall table/sofa-table $70
• Green padded bench $40 - we sold this one
• Magnavox 27" TV $40
• Stereo Sony $60
• Antique-finished white cabinet $60
• Aluminum outdoor furniture set (tables, four padded chairs, and umbrella) $175
• Outdoor chaise lounge $150
• 48" round, center-post table $500
• White curio cabinet $175 - we sold this one

Two weeks in and things are going well

I just wrapped up the second week of the new job and things are going well. I'm pretty much up to speed on current systems and procedures and am starting to get going on things. They are doing a really big Microsoft Office SharePoint Server roll out for the organization and I think I can help out there the soonest. I'm still getting use to the differences in size, though. Also, the office tends to be start-late/run-late kind of office where most of the work happens late into the evening and I've been an early morning person for a while. It seems unusual to get to the office 9am and only 10% of the office is in. I think it's a natural reaction to coordinating things with the Paris HQ since they only overlap Tokyo in the late afternoon.

The commute to work isn't bad. I experimented with a couple of different routes and I picked the one that is a little slower but less crowded. Part of the trip is on the Yurikamome ゆりかもめ line that most people think of as a "toy" train to get to the Odaiba お台場 area. It's a little slow but I can sit the whole time. Door to door, the trip takes about an hour if I hurry, but most of time I sit at Toyosu 豊洲 for coffee and take some time to study Japanese.

The company offers free Japanese lessons so I've started taking a Tuesday & Thursday class at lunch time. My current skill is in between their current classes so I'm taking the higher level one which is really, really, tough.

Simple photo that describes the limit of the Mac AIR

Check on the picture at http://blog.guykawasaki.com/2008/03/airlegance.html. That is probably the simplest description of the one big limitation of the Air...