Different Directions

I've been thinking about this blog now and then. I don't really see the reasoning behind writing a blog for the most part. Sorry if this is metablogging!

Anyway, I've come up with a new thing to blog about, which I'm hopefully going to be able to keep my attention on for a bit. It's occurred to me that some things in life will be difficult to explain to my son Nate, or I might someday forget I wanted to talk about it with him.

I'm hoping that by writing these things down, I might be able to gather my thoughts for the day, a decade from now, or maybe less, where I need to confront the topics I'll be blogging about.

I have a few topics in mind that I will cover as time permits, and hopefully more ideas will pop in and out as I go along.

-pete

Why you should love LVM as much as I do.

First off, want to start out by saying sorry to the three people who read my blog. Those of you who know me know I only blog about stuff I find truly interesting or useful, so the filter is generally set pretty high.

Anyway, I ran into a bit of a conundrum. I brought home a bunch of big SCA drives to replace my small 14GB drives in my raid5 at home.. At first I was unsure how I was going to go about getting the stuff off the raid5 array so that I could resize it. Would I have to dd the stuff over to another drive? How was this going to work.

Then I found out that since i'm using LVM on the array, I can use the "pvmove " command. pvmove essentially tells LVM "Hey, move everything off of the specified physical volume device onto all of the other physical volumes in a particular volume group. Since I alraedy had a 1-off 146GB scsi drive laying around, I was able to do this in a few simple steps:

1) install 146GB scsi drive and rescan scsi bus
2) pvcreate
3) vgextend
4) pvmove
(wait an hour)
5) vgreduce
6) pvremove (not really necessary if you're just gonna nuke the drive)

Voila! All of my running vm's are unaffected except for a slight slowdown during pvmove, and now I can reorganize the raid5 array. If I had a controller that supported it, I could technically have changed the raid5 out without even rebooting the system.

Once I had the new raid5 configured, I followed the same steps in reverse to get the vms off of the un-redundant 146GB drive.

This is one of the reasons why I love linux, evms, enterprise stuff, etc. It makes crap like this incredibly easy.

Anyway, back to your regularly scheduled nothingness!

--pete

It's finally hit me. I'm weird. (Oh, and how to use iscsi with free vmware server)

While pondering what my next project would be, I found myself evaluating what I have in my posession. I have two iSCSI SAN's, multiple machines running virtualization software. I've run up to 8 linux servers virtualized to run stuff in my HOUSE.

It's occured to me that most people aren't as balls-out crazy about computers as I am. This is quite a revelation.

Anyway, back to a very important thing that I should note in my blog. It is pretty difficult in the windows version of VMware's free Server product to utilize iSCSI devices as whole disks for VM's. In Linux, however, this is no problem at all. In the place where VMware asks you which physical hard drive to use, just put /dev/sda (or whatever iscsi device you have) and voila, you're utilizing iSCSI. There's really no way they can block you from this, as far as I know.

Also worth noting: DO NOT BUY THE PWSA8701T Intel iSCSI HBA! Well, let me qualify that. It works pretty darn well for $70. The problem is, they stopped supporting the driver for it in linux version 2.4. So, I'm not 100% pleased about it, but my vmware box is now running CentOS 2.1 to make use of the iSCSI HBA. I would have preferred some 2.6 love there, but concessions sometimes must be made.

Also in this week's errata: vmware does not like it when you hot-remove iscsi devices (i.e., rehashing ietd on your iscsi target box). Make sure you login to the vmware console and tell it to retry the disk operation it was attempting to do, or else your machine will "freeze" until someone answers the Retry. I wonder if there's a way to make it auto-retry. Normally in linux, if a drive has an I/O error, it will retry in (60?) seconds I believe. VMware obviously wants to be more nosy and let you know immediately there's something wrong.

Random picture time!

Sunset in November makes some of the prettiest luminous cloud formations. Here's one of them I snapped from the parking lot at work.

Why you should never completely trust commercial products. (OR, never trust past employees of your company)

So, I brought home a scsi drive array from work that was sitting idle with 18G. It was at our colocation center and just sitting in a corner. My boss said I could have it. Yahoo! Free hardware.

That is, until I found "it didn't work." I've spent the last week or two tinkering, fiddling, figuring out if the problems with the drive array were with the card, the cables, the drives, or the case. If I had the drives in certain combinations, the array would partially work. If I had the drives in other slots, the whole thing would produce a loud shriek from my adaptec raid card.

Finally I was so annoyed by this problem that I popped open the case to the array and took a look inside. I was expecting to see resistors for drive ID's and thinking it must be a lost cause to tinker. Much to my happiness, I found that there were indeed jumpers for the scsi ID. What I wasn't prepared for was the configuration thereof.

For those who are not aware, scsi ID's are based on the binary code for 0 through 15. It works something like this:

0000 - SCSI ID 0
0001 - SCSI ID 1
0010 - SCSI ID 2
0011 - SCSI ID 3

And so on and so forth.

You'd expect that the manufacturer of the drive array would realize this to be the case. However, what greeted me was some very interesting math. It appeared that whoever set up the jumpers set them up to use 0,1,2,3 as decimal math as opposed to binary math. So, it was a creatively odd hodgepodge of scsi ID's in the case. I think they would have been able to pull it off, too, except for the fact that one of the scsi slots had real scsi ID 7, (0111, or 2+1+0).

That was the holy shitstorm my scsi card was raising. One of the freaking drive slots had scsi ID 7! For those not in the know, scsi ID 7 is traditionally the ID of the scsi card. When things collide on the SCSI bus with the adapter, weird things happen, (like the card spitting out that it had 32 IBM drives in it when it could only physically have 15).

Anyway, a simple bit of tweezers maintenance and the drive array is good-as-new, and is now rebuilding it's 8-disk RAID5 in preparation for me to migrate my Xen install to vmware.

Yahoo!
---------------------------
NOTE: Frank brought to my attention that it probably wasn't the factory, but the previous people who worked at my company who did this little feat of brilliance. Upon further investigation, the fact that the jumpers are of differing designs lends a great deal of credibility to that assumption. Therefore, Sorry <People Who Made Random Unlabeled Drive Array>, it really wasn't your people who were stupid.

iSCSI Initiator on Linux

I'm sorry I haven't posted this sooner, it's a situation of "when something's this easy, does it even require writing down?" However, I promised it and so I will deliver. Using Linux as an iSCSI Initiator isn't all that much harder than windows. It's just a matter of getting the right command line arguments.

You want to either get a precompiled package of the open-iscsi project, or download and compile it yourself. The open-iscsi software is pretty easy to compile as well, just do the standard make && make install and you're rockin-and-rollin.

The primary tool for talking to the open-iscsi kernel modules is iscsiadm. iscsiadm is a bit difficult to learn at first so I'll give some quick info.

Q. How do I discover targets?
iscsiadm -m discovery -t sendtargets -p <target's ip>

Q. How do I login to a target I've discovered?
iscsiadm -m node -T <target's name> -p <target's ip> --login

This will create a new scsi drive on your system, check dmesg to see where it put it. I actually use udev on elysium to make it easier to find iscsi mounts by chain,id,lun rather than /dev/sdX.

The same command with --logout will remove the drive from your system.

Once it's on your system, you're ready to treat it like any other scsi disk. fdisk, mkfs.ext3 and rock and roll!

ISN/Freenum and why you should care.

I wanted to take a moment and discuss a cool telephony technology that I think everyone should be aware of. Why? It's going to change the world.

From the freenum.org website:
"The Freenum/ISN system is 12-digit-keypad (telephone handset) friendly method of providing mapping between users. While the eventual use of email-style URI pointers is the eventual goal for communications identifiers, it is still the case that the majority of the world's telephony users are trapped using a 12-digit keypad for extended numeric entry, and it is for the purposes of these devices that the Freenum system and ISN dialing was designed as a "stopgap" which may last many years. Using ENUM-like methods and IETF standards but not using telephone numbers, the Freenum/ISN dialing system is designed to initially allow SIP-capable proxies and iPBX systems to connect to each other in a free, open, and protocol agnostic manner over the Internet. Initially focused on SIP voice communications, the platform is hoped to eventually extend to other communications methods as those protocols become more widespread."

What does this mean in plain, simple English? It means that using this technology any person in the world can communicate to any other person in the world for no more than the cost of the bandwidth of the call and a little tech savvy. There are plenty of free VoIP clients (X-Lite, SJPhone, Zoiper just to name a few.) If you have asterisk or know someone who does (or any sip proxy, for that matter) you can likely piggyback on their system for ISN resolution.

Let me explain a bit about how this works such that a layperson might understand. Lets say I have relatives in, I dunno, Cambodia (Or insert random country here.) They have high speed internet access and so do I. How do I call them cheaply? Sure, I could use something like vonage and pay per-minute rates. But why bother? We can use the internet from one side of the call to the other, with no issues.

Now, a lot of people might be saying "But you can just use the remote computer's IP address and put that in the client to connect!" Yes, of course you can. The thing that makes ISN pretty cool, however, is that instead of remembering an ip address or a hostname, you only have to remember a relatively small number that is similar to a phone number. In my case, 230*739 will get my desk at home. In the previous case, you would have to type a line similar to sip:ip1-5-7-2-blah.core1.cambodia.has.long.dns.entries.kh or something. Even if you used their dynamic dns service (that you had to teach them how to use) it'd still be sip:petescambodianfamily.dyndns.org. Wouldn't you rather dial something like 1234*800?

The easiest method to use isn is to get on a free SIP provider such as Free World Dialup. They already have ISN built into their sip proxies and all you had to do was download a sip client and sign up for the free service. Now, granted, you could have both you and your cambodian buddies get free world dialup and never even have to use ISN. But, then you'd be missing out on learning about a new technology in the process that might just make your life easier!

ISN is designed to utilize DNS for it's resolution. When you dial 1234*800, what's really happening on the proxy you are connected to is, they're doing what's called an ENUM lookup to 4.3.2.1.800.freenum.org. This lookup produces (via a NAPTR dns entry) a URI that can be used to call. For example:

pgrace@fmservices:~$ host -t NAPTR 0.3.2.739.freenum.org
0.3.2.739.freenum.org NAPTR 100 10 "u" "E2U+sip" "!^\\+*([^\\*]*)!sip:\\peter.grace@hermes.fierymoon.com!" .

This record in simplified English just tells the sip proxy that it needs to call sip:peter.grace@hermes.fierymoon.com whenever someone dials 230*739 via ISN. The best part is, most people will never need to know how this works. It will "Just Work", thanks to you, the techie of the household.

There are plenty of people on the internet and in the isn community that will be happy to help you setup ISN if you already have a working sip proxy. There are also many people who will help you setup an asterisk box if you have a bit of linux skills and are willing to learn the nuances of voip.

It gets better though.

Lets say you have kids in college. You want to be able to call them from home cheaply, and they want to hit you up for cash regularly. Who wants to pay money for these calls? It's like you're really paying them 102.50$ for every 100$ you put in their bank account.

Luckily for you, a lot of universities have already signed up for ISN. They will be rolling out their extensions via ISN to link into their campus phone networks soon if they have not already done so. What that means is, your child who has an extension in their dorm room of say, 2325, all you'll have to do is dial 2325*<itad>. Lets say your kid goes to Wayne State University in Detroit (Random college I pulled from the ITAD List). Their ITAD is 533. By dialing 2325*533, voila, you have just connected to your irate daughter as she was about to go clubbing with her roomies, and the best part is, you're not paying for the call!

Little Jenny needs some more money? If you signed up for Free World Dialup or any of the other myriad sip providers that support ISN, all she'd need to do is dial <your fwd number>*262 and voila, her nice little neon lightup phone from the 80's has made a voip call to your PC.

There is a caveat to this, and that is that ISN is still considered early-adopters territory. Not everyone listed in the ITAD list has a legitimate termination setup for ISN. Some places have flakey sip proxies. This is definitely more bleeding-edge digital convergence stuff than some people are used to dealing with.

That being said, my asterisk box is always up, go on ahead and give me a call at 230*739!