The Deliverator – Wannabee

So open minded, my thoughts fell out…

Speakeasy – No Last (2) Mile Love

Posted by Deliverator on July 15th, 2006

I’ve had a frustrating couple of weeks, due largely to circumstances beyond my control. It has taken much longer than expected to get the server moved into Chris Flugstad’s office’s basement wire closet. He is giving us a very good rate, but I am beginning to think that I should bite the bullet and go with a more professional hosting solution. I would probably have to sink some $$$ into some higher capacity hard disks to get the server height down from 6u to 3u and get some rails. Paying for 6u of rackspace anywhere would not be fun. Chris doesn’t care too much about height at his office, but professional colo facilities tend to charge by the U, as well as by power usage and bandwidth. Local surplus server parts dealer Nautilus (who sold me the raid adapter and RILOE 2 board) has everything I need. I never like to do something half-baked. I would rather sink some cash and time to ensure that the server is easy to maintain in the future. Problem is, I already sunk some $$$ into the server with the assumption that I would be hosting in a space tollerant environment (either with Chris or at Ryan’s house). Now I am beginning to think I should have gone with a 2u server like a DL 380 G2 coupled with higher capacity drives. I think I will give Chris till the end of the week to get his closet cleaned up and then I will start looking at other options.

The other frustrating project that hasn’t gone as planned has been the Speakeasy DSL migration. I have needed more bandwidth for quite some time and was ready to make the switch to Comcast, but Speakeasy assured me they could get me more bandwidth by moving me to a ADSL dryline. They scheduled an install date and I rearranged appointments to stay home from work on the available date. This wasn’t too convenient for me, as the date happened to land on a day a client of mine was opening a new business. I waited as long as possible and then headed to my clients. I managed to get my parents to cover. The install tech never showed, but nobody (our dedicated install coordinater included) bothered to call and cancel the appointment or even just tell us they were running late. Turns out, QWEST had come by a couple days before and told Covad that “the customer’s phone lines are on a CSU multiplexer and there are no free pairs” and that we couldn’t get service, so Covad didn’t bother to show up, nor did they bother to tell anyone they weren’t showing up. Thing is, we already knew about the CSU and had told the installation coordinator that we couldn’t put the circuit over the same lines as the one that carries the voice circuit, but that we had a free copper pair available which was being used to run our SDSL service. All they needed to do was switch the pair at the pedestal to the new ADSL circuit at the pedestal and we would be good. The only concern I had was that the switchover occur in a coordinated fashion, so that I wouldn’t have to deal with much downtime. They scheduled a second installation date. A few days later, I got a call from QWEST, obviously coming from a call center in India, due to the accent and multi-second time lag. Some girl reading from a script informed me that a tech had concluded that my order couldn’t be completed and that she was cancelling my order, unless I wanted to pay QWEST to dig a trench and add more copper pairs. I tried to explain that we had enough copper pairs and that the tech (who I don’t think ever came to the house, but merely looked up our info on the computer, as our pedestal is in the front yard and the SNI is in the normally locked garage) had made a mistake. It took a couple escalations to get them to get them to admit that they might have been mistaken and to get them to send someone out to the house. The tech figured out that I was in fact right in the first place and offered to switch the pair over to the new circuit. I called the install coordinater and had a lovely four person chat to make sure that everyone could do what they needed in a timely manner. Covad switched an ADSL circuit to the pedestal and basically promised to come by the next day to troubleshoot and issues and finalize the install. The Qwest tech switched over the copper pair used by the SDSL circuit to the new ADSL circuit, I plugged in the modem from the self-install kit, put in one of my static IPs and I was online. Golden! So, I sent the QWEST tech home and Speakeasy told me that Covad would be by tomorrow to double check everything. A few minutes after I hung up, I noticed that my 3 mbit circuit was only delivering about 1.4 mbit of downstream bandwidth and only about 400 kbit of upstream. On top of that, the DSL modem was dropping out every few minutes and re-handshaking. I called Speakeasy up and they confirmed that they were seeing lots of line transmission errors, but not to worry, Covad would be by tomorrow to troubleshoot and yes, there should be no problem getting you the full 3 mbit despite being 10,700′ from the CO. Tomorrow came and went without Covad showing up and I called and was informed that the soonest they could come would be Friday. By this point I was getting pretty pissed off. The lack of communication, failed promises and left hand/right hand hijinks from all three parties involved really left me pretty frustrated. This was a business line and any downtime was totally unacceptable, especially given reassurances from all parties involved that the transition could be handled relatively seemlessly. Several fake escalations later (in that they resulted in a lot of soothing words and no results) and I finally got a Covad tech to come to the house, 5 days after the botched migration. He was supposed to arrive somewhere in the 8am-12 window and he arrived at 3:30 and was very eager to get his weekend started. He determined that the signal to noise margins were very low at 3 mbit, so tried 1.5. No dice. He eventually bumped it down to sub megabit on the download and 256k on the upload, which stopped the DSL modem from resetting and retraining the line every few minutes. Actual throughput is about twice what it was on SDSL on the downlink, but is almost half the speed on the uplink. We also tried connecting the modem directly at the SNI, but that only yielded improved margins of about a db. The burried service wire between the house and pedestal is probably 30 years old and original to the construction of the house. They didn’t exactly use plenum grade wiring in those days, so there is chance that the wiring between the house and pedestal is at fault, so Qwest is going to come out and run a overground wire between the pedestal and house and see if that fixes the problem, as well as check for bridge taps and other things that could degrade the signal. Other than that, my only options for improved bandwidth may be a faster SDSL line (which would cost upwards of $300/month), or switching to Comcast. Comcast has rediculously bad acceptable use policies and frankly I trust them about as far as I can shotput one of their service trucks, so I would probably provision some bandwidth elsewhere and tunnel all my traffic over it. My only other thought was to go knocking on doors of people on College Hill across the valley from my house on Woodridge and offer to pay for someone’s internet service in exchange for mounting some WiFi gear on their roof. College Hill is about a mile closer to the CO, so it should be much easier to get high speed DSL there. I have done some test shots to College Hill in the past and it was surprisingly easy to get a stable link with a very strong signal.

* begin paranoid depressed rant *

In the US, the “last mile” problem is an absolute pisser, yet other countries, which don’t have governments whose only purpose seems to be to protect the status que & existing business models, etc. seem to be able to actually get things done. The result is that the US is turning into a backwater for broadband and is loosing the technology lead in countless areas. Weren’t we supposed to be trading our manufacturing base for a high-technology leadership role? At least, that is the sound-bite that the last few administrations have been spouting at every opportunity. Instead, it seems like we have traded both and we will just be sipping our Starbucks and driving our Luxury SUVs until one day we realize that the oil has run dry, there is no money left in America (at least not for the shrinking middle class, at which point we begin class warfare all over again) and the debt collectors start foreclosing on all those variable rate mortgages that soo many of my friends thought were a good idea not so long ago. Oh, and nobody seems to remember how to get all that aging infrastructure to work, much less build new infrastructure (big dig, anyone?). After spending the last few weeks on the phone, on hold, trying to get 3 companies to remember something about yankee ingenuity, while listening to some basso-continuo background music of blood running in the streets in the middle-east, it is hard not to feel pessimistic about the future. Well, if worse comes to worse, my family has land, when the money doesn’t work, guns for when the neighbors start eating their own, and a few genuine Samurai swords for when the bullets run out.

* end paranoid depressed rant *

So, someone want to cheer me up?