-> ^ < -

Tin Cans and Fibre-optic String

"The wonder of a dancing bear is not how well it dances, but that it dances at all!" (unknown)

It seemed like a wonderful title and an appropriate quote for the way I felt when I sat down to write this article. I have spent the past several months dealing at both ends of the spectrum with a telephone company - in fact a couple. The level of frustration in the dealings with the sales and installation force for POTS (Plain Old Telephone Service) in this age of electronics leaves me worn.

In their defense I can say that the people who actually did the work that was ordered (after much gnashing of teeth and breaking of pencils) did a first rate job. The fact that in one case, the job was a month late and billed incorrectly was not their fault.

We are a multi-facetted company. Our original business was computer consulting and custom UNIX driver development. Along the way we became the focus point for all the Usenet and Internet E-mail to BBS systems in the area. Most recently we have taken our technical skills and created what we believe to be one of the best local ISPs (Internet Service Providers) around, and developed a pretty impressive WWW (World Wide Web) site - the one you are reading this on.

POTS and Pans

Our struggles on the POTS side began late last summer when the CRTC (Canadian Radio Telecommunications Commission) changed the rules on the use of Business Measured Overline to limit the number of lines on a hunt group to 3. Since virtually every BBS and ISP in BC with more than 1 line used this facility because it was less expensive than full business lines, panic ensued. The alternative, regular Business Overline service, was over 200% more expensive. Services that charged were predicting price increases while free systems contemplated cutting back the number of lines or closing altogether.

I'm not sure who came up with the alternative that most of the local BBS and IP providers have by now embraced, Centrex, but it wasn't long before those who could justify the minimum 50 line buy-in had consulted with the telco sales force and placed their orders. Ours was placed and we were given an install date of October 24th 1994. The interesting thing about all this is that when we had opened our new office in February we had contemplated Centrex as a technology that would allow us to continue our "distributed office". The Centrex offering includes a 56K data link which would mean faster access to our main system for our support people who work from home. In addition, the main reason for Centrex, the equivallent of a distributed PABX with individual locals and such all at separate locations, would enable us to transfer support and sales calls without having to resort to re-dialers or special OPX (Off Premises Extension) lines. We turned it down at that time but were now going to embrace it in a big way.

The October 24th date was significant because the tarrif change on the BMO lines was to kick in on November 1, and we didn't want to have to pay the higher fees for even a moment. We were assured that even if we didn't get the early install date, the fact that we had our order in was enough to forestall the higher charges. It turns out that we probably should have had this in writing, because the ensuing foulups and delays meant that some of the lines were not cut over until January!

We were changing every line we had in our office, voice, fax, modem; over to the new system. In order to make the transition as simple as possible the telco offered to put all new cable pairs in at the outset so we could do the inside work all at once without having to wait for them to do any exchange work, an admirable goal, but one it turns out they couldn't fulfill. We had over 60 lines all told at two locations, and that meant they were going to run in over 60 new pairs. It turns out that they could only find 42.

Delays and more Delays

Not only could they only find 42 new pairs, it took them several extra weeks to find even those. Now this would have been alright if they had told us at the outset that the install date was to be November 22nd, but they didn't. Instead, they pushed the date back a couple of days at a time, often with no notice to us. Where we had set aside several days in late October to do the inside wiring and cope with the expected training and problems, by the end of November when the lines were actually activated we were up to our ears in projects which simply could not be put off.

The local magazines that had our advertisements in them came out at the end of the month. These ads had our new voice/fax numbers, and since they were not yet installed we figured we were losing calls. We decided to have the new phone numbers call forwarded to the old ones. We even offered to pay to have it done. I called the business office and explained my request. The retort I got was something to the effect of "but we can't do that, the number hasn't been installed yet and so has no line on it!"

Exasperation!

How do you explain to a person who is simply a sales droid that the computer that runs the phone switch not only doesn't care if there is copper wire attached to the terminals, it can't tell. So instead, you ask for their supervisor, hoping that this higher-up will have a little more understanding of the problem and its very simple solution. Ah! success - well, sort of.

It turns out that instead of putting in the simple call forward that was asked for, they put the old operator intercept message on - "The number you have dialed has been changed - the new number is..." and our old phone number, not the message we wanted at all! We didn't discover this turn of events until after the 4:30PM closing time of the business office, so I called the business repair number hoping that they could do something.

Fortune

I guess, looking back on it all from the priviledged position of the future, this was the most fortuitous call of the whole affair. I got to know the evening repair supervisor fairly well over the next few months - and he saved my bacon several times. He got to know that I had a fair idea of what the phone system was and could do, and that my requests were reasonable in the circumstances; I wasn't just another dumb customer. In this instance he listened to my tale of the operator intercept and my request for a simple call forward, and quickly removed the former and installed the latter. He even admitted that because this was a Centrex switch he was dealing with, that they had had to get out the manuals to figure out how to do it. He knew it could be done, agreed with me that it should be a simple operation, and that this is what I had asked for - the account told the story.

One of the intersting side effects of them looking at the account was that I learned the cutover had been put off again - this time to the 18th of November. This really pleased me (not) because it was the 8th already, and I had cancelled my Remebrance Day long weekend plans so I could be there on the 11th, the day they had last given us.

Another problem that kind of crept up on us with all these minor delays was that we were behind on our line expansion plans. We had been adding an average of 6 new lines per month for three months prior to October, and one of the reasons we put in the Centrex order when we did was that we were told that the telco wasn't taking any more orders for expansion of the old BMO lines, unless we converted right then and there to the higher priced BOS and paid the new higher rates NOW instead of on the mandated cutover date.

The result was that we decided to change our cutover plans. We had originally decided to leave 3 lines on each of our old overline groups so that slow converters would at least get some service. Now we told them to simply cut all the lines over to the new numbers and put call forward on the old ones. We were to pay for the call forward, at the outrageous price of one normal line charge for each call forward, but the freed up pairs would mean 9 more lines on our new overline group. We also increased the number of lines requested for our new 28,800bps group by 10. All this was agreed to by our sales rep and because we were simply re-using more pairs of our old lines in addition to the new ones, there was no extra cable to find - only changes in the CO programming. Again, it turned out to not be quite so easy as there was the matter of switching line cards to the Centrex ones from the standard ones - but that was going to be done on 18 of the original order anyway.

By the second week of November we were fielding hoards of calls from customers saying the new phone numbers we had carefully told them about leading up to the October 24th cutover date were not working and "what is happening?" A year earlier we could have brushed this off to our highly technical customers and they would have found the information in our MOTD (Message of the Day) file, but now we had 6 times as many customers, and they were highly non-technical. A change of phone number in a program designed to dial in and log on automatically to a service that the customer doesn't understand, using a technology that they think is magic, throws a wrench into even the best laid support plans - and ours were already strained with the 600% growth we had to date in the year.

We Were Learning

This request went forward faster since I now expected the "you can't do that" line and pre-emptively asked for the supervisor. I had originally called for our installation coordinator to have this done, but I didn't get a call back from him, and found out that his whole office was out on a wildcat strike of some sort. I should have seen the next bit coming!

November 11th came, a holiday as previously mentioned, and I went into the office to catch up on some course preparation instead of going off for the weekend; lucky me. The phone was ringing when I unlocked the door, unusual for 7:30 in the morning. The caller said he was having trouble getting onto our system, and asked if things were really busy. I looked at our wall of modems and said "no, only 3 lines are lit up - you should have no trouble." He had two lines and so could dial and talk at the same time, he hit a busy.

Blindsided

I'm pretty quick on the uptake and realized that the telco had programmed the cutover of our lines into their computer some time in the past and noone had bothered to change it when the date for the change had been moved back. We were down to only 3 of our whole bank of 14,400bps lines - the 3 that were originally supposed to have been left on it when the cut was done. The rest were dead, CO battery but no dialtone, and no ringing. It took me until 11:30 that morning to get the programming changed back to what it had been; partly to convince the repair service that the date was wrong, and the rest because it takes time to upload the programming after it is changed. Again, I learned from repair that the date was now not the 18th, but the 22nd of November.

On the 17th and 18th of November a couple of outside installers showed up and ran in 40 of the expected 42 pairs. The last 2 were labeled but didn't work yet. They also ran in a whole new set of demark hardware. The previous month when we had put our inside wire in we had tied it down in the same fashion as the previous 100 pairs. Apparently this wasn't good enough now. The new stuff added 3 new connection points, each with the potential to either fail or add noise, all in the interests of making it abundently clear where the telco lines ended and ours began. I had evening functions all week and the 18th was courses and seminars all day and into the evening. We didn't have any time for playing with the phones, and I told the installers to make sure that nothing was done until I was called on the following Monday - the day they said was our cutover day.

On Monday, without any warning our voice and fax lines were cut from the old numbers to the new. I spent a hectic morning reprogramming our PABX to handle the new trunks and changing jumpers and such. I tried to get our coordinator but was told he wouldn't be in until Wednesday!

They did it again! This time they cut our smaller 28,800 group back to 3 lines from the 6 it had. Not nearly as devastating as the previous cut on our 14,400 lines, but a pain none the less - and this time at 6:30 in the evening! My friendly supervisor at the business repair centre took care of it with a single frantic call between two classes.

On the 23rd I came into the office and was informed that the phone was ringing off the wall with reports that people couldn't get in. After the debacle we had with all our other lines, I assumed that they had cut our old site's lines down to 3 this time. I couldn't get my business partner on the phone, the lines were in his basement, so I went over to his house at a dead run after lighting a fire under the telco people again. It turned out that it was just the normal flood of traffic and the fact that we were still running on 3 months ago's number of lines with this month's load. Got the supervisor on the phone again and appologized, and then laid down that we wanted the balance of the cuts to be done only on receipt of my signature, and would they please fax me a list of what was left to be done.

About this time another installer came in with a single order to install one of the 2 left over lines from the previous week.

Chaos over Christmas

From this point until the middle of January, 2 months later, so many incredible things happened to various of our lines that the details fill a small book. Lines that we specifically asked not be cut because we had to go on-site at several customer locations to personally change the phone numbers got cut with no warning, not once but time after time! We discovered a small overline on our PEP lines was cut back to 1 line and then summarily cut over, all with no notice and no appology. We were charged the BOS rate on our old lines even though we had been promised this wouldn't happen. We got charged installation as a package as well as on each individual line. We got a separate bill for each line, return envelope, message stuffer and all. We got charged a changeover fee to the unwanted BOS rate even though it was involuntary and not supposed to be charged. We lost lines that were not supposed to be cut. We still had dialtone on lines at our old office a month after they were to have been turned off. The list goes on and on.

And the Capper

Now, after all the above, we are told that we use the Centrex lines too much and the telco is going to cancel our contract. Even if they don't cancel it, they are refusing to put more lines in - lines we have been trying to order for 3 months. It turns out that when they added our almost 100 lines to the local Centrex slave switch they didn't add any trunks back to the main switch. This means that the other Centrex customers in our area are suffering because our lines are 80% busy 80% of the time and 100% busy 20% of the time, we eat up almost all the trunks! Another local ISP had to be moved from one switch to another 4 hours after they had their lines turned on. The Centrex customers in their area couldn't even get dialtone - the trunks didn't have enough capacity even for the one ISP, let alone the other customers!

What they do next will make another chapter I'm sure.

Richard Pitt

richard@wimsey.com

Copyright Notice

Wimsey's Home Page

admin@wimsey.com

Copyright (1995) Wimsey Information Services Inc.