Channel News and Analysis - Channel Insider
Empowering the next generation Channel
 

Bull’s Eye Awards
Nominations Open for Channel Insider 2009 Bull’s Eye Awards
Nominations are now open for the Channel Insider 2009 Bull’s Eye Awards, which recognize excellence in customer service, technology prowess, business acumen, channel leadership, communications and community building, and innovation among vendors, solution providers, distributors and channel services companies.



Sponsored Links
  • Control VM Sprawl, What You Don’t Know Can Hurt You
  • FREE Sophos Encryption Tool: Encrypt, compress and share files easily
  • LSI 6Gb/s Portfolio Expands to Include SATA+SAS HBAs
  • Reduce the cost of managing your mobile workers.
  • Find out 7 Ways to Drive Data Center Efficiency
  • SonicWALL breaks through network and email gridlock
  • Save up to 40% on calling costs with Avaya Aura™



  •  

    House of IT Horrors

    in Channel News and Analysis


    Article Rating:starstarstarstarstar / 0
    Article Views: 783

    Rate This Article:
    Add This Article To:
    Who needs ghosts and goblins, haunted houses, and smashed pumpkins, when sometimes your most harrowing horrors are only steps away from your technology desk?

    Ghosts and goblins, haunted houses, and smashed pumpkins—sometimes the most harrowing horrors are only steps away from your cubicle or office.

    Just ask Steven Calderon.

    Calderon was in his second week working as security guard for Fry's Electronics when Anaheim, Calif., police walked in and arrested him for crimes including child molestation and rape.

    Fry's had requested a background check on Calderon, which was done by The Screening Network, a service of ChoicePoint, the $1 billion-a-year data broker based in Alpharetta, Ga. When it came up with criminal warrants and felony charges, nobody—not Fry's, not the police—stopped to ask if the data supplied by ChoicePoint was accurate.

    Calderon spent a week in jail for crimes he didn't commit because an identity theft report he'd filed in Norwalk, Calif., in 1993 wasn't connected with the criminal files that were created in his name.

    He went to jail for an IT error.

    While, fortunately, not all IT disasters are of these cataclysmic proportions, every weathered IT professional has an all-too-real eerie tale about a day when everything went wrong. Even 10 and 50 years later, these pros retell their horror stories with a startling sharpness, more haunting than ghost stories.

    Resource Library:
    Invasion of the Inept VP

    John Mitchell (not his real name), a senior support analyst with an insurance company in Madison, Miss., like many worker bees, had his most chilling IT calamity at the hands of a higher-up who believed he had it all figured out.

    "A senior VP, who fashions himself a 'programmer,' decided to install the latest version of Visual Studio 2005 before we, the IT staff, had a chance to test it. Once installed, his code no longer worked. His solution? He uninstalled a previous version of Visual Studio," said Mitchell.

    Peter Coffee looks back at almost 25 years of IT screwups. Click here to see the slide show.

    Mitchell's IT group came to find out that he had not one or two, but three different versions of Visual Studio on his machine.

    "Yes, we should be able to control this since we're IT, but we've been warned against 'stifling a user's creativity,' which loosely translates into 'He's a VP. What are you going to do, tell him no?'"

    Adding insult to injury, the VP put in a Severity 1 trouble ticket, which was supposed to mean that he was incapable of continuing the duties of his job until it was fixed, when it was really more of a Severity 3. Mitchell's group, however, had no choice but to drop everything and make this machine their first priority.

    "Our programming staff attempts to decipher his code but can't because it seems they're not trained in early Egyptian hieroglyphics," said Mitchell. "It then gets transferred to me. I turn his machine inside out trying to make this thing work. I uninstalled and reinstalled every VB application on his computer multiple times, manually cleared the registry, Googled every variation of 'broken Visual Studio 2005' I could think of, lit a few candles, said a few prayers, but nothing worked.

    "Meanwhile, the VP, who took the day off, made sure to phone every 2 hours for updates and to offer 'suggestions.' For two solid days, I monkeyed with the problem, but couldn't fix it. His code continued to crash."

    So what happened? Mitchell came in to work on the morning of the third day to a voice mail from the VP who was driving him up the wall, in which he said that while he had been at the office that weekend, he realized that he'd mistyped a character in his data collection string. Once he made the change, the code worked perfectly.

    "'You can close the ticket,' he told me," Mitchell said. "Fortunately, my office has double-insulated walls; if not, I'd be telling you this from the unemployment line."

    Next page: Dawn of the Dismal Data Conversion.

    Dawn of the Dismal Data Conversion

    Stuart Robbins, founder and director of The CIO Collective, a nonprofit association of senior IT executives providing strategic guidance to emerging businesses, and author of "The System is a Mirror" (John Wiley & Sons, 2006), had his worst professional nightmare come true almost 10 years ago, when his group attempted to migrate from a very old to the then-current version of Sybase.

    "We were attempting to leap six versions [4.x to 11.x] because our third-party vendor promised it was the only way to maintain the viability of our Scopus data, which depended upon the Sybase upgrade," said Robbins.

    So, just how badly did it go? Cataclysmically, it could be argued.

    "There were so many problems that we needed a triage team dedicated to resolve issues that we had caused by the work on the previous day, and after delays of more than a year and many hundreds of thousands of dollars over budget, we backtracked and moved the data to SAP," said Robbins.

    Robbins added that in almost every nightmare project he's heard about from colleagues, custom data conversions, such as his Sybase disaster, were the root cause.

    "They never work and should be outlawed," he said. "No matter what the vendor promises, major vendor changes involving critical corporate data should simply be re-entered by hand using inexpensive Kelly Temp resources."

    Night of the Living Zero-Divide

    Jerry Luftman, now an associate dean and distinguished professor at Stevens Institute of Technology's School of Technology Management, was a young IBM systems engineer supporting one of the major television networks during the 1976 Carter-Ford presidential election when disaster struck.

    "Like today, even in ancient times, everyone was glued to their televisions (yes there was color, but we did have to get up to change the channel) watching the returns unfold. The networks competed heavily on who could predict the outcomes the fastest and the most accurate," said Luftman.

    A new security survey reveals that the majority of IT pros are most afraid of hacker attacks. Click here to read more.

    Suddenly, at the network where Luftman was supporting, each of the three mainframes—one primary and two backups—within 5 seconds of each other crashed at the peak hour of 9 p.m.

    "At first, the team from IBM and the television network looked at each other in fear and puzzled. We then looked at the TV monitor and saw that our [television] star did not skip a beat. They kept looking at their terminal, and without any sign of concern kept relaying the status of the election," said Luftman.

    As the stars were reciting the election results from memory, Luftman's team scrambled to find the cause of the outage, frantically trying to figure out how to get the systems back up quickly.

    "Within 5 minutes the core dump was printed and we were analyzing the problem. There was a local election in the Southwest that had a candidate running unopposed. Did you figure it out yet? There was a 'zero divide,' and the systems just could not tolerate it," said Luftman.

    As the clock still ticked and the broadcasters continued to present the results, someone came up with the ingenious idea to add a fictitious candidate to the local race and give them one vote.

    "It worked. The three computers came back up, and the broadcaster had current information on their screen. ... And we were the first to predict that Carter won, and we were the most accurate."

    Additional reporting by Debra D'Agostino, Debra Gage and John McCormick



    Discuss House of IT Horrors
     
    >>> Be the FIRST to comment on this article!
     

     
     
    >>> More Channel News and Analysis Articles          >>> More By Deborah Rothberg
     


     


    [ci] feeds
    XML
    Add Channel News, Product Reviews, Trends and Analysis to your RSS newsreader or My Yahoo!


    HTML PLAIN TEXT

    Keep on top of news for VARs and Resellers with CI's Weekly Newsletter and Alerts.

     


    CHANNEL RESOURCE CENTER
     
     
    Enterprise Mobility Zone
    The Enterprise Mobility Zone (EMZ) blog is a tool designed to help senior IT executives discuss, create and deploy next-generation mobile strategies in their organizations.
    Go beyond yesterday's tactical approach to mobility!
     
    Build A More Efficient Data Center
    Demands are growing but budgets are not. Solve your pressing IT issues using the resources you already have. Determine which technologies can help you drive efficiencies and how they are applied. Gain a quick ROI on new initiatives
    Find out how
    Let Enterprise TechBrief do the work for you. Aggregated content, tech news, product reviews, vendor updates, how-to’s—all you need to boost your efficiencies and cut costs, all from one place.
    enterprisetechbrief.com