Yet another list-of-predictions for 2009

Everyone likes to make predictions for the coming year. Here's my baker's dozen of 2009 predictions for system administration and technology in general.

1. Cloud computing will experience a major fail. As a result smaller clouds ("Mom and Pop Cloud computing" ?) will appear, powered by ASP/ISP/Enterprise "cloudlike" computing offerings from large vendors.
2. Further balkanization of the field of system administration. More divergence into "operator" and "analyst" type roles. MIS graduates rejoice.
3. Little progress will be made by ISP's for solving the last mile problem, U.S. will remain in the boradband backwater. Things might look more promising after the Obama administration kicks in with their broadband initiatives but it'll take more than a year. 3G cell service and 4G tech like WiMax and LTE really won't make a major impact unless you live in a test market and even then, you're paying to beta test stuff that will probably go away shortly after the kinks are worked out.
   
4. Video, facbook/myspace, twitter, blogging, RSS will start to surpass email. David Allen already prepping an updated revision of GTD with sections on coping with tweets and FriendFeed.
   
5. One major tech company will fail. IBM? AT&T? Sun? Vonage? Someone is going down. What happens in the aftermath will be interesting in terms of Free Software (will a failure result in a large drop of code from the cathedral?) and in terms of what everyone else does (I expect most companies will pull in risky ventures and focus on "core competencies" ... the funny thing will be that those who don't do that will probably be the most successful ones).
   
6. Configuration management will continue to see increased adoption rates; someone might actually figure out how to do something for unix and windows in one system. Regardless, expect management to start to want systems and networks to be more autonomic and less hand mangled^Wmanaged.
7. Linux won't happen on the desktop, but SugarOS will make play for the netbook market as a way to keep the OLPC project running.
   
8. More hardware will have iPhone-like touch input methods. For example: your fridge.
   
9. Digital Rights Management will see a major downturn in adoption. Consumers hate it, vendors find it challenging to have to support something forever, etc.... My supposition is either more trust with unencumbered formats, with more targeted RIAA/MPAA suits or larger adoption of "leased media" models where you can have all you can eat for a regular fee.
10. Computer security won't get much better, despite some very high profile breeches. Seriously, we're still trying to get patch management and virus mitigation under control. Managing a wider body of threat vectors is nearly impossible right now. See also: prediction #6.
    
11. Corollary to 5: the failure will cause some ramifications in terms of the internet continuing to function. Eg: If Sun poops, that's a lot of unsupported servers in critical roles that will need migrating to something and sitting unpatched and vulnerable. The loss of a major player in the tech sector will cause "availability ripple" as people react.
    
12. Someone will figure out how to do backups and restores right for mere mortals. Apple's Time Machine is pretty good, but it's ecosystem specific and has the usual list of Appleisms. Someone will figure out to take the elegance of Apple's interface, make it work on Linux/OSX/Windows and back up your cloud data too (Gmail, google docs, flickr, facebook, etc..) too. See also: prediction #1.
    
13. Corollary to 6: Configuration management and autonomic computing will spur better monitoring and log analysis. The lack of instrumentation in a lot of technology will finally start to be understood.
    

Making apline see a default CAFile on OSX

One of the problems with doing Linux almost exclusively is that I get real lazy when it comes to small things. Most Linux distros sand the rough edges well enough that there's stuff I just don't remember how to do anymore because "they just work" now, when previously they didn't. Dealing with OpenSSL's default CAFile is one of those things.

Luckily, OSX to the rescue. I've started using OS X as my primary desktop platform for 100% of my duties. This includes email so I need my mail client of preference: pine. For the last bunch of years I've just been using Fedora which has had alpine (and Dag had pine for RHEL in the pre-alpine days.. thanks Dag!). Even when I was using a Mac in the past, I'd ssh over to the Linux box to use pine because it was easier. Now, I'm not ditching Linux, but I am trying some different workflows and working methods out as part of my new job.

So I installed pine with MacPorts, which also installed a MacPorts version of OpenSSL, which also does not come with a default ca-bundle.crt. This causes alpine (and/or most SSL apps in MacPorts) to complain LOUDLY about not being able to validate the cert as trusted. Worse, is that after poking at pine with dtrace it was using what I think is a very linuxy version of the ca-bundle.crt (eg: /opt/local/etc/openssl/cert.pem instead.. ha!).

So to fix (after a few hours of poking around... this stuff is not well documented) do something akin to the following:

• Grab the mk-ca-bundle.pl script from the fine folks at cURL.
• This script is perl, and needs LWP. I'm assuming you're using MacPorts, so
  sudo port install p5-libwww-perl
• Run mk-ca-bundle.pl
• sudo -s
• cd /opt/local/etc/openssl
• mkdir certs
• cp /path/to/generated/ca-bundle.crt certs/ca-bundle.pem
• ln -s certs/ca-bundle.pem cert.pem
  
• cd certs
• c_rehash .
• ln -s ca-bundle.pem ca-bundle.crt
  

After which MacPorts bianries (including alpine) should have a nice large CAFile by default and much of your "O. M. G. YOUR SSL IZ BUSTED" errors should go away.

You'll also note that I rename the bundle with a .pem extension and then may a symlink for the .crt pointing at them .pem. I do this mainly because I'm kind of anal.. the ca-bundle.crt that is generated by mk-ca-bundle.pl is full of PEM formatted CA certs, as openssl expects. So I like to be extra descriptive about what exactly the file is, but lots of stuff seems to maybe expect a ca-bundle.crt as well.

Sigh. Security needs be easier than this.

Here we go!

So Long Archway

Damn.

I knew Mother's was going out of business, but I didn't know that Archway was the same company and that there would be no more Archway cookies either. I loved Archway cookies when I was kid, I thought they were so special because there were less in a pack than Chips Ahoy. Or something. Still, I'm sad to not have them anymore.

Damnit people, think of the cookies! Fix the economy, vote for Obama.

Vote.

I ALREADY DID.
















http://www.designforobama.org/index.php?p=102&page=2&sort=tv

Fall!

Wheeee!

It's fall in Milwaukee and the colors are great. I took some pictures last week, and they're linked below. After being in the south for 10ish years, it's nice to see trees turn a color other than a bland murky brownish-grey.

We've also gotten the chance to do some other great fall things like: roast a turkey breast, have apple cider, light a fire in the fireplace, and go get a pumpkin from a local church that was selling them on their front lawn. We're going to try to carve the pumpkin today before the trick-or-treaters show up. It's supposed to be cold (windy, with a possibility of flurries) tonight so we'll probably do another fire too. This place is pretty freakin' awesome. Winter, I fear, will suck but so far the rest of the seasons seem to make it worthwhile.

Another fall tradition is voting, which we did early here in Milwaukee. If you can, go do it now.

Fall in the neighborhood

Constraints are good

I may have written about this in the past.. I dunno. Also, someone seems to have put extra zen in my coffee this morning, so consider this a pontification warning.

If you check out this blog posting by Jamis Buck at 37Signals about embracing constraints, Jamis talks about self-immolation due to neato-creep and how important it is to realize the real world issues and separate them from the coolness factor issues you may have in your head. Part of me agrees with this -- part of me also thinks that sometimes we need to make the kinds of mistakes Jamis talks about in order to learn. If "less is more" then you need to understand more before you can understand less. That journey is incredibly valuable and I think Jamis' post is mildly dismissive of that. I don't think it was his intention to do so, but I think it bears pointing out the value of experience.

I think more importantly, his posting made me remember why I think the "embrace your constraints" is a valuable idiom. To me, it's because your constraints create the context in which you make decisions and by embracing your constraints you're lead to a better understanding of Why you're doing what you're doing. It means you can look at the environment, not just the moss on the side of the tree.

Why is this important for System Administration? I've too often seen admins get upset because the situation wasn't optimum for the conception of a system they had in their head. Instead of stepping back, embracing the constraints, and progressing toward their goal with a new understanding of the context, they'll shut down and not do anything, or rant-and-rave and make others fall into their line, or just do what they wanted in the first place and say "damn the torpedoes!" ignoring the constraints and end up with more enterprise cruft. That never works well for anyone and I think is a major reason why smart people hate enterprises.

Embracing constraints requires thoughtful reflection of the situation. Not doing so creates a toxic environment of anti-intellectualism [1].

Look at any ERP or Business Intelligence implementation and then look at what it replaced, and you'll see that the names are different but the larger issues are probably the same. It's because those systems, in a very real-world way, interface not just at data levels, but at context levels. They push and pull data that has a meta-value in the context of that data. Most people don't look at those contexts, or the corollary constraints those contexts give. So you end up with large brittle systems that mingle meaning and action and break when either change.

So, I guess, I don't think "embracing your constraints" as something of a trendy productivity enhancement catchphrase. Instead, I think it's a mantra for doing.

1 - Why is it toxic? Programming, System Administration, and computing in general all deal with virtual, logical, ephemeral things. They can and do change rapidly. Taking an anti-intellectual stance towards your environment means you'll never grow or learn with those things and always be in a reactionary, fire-fighting mode of operation. And you won't even know it. If you read The Daily WTF? you'll see examples of this every day. It's sad, really.

Livna -> RPMFusion migration

Thorsten Leemhuis writes to remind us about the loss of the Livna mailing lists and the continuing migration of the Livna repos for Fedora to the RPMFusion repos announces a few months ago.

If you're a Livna, FreshRPMS, or Dribble user (and if you have an nvida card, ati card, mp3, or dvd around, you most probably are) you should probably start taking heed of the changes.

live blogging and iphone

I live blogged a bunch of reactions to the debate tonight. Sadly, I did so on my iPhone whihc really isn't the greatest tool for that, so the spelling and posting is funkified.

I don't hate people

For anyone who might possibly be reading the notes on my Shared Items Google reader page you might think that I hate people. That's not true. Part of it is that I've been in a less than great mood this week for various personal reasons. Chief among them is that I seem to have caught a cold.

Another big part though is that I am continually annoyed at how technology is marketed. We continue to push forward the myth that "computers make things easier" and continue to foist metaphors of technology with meatspace equivalents. The thing is computers don't make things easier, they make things faster and more widely available. Where things start to break down is when the metaphors we use fail to address the issues of speed and availability. I think when that happens for non-techy people they just give up and figure out a way to muddle though. I think that's where we're at now: the majority of people muddling though a day full of computing by blasting email out on a near continual basis because they know it sorta works for everything they need. That frustrates me because I can see their frustration.

IMHO, the fix is not to simply show a person how to use Word. Instead we need to educate with information and time management skills and work harder to make the tools they use simpler. People shouldn't need training on how to use a pile of software, but people do need training on how to deal with the junk that software creates.

Carpet Fresh and Assumptions

Martha Stewart believes you should clean your dishwasher. If you have dogs, you also know that even that spendy Dyson Animal needs a good cleaning once in a while because, frankly, dogs are damn gross. This is mainly because even though we assume our cleaning tools are by nature clean, it's a wrong assumption.

We recently moved into a new apartment which has wall-to-wall carpeting. The previous residents used Carpet Fresh. How do I know? Because today I went to empty the bin on our new vacuum cleaner it was full of carpet fresh like stuff. But we've been sucking it up since we moved in, it hangs around in your carpet forever and starts to stink after a while. That there was funk in the bin didn't surprise me. I also noticed that there was some dog fuzz in the part of the bin that caused the dirt to spin around. I popped it out to unwrap the hair and *plonk* out came a giant mess of dog fuzz bound with a huge amount of Carpet Fresh. Dust flew everywhere causing me to get insanely pissed. It also made me think "Huh, that's why every time we vacuum it smells worse.. I just assumed we were kicking up leftover Carpet Fresh from the carpet." I took everything apart, cleaned it, and put it back together. Now things work better. I knew that this was something I really should do once a week, but I got lazy and assumed that my shiny new cleaning tool was by nature, clean.

So how does this relate to system administration? We assume a lot in our line of work. We assume (or perhaps "believe") that things break in the same way, that data formats are correct, that things that do have inter-relations don't, that someone else hasn't changed that script in the past year. That cleaning tools don't also need a good cleaning once in a while. When we don't check what we definitively don't know about a system or process before we build upon those assumptions, we'll get bitten at some point.

identi.ca

So, I figured since keeping up with being articulate here is only working so well, I'd also try easing into it with some microblogging at identi.ca also.

Install Fest at Milwaukee LUG!

Advance warning that the Milwaukee Area LUG is having an Install Fest on Sept. 13th. More info can be found in the wiki. I'm going to try to be there with my laptop and a copy of liveusb-creator to make some USB Fedora keys for folks who want them as well as lend a hand.

If I have extra time, I'll try to setup DHCP on my laptop to run as an option server to present PXE'able images for various installers. This will require a bunch of downloading across my poor little cable-modem to get said images, so that might not happen, unless of course I travel to $hq and can co-opt their link.

If you're in Milwaukee, drop by and say "Hi" or bring a machine to install Linux on!

I hate interviews, interviewing, resumes and cover letters

Ahem. yep. That's it. The job search and interview process scares the crap out of me and I hate it.

I shake when writing cover letters. I've lost jobs because I got nervous during on-site interviews and babbled the first thing that sprang into my head. So much so that one recruiter even asked me flat out "What the hell happened in there?" It's worse now that I've learned how to keep calm during phone interviews, because my inability to sound intelligent during an on-site generally rings bullshit alarms even louder now.

So to all of you who will have to interview me, read my resume or cover letter, or generally deal with me on the applicant end of a hiring process: I apologize. I'm smart, I can do the job, I just evidently have some sort of emotional interview baggage I haven't fully realized yet.

Or something. Maybe airing my issues publicly is the first step to fixing them, I dunno.

CAR-MAX Sucks

So we're moving and the place we're moving to only has a 1 car garage. Since I'll be telecommuting for the near future, we figured we'd ditch one of the cars because we wouldn't need it. To do so we sold the car to Car-Max on Tuesday.

It's now Saturday and they have not paid off the loan I had on the car as they said they would. After several phones calls trying to talk to their business office (several times I was sent to voice-mail for some other random department and had to call back) I finally got someone. They claim their check-scanner was broken and so, everything needed to be hand-entered. That wasn't the case on Tuesday and I know it doesn't take the better part of a week to get a scanner fixed or do hand entry.

Their suggestion? "Call back on Tuesday if it doesn't post on your account, kthxbai!"

Bimbos.

If it doesn't post by Tuesday I'm reporting the vehicle stolen.

UPDATE: Aug 17, 2008: Car-Max figured their stuff out by Wednesday, but still, please.

Please leave a message at the sound of the tone....

I'm quiet not because I'm lazy but because I'm moving. More on the move, and the destination at some point.

Things that are interesting to me, that I'd like to write about in the near future:
- http://lopsa.org/pipermail/discuss/2008-July/003730.html
- http://times.usefulinc.com/2008/06/16-ops-now
- http://nasrat.livejournal.com/52943.html

Enjoy!

Red Hat Frees RHN Satellite product

Wootness!

Red Hat has finally opened up their RHN product. I guess they're finally taking that "we're a solutions company, not a software company" thing to heart. Good for them. I know this has been a large part of their revenue model since they started it, so releasing the code must not have been easy for them. I don't think it will hurt them in the long run and hopefully will help because now it would/could/should be possible to integrate stuff like puppet, cfengine, func, cobbler/koan and all sorts of other fun stuff.

Fun times ahead.

• http://dgoodwin.dangerouslyinc.com/node/99
• http://www.redhat.com/spacewalk/

obligatory iPhone post

Please excuse this mindless post while I futz around.

Also: packetshaper + steelhead = funky traffic data

one step forward...

Things that rock: Documentation

Things that suck: Documentation in a Word .doc file that's 95% composed of mostly illegible screenshots.

USENIX, LOPSA continue fight; Community looses

First, the various announcements about the lawsuit:

• USENIX
• LOPSA

So, at this point you might be thinking "Huh?" so here's my attempt at providing some small bit of context to these current events:

Several years ago the small group within USENIX called SAGE (a system administration focused community) tried to break off from USENIX, ostensibly with the support of USENIX, to further its' own goals. Somehow, something went very wrong with the process. The break never happened and many people were left angry and hurt. From that aborted process LOPSA was born as a separate entity to replace what would have been the spun-off SAGE group while SAGE continued to exsist within USENIX as a special intrest group.

I'll not link to the documents and email threads of the past that describe the details, because I think there's no clear series of well-documented events that chronicle everything. If you need to know Google is your buddy, as is the sage-members list archive, if it's still available publicly.

This suit is one of the closing matters of that aborted process, namely, who should be paying the bill to Association Headquarters (aka: "AH" -- the management company that provides the business infrastructure for the organizations) for the time they spent with the pre-LOPSA organization during the failed separation process. AH started this suit in order to get payment on an outstanding account, AH looked at USENIX who's now pointing the finger at LOPSA. Who really is responsible for that payment can only truthfully be determined by looking at the aborted process and picking it apart. In court. With expensive lawyers. I also doubt that the legal proceedings would identify one specific organization as fully responsible either.

I can't imagine that the outstanding bill AH is holding is a substantial amount or that it is more than what the combined legal fees that AH, USENIX and LOPSA will have to incur just to deal with the legal proceedings of this suit. Why not just split the AH bill between USENIX and LOPSA and everyone pays their own legal fees? I suspect the out-of-pocket expense would still be less than the several years of litigation for this matter to be settled.

I beg all parties to not bring this issue further into the community, to sit down with cool heads and find a solution to this issue quickly.

This suit, its' exposure in the public, and the dialogue it has created does not help the members of the organizations, does not help the community at large, and does not help the practice, art, or science of System Administration. If anything, it paints us all as petulant children who can do nothing more constructive with their time then squabble and taunt each other. I've said before that the biggest hindrance to getting System Administration recognized and respected as a real honest-to-god profession is ourselves. The entire USENIX/SAGE/LOPSA drama, and this suit in particular, again underscores that sentiment. We're sabotaging ourselves. We need to get over it. We need to move forward.

And we need to do it now before the world writes us and our profession off as nothing more than a real life version of a Saturday Night Live sketch.

zombie meme

mindlessly following the meme:

> You are in a mall when zombies attack. You have:
> 1. One weapon
Flamethrower (I figure the fry-o-lators in the food court should provide plenty of ammo)

> 2. One song blasting on the speakers
It's a Small World (After All)

> 3. One famous person to fight along side you.
Jack Bauer (or if it has to be a *real* person, Keith Olberman)