How good people turn failure into success

Quite a few years ago, I made a mistake.  This wasn't your average 'oops!' type of mistake.  I deleted a lot of data and rendered a server useless due to careless typing.

The difference between 'rm -rf /home/bob/dir' and 'rm -rf / home/bob/dir' appears subtle but it really isn't, that space makes the world of difference!

Petrified, I went to the administrator and explained what I had done.  I never got the impression he thought much of me, after all, next to him I knew nothing.  I knew I had messed up big time and I was sure my access was about to be removed and any thoughts of being a linux admin were down the drain.  Thoughts of all the nasty things coming my way raced through my head.

He smiled.  Literally, his first response was to smile.  Calm before the storm I feared.  He reassured me that such things happen and that there were backups.  He 'punished' me by making me help build a replacement, an opportunity I'm sure he knew I would have loved to have had anyway.

Now I'm a team lead, leading a team of six linux administrators in one of the countries larger IT providers.  Our team do great work that enables world changing work.  From time to time someone will make a mistake in the team, and I get the opportunity to smile and help people grow.

If I had never made that mistake, or if that mistake had been dealt with differently, I wouldn't be where I am now.  It's incredible the difference good people can make with just a little positivity.

I'm happy to say that the scary admin is now someone I consider a good friend, and an all-round great person.  When I left that team he gave me this gift, something I treasure to this day.

Maintaining a Debian/Ubuntu repository on Redhat (or other linuxes)

At work we have a repo server that runs Redhat.  I couldn't find a good way to maintain a Debian/Ubuntu repository on it, so I built a solution.

Basically I built a chroot environment that when run takes packages from the incoming folder and turns them into a repository that you can put on your webserver.

I'm sure there are more elegant solutions (though I couldn't find them), but this serves my purpose at least.

This script will create the chroot environment for you.  Run the script on a debian box, copy the tar ball to another server, untar it, put files in the incoming directory and run the included script.

Fixing broken trust relationships in windows server

I was looking around for a reference to send to a colleague who had a server lose it's domain relationship and found this one which seems to work really well.

Basically, log in to the affected server and run:
netdom.exe resetpwd /s:<DC hostname> /ud:<domain\user> /pd:*

Over riding puppet modules

I've discovered that you can overload parts of puppet modules.  This means I can still use the module but just overload what I need.

Of course, modifying the module may be tidier but modifying a module used by lots of production servers is not a trivial undertaking.

Here's how it works:

class systems::section::authssh inherits auth::ssh {
   File['/etc/pam.d/sshd']{

      content=>"blah"

  }

}

This means I get all the good stuff from auth::ssh but I can override the sshd pam configuration for this server/group of servers.

The case I was dealing with was a bit more complicated (though same concept), in modules I have auth::thistype which includes auth::pam

I would normally include auth::thistype and just to complicate things the setting I want to change is in auth::pam.  Unfortunately (in this case) puppet won't let you modify an argument in a class that isn't the one setting it (or a sub class thereof) so I can't set it in the class as above, I need to set it in the correct class, like this:

class systems::section::auththistype inherits auth::thistype {

  include systems::section::authpam

}

class systems::section::authpam inherits auth::pam {
   File['/etc/pam.d/sshd']{

      content=>"blah"

  }

}

(Please note, code sections were genericised and should be used as reference only) 

Install TSM on Debian / Ubuntu

I've had to do this a few times and normally kludge my way through.  The internet seems full of older versions and convoluted/partial instructions.

I decided I should just fix it properly.

The script I put together takes the tar ball from IBM and turns it into debian packages.

I tested on Precise server but it should work on most debians.

You can download it from http://nzgeek.com/install_tsm.sh

OSX VPN issue

For a while I've noticed that if VPN dies while connected (eg. If you put your computer to sleep before disconnecting) it won't reconnect until after a reboot.

I checked routing tables and logs etc with little success and I JUST realised that when it starts 'racoon' (a key management daemon) starts up.  A stale racoon process causes issues when trying to connect hence if you kill it first then try and connect it works fine.

Something like this should do it (from terminal):
 sudo kill `ps -U root -o "pid,comm"|grep racoon|cut -d" " -f1`

SCCM PXE and WDS

Last weekend I started setting this up and had a few problems.  As is my way I checked the docs which appear to be non-existant and then the community and finally found one page with some details.  Basically these are the hints you need.  The last two are the ones that caused my troubles.

• As is always the way in SCCM, check the package status before assuming a package is available or removed
• Check the pxesetup log before assuming pxe is (de)installed
• If everything is in one subnet and you can probably leave DHCP options alone
• Start with WDS and PXE uninstalled, then install wds, reboot, install pxe, reboot
• Make sure you have both 32 and 64 bit PE images, even if you will ever use them both
• No matter how much windows tells you off for not configuring WDS, don't configure it