Archive for the 'System Administration' Category

May 01 2008

I’ve been a busy little geek

So far this week I’ve:
Finally gotten a working Xen system that will boot a Debian guest.
Successfully installed ispCP on the Debian guest.
Built another Debian guest to be an OpenVPN server.
Successfully built an OpenVPN server and got two clients to connect from outside the network, through the DSL modem/router.
Correctly configured the VPN server to give the client access to the full network via IP masquerading (next trick: get the network to simply route the packets instead of having to use masq).
Got ddclient working on the VPN server to keep dyndns updated so I don’t have to hard code an IP address in my VPN clients and check various server log files to see if it changed.
Fixed ddclient, when it failed to update dyndns with new IP address after my DSL provider mysteriously issued a new one, not 3 hours after setting up ddclient in the first place.

I can now log into my ispcp box from my desk at work, as though it was on the same network. I can now proceed with trying to get Mailman to play nice with ispCP when it’s slow at work.

I get productive when I ignore my games.

No responses yet

Apr 05 2008

Getting hpasm installed on Ubuntu server

Published by Andrew under Linux, System Administration, geeking

While installing Ubuntu Server 8.04 beta on an HP DL-320, I discovered I had some trouble getting HP’s “Proliant value added software” (hpasm) package installed. This package contains their system health check and control software which, among other things, switches the fans from “full-time full speed” (which is quite noisy) to temperature controlled speed (eg: normal (read: quiet) fan speed when system temp is normal).
The problem with installing and runnning this software stems from the fact that Ubuntu, for some reason, links /bin/sh to dash instead of bash. Dash is another bourne shell clone, but doesn’t understand Bash (bourne-again shell) specific syntax.
Re-linking /bin/sh to bash instead of dash solved the problem and the server is now humming (quietly) along.

No responses yet

Feb 22 2008

SSH connection mastering

Published by Andrew under Linux, System Administration, geeking

While I get back into the swing of blogging from a fairly long hiatus, I thought I’d start with something useful I discovered a while back, but that has recently shown just how really cool and useful it is.
With ssh connection mastering, you can open multiple sessions to a single host. Once you have satisfied all the authentication / authorization requirements of the host to log in the first time, as long as you have a master connection open, logging in additional times not only doesn’t require going through authentication again, it’s much, much faster.

For example, where I work, we SSH to a single server, we’ll call it jupiter, a “bastion host” if you will, from whence we can make ssh connections to any of our client’s servers. All our client’s servers have our public key in the root authroized_keys file, and all technicians on jupiter can ssh using the private key associated with that public key. To log into jupiter the technician must enter his RSA SecureID pin and rolling key.

If you’ve never used SecureID, it is a “two factor” authentication system. Two factor authentication basically means to log in, you must present both something you know (thus it can’t be taken away from you by force, though you can be coerced into revealing it) as well as something on your person. In this case the “something you know” is your PIN, which doesn’t change. The “something on your person” is the 6 digit number displayed on the SecureID token. This is more secure than a simple password or PIN in that you must have both that and a physical device that displays a constantly changing number. Either one by itself is insufficient authentication to log in.

So getting back to jupiter, to ssh to this server we use SecureID authentication. This means we must ssh to the server, wait while ssh negotiates keys and does any DNS foo the server wants to do, type our PIN, then dig out our token and enter the displayed number, then wait while jupiter checks with our SecureID server to verify the numbers entered.

Using the ssh connection mastering technique I’ve linked to, you only have to do this once. As long as that original connection is open, you can open another terminal window, or even a virtual terminal, and ssh to the server and be in, instantly. No delays for ssh key exchanges, DNS lookups SecureID. You don’t have to enter your PIN and token again. You’re just in. This is very useful if you have to log into multiple client servers at once.
Just don’t tell the security admin.

For some reason the trackback link isn’t showing up in the blog. The link referred to above is
http://www.newartisans.com/blog_files/ssh.connection.mastering.php

No responses yet

Aug 07 2007

Linux and Realtek RTL8139 Ethernet drivers

Published by Andrew under Linux, System Administration, geeking

Recently the network connection on my Ubuntu (7.04 “Feisty Fawn”) desktop failed. It had been working perfectly for weeks when suddenly it just couldn’t connect to my home network. Everything worked fine in Windows.
During boot, it would properly detect the device, load the driver, the networking init script would run without errors, and but for the fact that it couldn’t talk to the network, and thus couldn’t get a response from it’s DHCPDISCOVER probes, no errors.
A static IP configuration worked just fine, but still wouldn’t talk to the network.

the_title

No responses yet

Jul 09 2007

Just some notes for future reference

Published by Andrew under System Administration, geeking

When building new Unix servers and installing sshd…

  1. Be sure to install xauth
  2. Make sure to turn off “UseLogin”

No responses yet

Jul 08 2007

Ah, the good ol’ days…

When building a Sun / Solaris box meant downloading and installing all the nice little Gnu toys we take for granted in Linux…
(Thank goodness for sunfreeware.com, that has them all pre-pacakged for you!)

(Edit: That doesn’t really make sense, does it? Not “the goold ol’ days…”, since that implies that’s no longer the case. Guess that should be “Ah, now I remember the good ol’ days!)

No responses yet

May 31 2007

Oh noes! I’m turning into a DBA!

I swear, I’ve learned more SQL in the last month than in the previous 10 years of system administration.
I may end up having to add “data base administrator” to my resume skills section after all.
Looks like that copy of MySQL & mSQL from O’Reilly will come in useful after all.

No responses yet

May 31 2007

debugging overnight mysql dump…

debugging overnight mysql dump scripts.

No responses yet

May 17 2007

Just for my records

This week, I…

  1. Fixed email address for Bacula reports
  2. Fixed FTP server quotas
  3. Fixed mailing list for post access without receive access
  4. Fixed broken NRPE2 agent on mail server

Morton:

  1. Spent several hours talking on phone with Kronos support trying to figure out how to fix the timeclock

No responses yet

May 12 2007

This process is taking forever

Published by Andrew under ${POE}, System Administration, work

I can’t believe the data migration I started yesterday afternoon is STILL running.
Jeez, how long does it take to move 372GB from one drive to another?

No responses yet

May 09 2007

I have a time warp in my network somewhere…

The mail server is receiving the output of a cron job 20 minutes before the cron job even runs.
If I can find out how that’s happening, I should be able to make a fortune!


Received: from < deleted> (deleted[127.0.0.1]) by (deleted) (8.13.6/8.13.6) with ESMTP id for deleted>; Wed, 9 May 2007 03:01:06 -0500 (CDT) (envelope-from root@deleted)

Received: from unknown (HELO deleted) (deleted)
by mail.deleted with (DHE-RSA-AES256-SHA encrypted) SMTP; 9 May 2007 07:39:21 -0000

Now, allowing for the time zone difference of 5 hours, how does that mail arrive 20 minutes before it was sent?

The sending server:
> date
Wed May 9 17:08:44 CDT 2007

The mail server:
$ date
Wed May 9 17:09:56 CDT 2007

As you can see, they’re clocks aren’t off (allowing for the time it took me to log in).

No responses yet

May 09 2007

The pain! The pain! Make it stop!

Published by Andrew under ${POE}, System Administration, work

Gotta love working with an “engineer” who can’t explain anything and thinks your an idiot for asking detailed questions.

We started a new “fault tolerance” project to make all of our services completely redundant across data centers. When trying to explain how it works, my co-engineer used DNS for an example. All he could tell me is that “all the data will be replicated across all data centers so if one goes away, the others will answer”. When I started to try to get more details about how this will work (”Are all servers just going to point to the same IP and the system will redirect that IP to a working server?”) he couldn’t tell me. Then got frustrated with me because I was focusing on DNS. He kept wanting to give me the Executive Summary of the project, while I was asking specifics. And he doesn’t know the specifics and doesn’t understand the relevance of my questions.

To put it another way: this guy is not an engineer. He wants to be an engineer, he got the job title, learned a few things about a few specific services (and specifically, the software we use to implement them) but get outside of that and he’s lost. Problem is, I know next to nothing about the software we use to implement these services. I AM familiar with the concepts however. He mistakes not being familiar with our specific software for ignorance (even though he doesn’t know the general concepts.)

No responses yet