Unplanned Outage - 2013.11.06

[John Adams]

The main server for all my services has lost a hard drive from the mirror, so I am turning off non-essential services until I can get a replacement. Web, IRC and EQ2Emu services have been migrated to a backup server and should remain functional, but maybe a little slower.

Let me know if you experience any major problems.

[Zcoretri]

It does seem things are a little slower to pull up...Thanks for the heads up JA

[John Adams]

Sorry about IRC, some channels became unregistered again and corrupted the DB somehow so Services wouldn't start. Bleah... and Xinux and his war-dialing reconnector kept grabbing the channel for himself so I couldn't re-register So I had to block IRC for a moment. You'll probably need to reconnect.

Meanwhile, the VMG/EQ2/VGO NOC is getting a much needed upgrade. I pulled the trigger today on parts for building a new server, since the PowerEdge 2900 is 7 years old, and it has been giving me trouble lately.


Not that you care, but I like hardware - The 2900 is a dual quad core 1.8GHz box with 24gb ram and a raid1 500gb drives... fairly simple, but has run upwards of 15 VMs for me for years.

It will now be replaced (or clustered with) this:

1 x http://www.newegg.com/Product/Product.a ... 6813131372

2 x http://www.newegg.com/Product/Product.a ... 6819117258

6 x http://www.newegg.com/Product/Product.a ... 6820161459

1 x http://www.newegg.com/Product/Product.a ... 6817153170

1 x http://www.newegg.com/Product/Product.a ... 6811129179
Although I think I goofed up on the case... not sure the 13" x 12" mobo will fit

Just hope it doesn't blow up my house when I plug it in.


btw, everything is currently running on a 12gb ram i5-2500k desktop with 1 single 64gb SSD os disk... I am wetting my pants how close I am to losing everything

[thefoof]

<3 upgrades

[Zcoretri]

Looks like one nice server rig JA

[John Adams]

The outages continue today, likely through mid-week as I get the new server built and configured. I need to move some VMs off my iSCSI NAS, since it apparently is not designed to run 20 VMs at once... murdering my home network

EQ2TC will be back up in a few hours, as will my other emulators I host, and SVN will return shortly as well.

Web and IRC remain up, but running a bit slow.

[Secrets]

The outages continue today, likely through mid-week as I get the new server built and configured. I need to move some VMs off my iSCSI NAS, since it apparently is not designed to run 20 VMs at once... murdering my home network

EQ2TC will be back up in a few hours, as will my other emulators I host, and SVN will return shortly as well.

Web and IRC remain up, but running a bit slow.

IRC seems to be freaking out again >_<

[John Adams]

Hmm, what's wrong with it? I am logged in right now. I see no one else, which is unusual. I usually see Xinux.

[Secrets]

Hmm, what's wrong with it? I am logged in right now. I see no one else, which is unusual. I usually see Xinux.

It was disconnecting once every 5 minutes or so. Same with the EQ2Emulator LS.

[John Adams]

Yep, just noticed that my Cox ISP is dropping me every 5 mins so I have a report in to them too to diagnose that problem as well. No wonder I saw so many broken pipes in LS console.

[John Adams]

Okay, I have restarted every piece of hardware in the house, from modem to firewall to VMs. I have a constant ping happening and not seeing anymore outages. Sitting on IRC too so if I fall asleep any second now I will see in the morning if it held up.

[John Adams]

Parts are installed. Now if I could just get it to POST

NewServer.jpg

[thefoof]

Parts are installed. Now if I could just get it to POST

NewServer.jpg

Percussive maintenance

[John Adams]

Well I'm a dummy. Note i the pic, I forgot the 2nd ATX12v connector. Spent all day troubleshooting why the system wouldn't boot... this is my first (man-made) dual proc

[John Adams]

Ahh, shit. All services coming down for about 10 mins. I just screwed up my iSCSI target and need to reboot both servers. Sorry

I'll take the forum down last, since no one is on IRC.