Server Stats considerations

EQ2Emulator Development forum.

Moderator: Team Members

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Re: Server Stats considerations

Post by John Adams » Fri May 22, 2009 5:16 pm

Yeah, no doubt. I think our world is fairly light-weight, but we have yet to see full content, full player utilization.

Tests I've run, I am suspicious of memory leaks but have no proof ;) except a creeping RAM usage. I cannot really compare EQ2Emu to EQEmu, because EQEmu is broken up into separate binaries, and a lot less 'weighty' than the EQ2 content, imo... but I think there will come a day when we have to at least consider a cluster-like system like EQEmu zone servers. But that'll only be if this is ever used as a large-scale public server. Right now, meh... 2-3 players concurrently. Oh wait, that's a stat! ;)

I already asked Scatman at initial design to consider customizable intervals, but for this first round, it's static.

Image
Retired
Posts: 251
Joined: Sun Oct 26, 2008 10:07 am

Re: Server Stats considerations

Post by Image » Fri May 22, 2009 8:36 pm

well I figure in a few years we will be using 8 core processors at least... RAM usage is high of course as you mentioned because we load the entire world into one executable (thats the world server and zone servers). We are also using XML now to handle configuration structures, XML is very heavy on memory... Debug mode makes it worse, linux makes it even worse.

I am actually looking at updating the XmlParser right now... I think it is the culprit.

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Re: Server Stats considerations

Post by John Adams » Fri May 22, 2009 8:38 pm

Right on, at least you guys have some idea what might be causing it.

I personally haven't had the nerve to ask LE what he thinks about splitting World/Zone server ;)

Image
Retired
Posts: 251
Joined: Sun Oct 26, 2008 10:07 am

Re: Server Stats considerations

Post by Image » Fri May 22, 2009 8:46 pm

Try replacing the xmlParser.cpp and .h files included in http://coregame.googlecode.com/files/xmlParser.zip (common directory)

version on the file should be 2.23 now. I would prefer not to upload to SVN as I don't know if LE applied any special fixes. But it would be worth a shot.

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Re: Server Stats considerations

Post by John Adams » Tue Jun 02, 2009 7:06 pm

Bah, Scatman... I am reporting a bug with my own code!! ;)

The player Online (is_online) seems to be resetting itself to 0 after some mysterious thing happens. I haven't tracked it down, but from what I remember, there's only two places that is_online gets set back to 0; server startup, and client disconnect.

Is it possible clients are being disconnected while moving around, or zoning, or afk? I'd like to track this down, but it's not super important. I just noticed all is_online fields are 0 while there are players on Tess :( and can't figure out why.

User avatar
Scatman
Retired
Posts: 1688
Joined: Wed Apr 16, 2008 5:44 am
EQ2Emu Server: Scatman's Word
Characters: Scatman
Location: New Jersey

Re: Server Stats considerations

Post by Scatman » Tue Jun 02, 2009 11:30 pm

Hmm maybe there's a bigger issue of when it gets set to 1. I'll look at it tomorrow since the government sucks at processing paperwork in a timely fashion, I'm bringing my laptop to work since I have nothing to do yet.

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Re: Server Stats considerations

Post by John Adams » Wed Jun 03, 2009 9:24 am

I am >pretty sure< the values get set to 1 ok, because I can refresh the page and see the player. Then later, I refresh again and they are gone (0) but, I still see them online playing with spells and items.

User avatar
Scatman
Retired
Posts: 1688
Joined: Wed Apr 16, 2008 5:44 am
EQ2Emu Server: Scatman's Word
Characters: Scatman
Location: New Jersey

Re: Server Stats considerations

Post by Scatman » Wed Jun 03, 2009 1:50 pm

Ya I just looked at it today at work and I'm not exactly sure why that's happening. Might just have to hop on the server and try different things.

LethalEncounter
Team: Zombie
Posts: 2717
Joined: Wed Jul 25, 2007 10:10 pm

Re: Server Stats considerations

Post by LethalEncounter » Wed Jun 03, 2009 2:13 pm

John Adams wrote: I personally haven't had the nerve to ask LE what he thinks about splitting World/Zone server ;)
I don't think it would be a good idea. EQEmu uses shared memory to accomplish this, but I think it is more of a headache that it is worth. The benefit of separating World and Zone would be that you could run the zones on another computer (which btw thereby makes the shared memory advantage invalid), but we could link multiple worlds together to create the same concept. When I was designing the emu I specifically decided to avoid the shared memory method that EQEmu used and use threads instead. I think the server is fairly stable at this point in it's life cycle and it will only grow more stable as time goes on. I stand by my decision and think it is correct. However, if anyone has a problem with the design please let me know so that I can come up with a solution if there is a problem. I'm not going to get pissed off for discussing something that I disagree with :P

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Re: Server Stats considerations

Post by John Adams » Wed Jun 03, 2009 2:16 pm

Heh no way man. I love the design we got. I was just curious how we might handle things once an EQ2Emu world had 200+ connections to one single binary. I know when I did the performance testing with 15 logins, my CPU barely woke up. That was impressive.

User avatar
Scatman
Retired
Posts: 1688
Joined: Wed Apr 16, 2008 5:44 am
EQ2Emu Server: Scatman's Word
Characters: Scatman
Location: New Jersey

Re: Server Stats considerations

Post by Scatman » Wed Jun 03, 2009 2:21 pm

The design rocks :) and it is exactly how SOE does it. Each of their computers holds x amounts of zones. I remember when we would have x8 vs x8 battles in Kylong plains, the people raiding Veeshan's Peak would lag out because the developers had to actually intervene and tell us that the two zones were on the same server :D

LethalEncounter
Team: Zombie
Posts: 2717
Joined: Wed Jul 25, 2007 10:10 pm

Re: Server Stats considerations

Post by LethalEncounter » Wed Jun 03, 2009 2:25 pm

lol, that is funny. Not even SOE's huge budget and uber network could stop lag on their servers :P Strange that they wouldn't run more servers tho

User avatar
Scatman
Retired
Posts: 1688
Joined: Wed Apr 16, 2008 5:44 am
EQ2Emu Server: Scatman's Word
Characters: Scatman
Location: New Jersey

Re: Server Stats considerations

Post by Scatman » Wed Jun 03, 2009 2:29 pm

Hehe ya. It became a huge thing because competing guilds would know the other guild was raiding VP and would constantly zone in raids of their guild into and out of Kylong Plains to crash the server or lag them out in Veeshans. They ended up actually separating the zones from the same computer.

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Re: Server Stats considerations

Post by John Adams » Thu Jun 04, 2009 4:58 pm

Scat, I think we're going to have to come up with something different for server stats than REPLACE INTO. A World just sitting idle even with no players, the stats threads are running, updating. Well, what I had forgotten about REPLACE INTO is that the function doesn't update the existing record in-place, but deletes the old one and replaces it with a new insert.

http://dev.mysql.com/doc/refman/5.0/en/replace.html

Look at dev's `statistics` table sometime. We've had stats on Dev for maybe a few weeks, and the PK for the table is already over 41,000. I just see no reason why that ID should ever grow to the millions, which it will on long-running servers. Maybe it's just my quirky dislike of things that make no useful sense...

So I'll see if I can come up with an alternative. Maybe if it exists, hit an UPDATE query, otherwise INSERT. Not sure if there's an UPDATE INTO, but I'll check. Either way, toggling between UPDATE and INSERT shouldn't really be a problem.

Performance considerations:

Please note that REPLACE INTO is a much slower performer than an UPDATE statement. Keep in mind that a REPLACE INTO requires a test on the keys, and if a matching unique key is found on any or all columns, a DELETE FROM is executed, then an INSERT is executed. There's a lot of management of rows involved in this, and if you're doing it frequently, you'll hurt your performance unless you simply cannot do with any other syntax.

The only time when I can see where you'd actually need a REPLACE INTO is when you have multiple unique constraints on a table, and need to drop any rows that would match any of the constraints. Then REPLACE INTO becomes more efficient from DELETE FROM... INSERT INTO...

If you're looking at a single unique column table (Primary Key), please use UPDATE, or INSERT. Also, check out INSERT ... ON DUPLICATE KEY UPDATE... as an alternative if you're willing to stick to MySQL 4.1+

I tested this query in query editor, and it seems to do what I want.

Code: Select all

INSERT INTO statistics (char_id, guild_id, stat_id, stat_value, stat_date) VALUES (1, 0, 1000, 18, 1243792314)
	ON DUPLICATE KEY UPDATE char_id = 1, guild_id = 0, stat_id = 1000, stat_value = 17, stat_date = 1243792314;
I'll change dev and give this a try, see what happens ;)

LethalEncounter
Team: Zombie
Posts: 2717
Joined: Wed Jul 25, 2007 10:10 pm

Re: Server Stats considerations

Post by LethalEncounter » Thu Jun 04, 2009 5:27 pm

We use the ON DUPLICATE KEY a couple of places in the code, so yours should work just fine too :)

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests