Page 3 of 4

Re: Server Stats considerations

Posted: Fri May 22, 2009 5:16 pm
by John Adams
Yeah, no doubt. I think our world is fairly light-weight, but we have yet to see full content, full player utilization.

Tests I've run, I am suspicious of memory leaks but have no proof ;) except a creeping RAM usage. I cannot really compare EQ2Emu to EQEmu, because EQEmu is broken up into separate binaries, and a lot less 'weighty' than the EQ2 content, imo... but I think there will come a day when we have to at least consider a cluster-like system like EQEmu zone servers. But that'll only be if this is ever used as a large-scale public server. Right now, meh... 2-3 players concurrently. Oh wait, that's a stat! ;)

I already asked Scatman at initial design to consider customizable intervals, but for this first round, it's static.

Re: Server Stats considerations

Posted: Fri May 22, 2009 8:36 pm
by Image
well I figure in a few years we will be using 8 core processors at least... RAM usage is high of course as you mentioned because we load the entire world into one executable (thats the world server and zone servers). We are also using XML now to handle configuration structures, XML is very heavy on memory... Debug mode makes it worse, linux makes it even worse.

I am actually looking at updating the XmlParser right now... I think it is the culprit.

Re: Server Stats considerations

Posted: Fri May 22, 2009 8:38 pm
by John Adams
Right on, at least you guys have some idea what might be causing it.

I personally haven't had the nerve to ask LE what he thinks about splitting World/Zone server ;)

Re: Server Stats considerations

Posted: Fri May 22, 2009 8:46 pm
by Image
Try replacing the xmlParser.cpp and .h files included in http://coregame.googlecode.com/files/xmlParser.zip (common directory)

version on the file should be 2.23 now. I would prefer not to upload to SVN as I don't know if LE applied any special fixes. But it would be worth a shot.

Re: Server Stats considerations

Posted: Tue Jun 02, 2009 7:06 pm
by John Adams
Bah, Scatman... I am reporting a bug with my own code!! ;)

The player Online (is_online) seems to be resetting itself to 0 after some mysterious thing happens. I haven't tracked it down, but from what I remember, there's only two places that is_online gets set back to 0; server startup, and client disconnect.

Is it possible clients are being disconnected while moving around, or zoning, or afk? I'd like to track this down, but it's not super important. I just noticed all is_online fields are 0 while there are players on Tess :( and can't figure out why.

Re: Server Stats considerations

Posted: Tue Jun 02, 2009 11:30 pm
by Scatman
Hmm maybe there's a bigger issue of when it gets set to 1. I'll look at it tomorrow since the government sucks at processing paperwork in a timely fashion, I'm bringing my laptop to work since I have nothing to do yet.

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 9:24 am
by John Adams
I am >pretty sure< the values get set to 1 ok, because I can refresh the page and see the player. Then later, I refresh again and they are gone (0) but, I still see them online playing with spells and items.

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 1:50 pm
by Scatman
Ya I just looked at it today at work and I'm not exactly sure why that's happening. Might just have to hop on the server and try different things.

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 2:13 pm
by LethalEncounter
John Adams wrote: I personally haven't had the nerve to ask LE what he thinks about splitting World/Zone server ;)
I don't think it would be a good idea. EQEmu uses shared memory to accomplish this, but I think it is more of a headache that it is worth. The benefit of separating World and Zone would be that you could run the zones on another computer (which btw thereby makes the shared memory advantage invalid), but we could link multiple worlds together to create the same concept. When I was designing the emu I specifically decided to avoid the shared memory method that EQEmu used and use threads instead. I think the server is fairly stable at this point in it's life cycle and it will only grow more stable as time goes on. I stand by my decision and think it is correct. However, if anyone has a problem with the design please let me know so that I can come up with a solution if there is a problem. I'm not going to get pissed off for discussing something that I disagree with :P

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 2:16 pm
by John Adams
Heh no way man. I love the design we got. I was just curious how we might handle things once an EQ2Emu world had 200+ connections to one single binary. I know when I did the performance testing with 15 logins, my CPU barely woke up. That was impressive.

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 2:21 pm
by Scatman
The design rocks :) and it is exactly how SOE does it. Each of their computers holds x amounts of zones. I remember when we would have x8 vs x8 battles in Kylong plains, the people raiding Veeshan's Peak would lag out because the developers had to actually intervene and tell us that the two zones were on the same server :D

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 2:25 pm
by LethalEncounter
lol, that is funny. Not even SOE's huge budget and uber network could stop lag on their servers :P Strange that they wouldn't run more servers tho

Re: Server Stats considerations

Posted: Wed Jun 03, 2009 2:29 pm
by Scatman
Hehe ya. It became a huge thing because competing guilds would know the other guild was raiding VP and would constantly zone in raids of their guild into and out of Kylong Plains to crash the server or lag them out in Veeshans. They ended up actually separating the zones from the same computer.

Re: Server Stats considerations

Posted: Thu Jun 04, 2009 4:58 pm
by John Adams
Scat, I think we're going to have to come up with something different for server stats than REPLACE INTO. A World just sitting idle even with no players, the stats threads are running, updating. Well, what I had forgotten about REPLACE INTO is that the function doesn't update the existing record in-place, but deletes the old one and replaces it with a new insert.

http://dev.mysql.com/doc/refman/5.0/en/replace.html

Look at dev's `statistics` table sometime. We've had stats on Dev for maybe a few weeks, and the PK for the table is already over 41,000. I just see no reason why that ID should ever grow to the millions, which it will on long-running servers. Maybe it's just my quirky dislike of things that make no useful sense...

So I'll see if I can come up with an alternative. Maybe if it exists, hit an UPDATE query, otherwise INSERT. Not sure if there's an UPDATE INTO, but I'll check. Either way, toggling between UPDATE and INSERT shouldn't really be a problem.

Performance considerations:

Please note that REPLACE INTO is a much slower performer than an UPDATE statement. Keep in mind that a REPLACE INTO requires a test on the keys, and if a matching unique key is found on any or all columns, a DELETE FROM is executed, then an INSERT is executed. There's a lot of management of rows involved in this, and if you're doing it frequently, you'll hurt your performance unless you simply cannot do with any other syntax.

The only time when I can see where you'd actually need a REPLACE INTO is when you have multiple unique constraints on a table, and need to drop any rows that would match any of the constraints. Then REPLACE INTO becomes more efficient from DELETE FROM... INSERT INTO...

If you're looking at a single unique column table (Primary Key), please use UPDATE, or INSERT. Also, check out INSERT ... ON DUPLICATE KEY UPDATE... as an alternative if you're willing to stick to MySQL 4.1+

I tested this query in query editor, and it seems to do what I want.

Code: Select all

INSERT INTO statistics (char_id, guild_id, stat_id, stat_value, stat_date) VALUES (1, 0, 1000, 18, 1243792314)
	ON DUPLICATE KEY UPDATE char_id = 1, guild_id = 0, stat_id = 1000, stat_value = 17, stat_date = 1243792314;
I'll change dev and give this a try, see what happens ;)

Re: Server Stats considerations

Posted: Thu Jun 04, 2009 5:27 pm
by LethalEncounter
We use the ON DUPLICATE KEY a couple of places in the code, so yours should work just fine too :)