Server Stats considerations

[John Adams]

Er, modules, includes, headers, whatever! Damnit, Jim, I'm a QA expert, not a software engineer!

I guess that's what I was asking. Does world currently include this kinda stuff, and if not, what are they, and can they get included for use, and will they hurt anything by being included?

Also, Image's "profiler" build... I would think perfmon-type counters would already be in there, but I have absolutely no idea how to utilize them. In fact, I have no idea how to do just about any of this, really. I'm just a design guy

[LethalEncounter]

Er, modules, includes, headers, whatever! Damnit, Jim, I'm a QA expert, not a software engineer!

I guess that's what I was asking. Does world currently include this kinda stuff, and if not, what are they, and can they get included for use, and will they hurt anything by being included?

Also, Image's "profiler" build... I would think perfmon-type counters would already be in there, but I have absolutely no idea how to utilize them. In fact, I have no idea how to do just about any of this, really. I'm just a design guy

No, the 'guts' necessary to perform the functions you need are not currently in the code. Unfortunately I also don't know how to get the information you require as I have never needed to get the information before. However I searched on google and although a bit cryptic I managed to find a few examples on how to get the information. Unless it is absolutely necessary, save this for last and I'll get it done for you after you get the stats infrastructure the way you want it. I might not be able to get to it right away but I'll work it in somewhere

[John Adams]

No problem. As I said, only interested in statistical data to help with development. Otherwise, it's just whistles and bells and unnecessary.

If you want to link a few of your discoveries here, we can refer to them at a later date. Gonna hit Scatman up for the "Server Stats Timer" soon

[John Adams]

Scat, I started playing with the new Server Stats timer (thank you for that!) but I have a question about the updates.

For one thing, I'd like the timer to be configurable for all these stats threads - though right now it is not important. But for now, also, I'd like to bump the Server Stats from 60 seconds to every few hours, or even 30 mins. I don't think that needs to run every minute. But other than that, looks awesome.

So the question - I understand how stats change when connections are made, peak connections, etc. But what about relatively static things, like total unique accounts (104), total characters (105), average character level (106)?

These are the SQL's:

Code: Select all

select count(distinct account_id) from characters;
select count(id) from characters;
select round(avg(level)) from characters;

I am just not sure where to stick them. Inside the UpdateServerStats() function?

This goes along with the concept of calculating the "Most/Highest" stats on a cycle. Maybe I shouldn't be using ServerStats for that, but you tell me...

[Scatman]

Do you mean configurable like having a variable in the variable tables?

For the unique accounts, total chars, etc, we can just have it do that every time the timer expires. So, once an hour or however long we move to timer to?

[John Adams]

Right, but does that mean I put the queries/DB update right in the UpdateServerStats() function, directly? Just adding lines of code running replace into queries? Wasn't sure if that was your design or not. I can do it that way.

Ie., change this function:

Code: Select all

void WorldDatabase::WriteServerStatistic(Statistic* stat) {
	if (stat) {
		Query query;
		query.RunQuery2(Q_REPLACE, "REPLACE INTO statistics (char_id, guild_id, stat_id, stat_value, stat_date) VALUES(0, 0, %i, %i, %i)",
			            stat->stat_id, stat->stat_value, stat->stat_date);
	}
}

and just add those queries below it?

[Scatman]

Nah. I'd say do it in World::WriteServerStatistics(). We won't use the dity bit with those stats since they'll be checked every hour or how ever long. Just call database.WriteServerStatistic() explicitly for those under the loop that goes through the list. We won't add those special statuses you're talking about to the list since they'll be updated every time.

[John Adams]

Scat, checking out the data today, I see something odd. Stats 104, 105, and 106 stat_date are wrong. Digits missing or something.

I also noticed in the WorldDatabase::WriteServerStatistic(int32 stat_id, int32 stat_value) function, you have the Timer as Timer::GetUnixTimeStamp only, where in World::UpdateServerStatistic(int32 stat_id, int32 stat_value, bool overwrite), it's setting stat->stat_date as Timer::GetUnixTimeStamp(). Should it have the ()?

[Scatman]

Hmm. The nice (and not nice) thing about vs2k5 is that when you're typing out a function it tries to spell it out for you. I probably hit tab to insert it and forgot the parenthesis (WOW me forget parenthis?) but very weird that it compiled?

Lookin into the other stuff tomorrow morning...

[Image]

How much data are we planning on logging? We will probably need levels of logging, you lose performance spending a lot of time sending these to MySQL.

[John Adams]

Scatman built us some timers, so we're not constantly spamming the DB with data updates. The "Server Stats" that I am thinking of, core OS, Memory, etc, would only update during a longer timer - perhaps every hour or whatever. Point is not to have a performance monitor running constantly, but to give the admin an idea that, at some point during his server run, it used so-much CPU, so-much RAM, etc.

All the data is shoved into a struct, then that struct gets evaluated every 'n' hours (in milliseconds), and pushed to DB. You think that'll keep the load down?

Also the goal is to have these types of options configurable, so the Admin can set the timer length, or turn it off with a 0.

[Scatman]

The stats are cached in memory until a timer fires off. Each stat has a dirty bit that is set when it has been changed so only the stats that have changed since the last write will be written to the database.

[Image]

Seemed more like in the sql example you were putting character stats, why are we trying to monitor cpu and memory usage inside the program? Can just as easily use native programs available in the OS to monitor or download a variety of tools that can be ran such as Cacti. Just trying to understand what we plan to accomplish doing this inside the eq2emu source opposed to just allowing already available tools to do it instead.

[John Adams]

It was just something to consider because other emulators do it. I simply made a list of stats I see others doing, and figured we could too. Doesn't need to be done. None of it needs to be done, because none of it is really that important I guess.

[Image]

I think it is still acceptable to do, I would make the interval customizeable, I am sure in a few years running a eq2emu server will be nothing on the processors.