Implementing: LogWrite()

[John Adams]

Sure. I can add that to the configs. What I'll do it use the exact #define CHANNEL_COLOR_* variables so you can make it whatever you want. But I'll make those the hard-coded defaults.

Scat, what about offering a parameter in the logfile->type call, COLOR or something? so if it's not there, use the defaults defined by EQ2Emu - but if there's a specific debug line we want to see during development, we can send COLOR so it stands out?

[John Adams]

One more thing Scat'o...

Will it be hard to receive additional info BACK from LUA? Like right now we have sorta generic errors for LUA. Mostly just run-time syntax... but I am wondering if we can get a little more detailed (or if we really need to?) on what exactly the error might be?

If not, I'd like to DEBUG output any and all params being sent to LUA from C++, during debugging of the LUA system. For that, we could have a setting like LUA__TRACE, which spews every freakin thing going on between the server and the script engine for troubleshooting.

But you tell me (lua master) if you think that would even be beneficial at all --> to the absolute novice script writer... not you, LUA Jedi... lol

[John Adams]

ANNNND one more question... haha.

When this is integrated into the EQ2Emu code, can we ensure this Logging Method can he used as "common" code, throughout all EQ2Emu projects? World, Login, Patcher, Parser, etc? Not sure it would do any good for Collector, but I think it too uses 'common' code (afterthought: we probably don't want Collector doing anything more than it already is, I guess. It's kinda mindless...) .

[Zcoretri]

Scat, what about color coding?

red: error
yellow: warning
green: all cool
white: debug, info, system
etc...


The EVEmu and WoW emu's have this, while I think EQEmu still just does plain white.

Are you talking about color in the server console? If so, that would a great addition

[Scatman]

Sure we can probably output some additional LUA debugging - I'll have to poke around more to see where it'd be beneficial. And yes, I planned on putting the Logging in the common/ trunk.

[John Adams]

Are you talking about color in the server console? If so, that would a great addition

Yes, not in the log files, just the screen output.

evemu-console.jpg

EVEmu Errors

evemu-console2.jpg

ArcEmu (WoW)

arcemu-console.jpg

Note from previous examples I gave, the "Service" field in the log command line is the text we're seeing as bright white, vs normal white or colored text. This helps clearly identify the service/system/function that is generating the log.

[Scatman]

Newest logging code has been committed along with the config file 'log_config.xml'.

[John Adams]

Rockin, Scat! Thanks for that, can't wait to get in there and check it out.



Edit: And, of course I'm asking him to make changes already, so stand by!

[John Adams]

Ok, just for absolute clarity, here's what I'd like the Log Engine to be capable of.

First off, I'd like to build Categories of logging events:

Database
Network
Client
Spawn
Item
NPC
Spells
Quest
Market
Common
Server
Command
etc...

That is the MAJOR categories for logging something. If you turn on or off a Category at this level, ALL logging types for that category are either on or off (see below for an example).


Secondly, I'd like each CATEGORY to be expandable to however we need logging to be broken out in detail. Let's take Database, again.

Database, Message
Database, Error
Database, Query
Database, Result
Database, Trace

Each one of these can be either on or off, individually... with a default set by us as recommended settings (in case log_config.xml is missing).

A sample log_config.xml entry might be:

Code: Select all

<LogConfig Category="Database" Type="Error" Enabled="1" Output="2" Color="FF0000" />

This is an entry to tell the logging system to output Database, Errors to both the log file only (bitwise=2) with a color of RED (Color is optional, as we will set default colors in the code - this tag is only here to allow customization).


This sample log_config.xml shows how you can set the Category to output (or not) every possible database log Type in 1 line:

Code: Select all

<LogConfig Category="Database" Enabled="1" Output="3" />

Notice Type is not defined. That means, output the entire category to the console -and- file (bitwise=3) using the default color.


And yes, Color only effects the console screen, not the Text Log files



A more complete example might be:

Code: Select all

<LogConfig Category="Database" Type="Message" Enabled="0" Output="1" />
<LogConfig Category="Database" Type="Error" Enabled="1" Output="3" Color="934BAC" />
<LogConfig Category="Database" Type="Query" Enabled="0" Output="1" />
<LogConfig Category="Database" Type="Result" Enabled="0" Output="1" />
<LogConfig Category="Database" Type="Trace" Enabled="0" Output="1" />

<LogConfig Category="Network" Type="Error" Enabled="1" Output="3" />
<LogConfig Category="Network" Type="Debug" Enabled="0" Output="1" />
<LogConfig Category="Network" Type="Packet_Trace" Enabled="0" Output="1" />
<LogConfig Category="Network" Type="Packet_Error" Enabled="1" Output="3" />
<LogConfig Category="Network" Type="Op_Error" Enabled="1" Output="3" />

<LogConfig Category="Spawn" Type="Error" Enabled="1" Output="3" />
<LogConfig Category="Spawn" Type="Warning" Enabled="0" Output="1" />
<LogConfig Category="Spawn" Type="Message" Enabled="0" Output="3" />
<LogConfig Category="Spawn" Type="Pop" Enabled="0" Output="1" />
<LogConfig Category="Spawn" Type="Depop" Enabled="0" Output="1" />

<LogConfig Category="Item" Enabled="1" Output="3" />

Here you see Database, Network, Spawn all configured to the Type level, with Item being on globally for all possible defined Item-type logging, outputting to the console and file. Note the COLOR setting only on Database, Error because I happen to be testing something today that I want to stand out in the logs.


Note: Your log_config.xml can have NOTHING in it, or missing entirely, and the Log System will still output our pre-defined log items.

• In the C++ Code, the logging would be initiated with this type of command line:

  Code: Select all

  	if(query.GetErrorNumber() && query.GetErrorNumber() < 0xFFFFFFFF){
  		LogWrite(DATABASE__ERROR, "Error in query '%s': %s", query.GetQuery(), query.GetError());
  	}

  ...and on the Logging engine side, it knows that A) this is a Database category, and B) it is specifically an ERROR log type, so output it according to defaults or config settings.

I do not feel we need:

Code: Select all

	<Log ID="0">
		<Setting Name="Error" />
		<Setting On="true" />
		<Setting Color="red" />
	</Log>
	<Log ID="1">
		<Setting Name="Debug" />
		<Setting On="true" />
		<Setting Color="white" />
	</Log>
	<Log ID="2">
		<Setting Name="Warning" />
		<Setting On="true" />
		<Setting Color="yellow" />
	</Log>
	<Log ID="3">
		<Setting Name="Status" />
		<Setting On="true" />
		<Setting Color="white" />
	</Log>

since we'll be controlling not only enabled/disabled and color from each individual config entry, but I did not want to hard-code Error, Debug, Warning, Status as our only possible options to log things with. Ie., that's why I wanted "Types", so we can easily make new types up as needed.

Anyway, please discuss if this makes no sense.

[John Adams]

On another topic, Scatman... where do you think we should be checking if we even want to submit the Log at all? Should it be configured at the time LogWrite() is being called (which I imagine could bloat our code a lot), or ALWAYS call LogWrite() and check inside that function, which might slow things down a bit with a ton of log checks going on constantly?

Which is better? Or, does it matter to C++?

[Eradani]

On another topic, Scatman... where do you think we should be checking if we even want to submit the Log at all? Should it be configured at the time LogWrite() is being called (which I imagine could bloat our code a lot), or ALWAYS call LogWrite() and check inside that function, which might slow things down a bit with a ton of log checks going on constantly?

Which is better? Or, does it matter to C++?

Imo,
if(error && dowewriteerror) LogWrite();
is much easier to screw up than having dowewriteerror in LogWrite() itself.

You're right on the function call overhead, but it's only going to get to LogWrite() if there's some kind of error or info msg?

When I'm coding something medium to large, I usually put anything I always might want to look at as code and any pure debugging msgs in
#ifdef DEBUG
However, that isn't as flexible as it calls for a recompile.

[Scatman]

Personally, I'd like the checks to get inside of the function call. I would hate to have an if statement encapsulating the function call EVERYWHERE we have a log write. You guys are right, there is overhead associated in setting up a function call but it's honestly not huge. Plus, as Eradani mentioned, the error functions will only be called when something goes wrong (which shouldn't be often ). The debug statements, well it's debug so expect the server to be slower anyways, and things like status updates aren't very often either.

The checks using if's and not #defines is nice because now we can do a /reload log configs and reload settings without a recompile, again as Eradani mentioned. Once we mature the server a little more we start thinking about using macros instead, where we hopefully won't be debugging as often. But for now, I say we leave the checks in the function call itself.

[John Adams]

Perfect. I knew there'd be some overhead, but I know modern processors/ram should make these checks barely noticeable. Hopefully, anyway

Where Error is concerned, yes... rarely see those showing up unless we're developing.

Info (or Status) might be more often, because that's regular friendly console/file spew to the admin so they see the server is functioning normally.

Debug and Trace types are going to be very noisy, at least that's my intent - and my sole motivation for this log system update... because we have GOT to find out wtf is going on with these hangs for no apparent reason, now that normal deadlock trapping seems to have stopped working :p


Just keep in mind, I want this to be flexible enough where those above mentioned 5 are not the ONLY things we can log. Keep in mind the example of Message, Query, Result for Database calls alone... and there are about 100 other systems where non-static log Types will be needed.

Whatever we Send to LogWrite, needs to be interpreted and handled.

[Scatman]

John made me redo the logging system. So it's been redone On SVN.

[Scatman]

So our log config.xml file looks like so

Code: Select all

<LogConfig Category="SERVERINIT" Type="STATUS" Color="0" Enabled="True" LogFile="False" Console="True" />
<LogConfig Category="SERVERINIT" Type="WARNING" Color="2" Enabled="True" LogFile="False" Console="True" />
<LogConfig Category="SERVERINIT" Type="ERROR" Color="3" Enabled="True" LogFile="False" Console="True" />

Well, Mr. Emperor, what do you think of this instead?

Code: Select all

<LogConfig Category="SERVERINIT">
	<LogConfigType Type="STATUS" Color="0" Enabled="True" LogFile="False" Console="True" />
	<LogConfigType Type="WARNING" Color="2" Enabled="True" LogFile="False" Console="True" />
	<LogConfigType Type="ERROR" Color="3" Enabled="True" LogFile="False" Console="True" />
</LogConfig>

More "XMLie" ?

Edit:
I've also changed the value fields of each key to not care about case. Also, the color field can accept a number OR a color's name - (0 or White), (1 or green), (2 or YELLoW), etc