Duplicates: How to identify a dupe spawn

Tutorials on setting up a server, configuring your client, and connecting to an EQ2Emulator server.
Only moderators can start new topics here
Forum rules
Most information about EQ2Emulator and Tutorials can be found at the Project Wiki. Look there for the most current information.
Locked
User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Duplicates: How to identify a dupe spawn

Post by John Adams » Tue Jul 29, 2008 5:11 pm

(edit: These posts were moved from Content to Tutorials to give you a general idea how we're cleaning things up - and are based on the results of EQ2PacketCollector data and PacketParser.exe results)


First, a definition:
"spawn" - the common definition of an object/npc that can be spawned in a zone.
Currently, there are spawn_npcs, spawn_objects, spawn_widgets, spawn_signs, and spawn_loot. All these tables tie back to "spawn" - the root.
"zonespawns" - the instance of said object/npc that appears in the zone
Currently, we have zonespawngroup, zonespawnentry, and zonespawns, which zonespawnentry is the intersection between a zonespawngroup and a "spawn" id.

An example of one complete spawn would be
1 record in the "spawn" table - ID 1.
1 record in the "spawn_npcs" table, referencing spawn_id 1.
1 record in the "zonespawngroup" table, naming the spawn point in the zone named "antonica_rats1" (example)
1 record (at least) in the "zonespawnentry" table, linking the zonespawngroup to individual spawn_id's
1 record (at least) in the "zonespawns" table coordinating the location where the spawn appears.

Don't forget your two npc_appearance* tables, if you want your NPCs to look a certain way. This topic will be discussed later. For now, we will adjust as needed the collected data (which should be mostly accurate)

Finally, onto Dupes! A "duplicate spawn" has two actual meanings:
1) The more troublesome issue is that we somehow inserted two identical objects into the "spawn*" tables.

Example:
You take 3 logs collected from various sources and run each one through the parser and -populate your DB separately (not the best process, but ok). Chances are, all 3 collected logs will have at least SOME of the same spawns in them. Take Dockmaster Wilson for example. He is in all 3 logs, and now you have a Wilson of spawn_id 120000, 121000 and 122000.
There can be only one. Maybe they should duel? Anyway... if you identify THIS kind of duplicate, you should probably report it to the DB Team to get it cleaned up and re-indexed (until there are tools available for you to do it yourself). Reason being, you will need to re-assign EVERY referring spawn_id that used to point to the old one, to the new one. No small task.

2) The lesser issue is that we simply captured/parsed too many of the same spawn_id in an area of a given zone. Here's an example that anyone who has ever logged into TessEQ2 has seen:
dupes.jpg
If you see a sapwill gobby with 10 "copies" of himself standing around, it is likely 9 of them are re-captures of that 1 gobby wandering around. Everyone remembers this great little quest on the Isle, where you have to help defend the outpost while hordes of these gobbies wander up. Each time the collecter sees a new NPC, it collects it. Thus, now we have 20 of the same NPC wandering from their spawn point to the outpost, when in fact maybe there are 3. Using sound judgement, and your expertise in your selected zone, eliminate the dupes by using the in-game command:

/spawn remove 1 (the 1 removes it from view immediately)

VERY IMPORTANT: Not all duplicates are duplicates! Just thought I would really confuse what might have been a simple task. One word: Placeholders

Sometimes you run passed an NPC collecting data and "an average sparring partner" is standing there. But the next time, or from another log you parsed, you wind up seeing 2 or more standing in the same x,y,z coords:
spar1.jpg
If you look REAL close, you can almost see they are not the same exact models. The only way to really know is to almost peel them apart to see.
spar2.jpg
Well what do you know, they are not dupes! So what this means is that sometimes the first model spawns, sometimes the second, but never at the same time. Voila! Placeholders. Unfortunately, PH's do not currently work so we'll skip those for now (update: 01/01/2009 - Placeholders have been working for some time :)). But eventually we'll need to set their "spawn_percentage" chance in the DB so they do not spawn together anymore.

So please, watch what you delete. It might be valid data!

More to come once this nonsense is absorbed. :)
You do not have the required permissions to view the files attached to this post.
John Adams
EQ2Emulator - Project Ghost
"Everything should work now, except the stuff that doesn't" ~Xinux

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Post by John Adams » Tue Jul 29, 2008 6:20 pm

Wait! Here's one more thing I just thought of. See this guy?
dead.jpg
He's dead, I think. Yet, he stands there. I am not 100% on this, but I think this is a dead corpse that should be laying on the ground about to disintegrate, when the collector collected it's presence. ( LE? )

What we do with these is simply delete the spawn (/spawn remove 1), since we do not need them. They will spawn on the normal cycle (for sapwills, every 30 seconds or so) and walk up to the guards to be murdered.
You do not have the required permissions to view the files attached to this post.

User avatar
John Adams
Retired
Posts: 9684
Joined: Thu Jul 26, 2007 6:27 am
EQ2Emu Server: EQ2Emulator Test Center
Characters: John
Location: Arizona
Contact:

Post by John Adams » Fri Aug 22, 2008 5:11 pm

Now that LE has made our lives so much simpler with the /spawn combine command, I wanted to take a moment to propose when this should and should not be used. Zcoretri and I played around with different concepts extensively, and here is the conclusion we came to.

As described above, some NPCs are the exact same everything - same name, race, gender, face, beard, eyes, armor - everything. Those, definitely duplicates.

Example:
dupes3.jpg
If you just lasso Trapper Coalbear with a /spawn combine 10, you have just saved 3 copies of the same exact NPC to a single spawn group, with a 33.33% chance any of those copies will spawn. What's the point? Delete 2 of them, leave the last one, done.

The point of "cleaning" spawns is not just to make placeholders out 20 of them, but to make the database as lightweight and efficient as possible as well. "Placeholdering" (/spawn combine) should only be used when the pileup of NPCs are unique.

Example 2:
dupes2.jpg
Here you can clearly see, the left group are 3 different NPC, while the right group has 3 different ones (the second ravager looked like a different texture), but 1 of them is a copy. DELETE HIM! :) then /spawn combine 1 the rest.
Make sense? I know we all "get this", but I want to be absolutely clear on how we are grouping our NPCs, because there is no need for excess data if we do not need it.

Something else we learned (and this is why there is a feature request to re-design spawn_script table entries) - if you place the lookout and the ravager in the same spawngroup (as the /spawn combine will do), now they both have to use 1 single spawn_script. But in reality, the two NPCs might need different scripting altogether. I'd like to address this sooner than later in our spawn development, but for now I am giving LE 2 hours to sleep. ;)

Hope this continues to help explain our zone cleanup efforts. Questions? Just ask!
You do not have the required permissions to view the files attached to this post.

LethalEncounter
Team: Zombie
Posts: 2717
Joined: Wed Jul 25, 2007 10:10 pm

Post by LethalEncounter » Fri Aug 22, 2008 5:18 pm

Wow, great work John. I apparently missed this thread until now. It should help out a lot for anyone creating their own worlds.

Locked

Who is online

Users browsing this forum: No registered users and 0 guests