Page 1 of 1

MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Sun Mar 08, 2020 5:51 pm
by Cadimiom
The CombatProcess for clients is already in a state where MSpawnList is locked, see bool ZoneServer::SpawnProcess()

MSpawnList.readlock(__FUNCTION__, __LINE__);
for (itr = spawn_list.begin(); itr != spawn_list.end(); itr++) {...

CombatProcess(spawn);


problem is CombatProcess calls ZoneServer::KillSpawn while MSpawnList is already locked and then deadlocks.


These are the changes I am testing right now, its unfortunately merged with the disarm code I was working on: http://cutpon.com:3000/devn00b/EQ2EMu/c ... 41d5d434cd

We are still testing this just FYI, but heard this has been an issue for some time, maybe someone can find a better approach here, but this is the jist of the deadlock that causes the client to sit silent in the zone (no commands work, Client Process doesn't work)

I am a bit iffy about saying ZoneServer::CheckDeadSpawnRemoval has a MSpawnList lock (its called from the main zone process loop). However not saying the lock is already in place causes a deadlock.. cause MSpawnList is apparently locked.

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Sun Mar 08, 2020 6:30 pm
by Cadimiom
and an additional deadlock for GetSpawnByID... http://cutpon.com:3000/devn00b/EQ2EMu/c ... a3293cdd95

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Sun Mar 08, 2020 6:59 pm
by Jabantiz
On the first one I see you added a bool to KillSpawn but you never actually use that bool? Did something get left out or just never finished?

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Sun Mar 08, 2020 7:07 pm
by Cadimiom
Added in http://cutpon.com:3000/devn00b/EQ2EMu/c ... a3293cdd95 potential deadlock on GetSpawnByID

So far I have not gotten any reported hangups from the team with the new changes, but still considered a work in progress for now.

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Sun Mar 08, 2020 7:49 pm
by Jabantiz
I see it now, didn't go over the code in the second post before posting, my bad.

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Sun Mar 08, 2020 8:25 pm
by Cadimiom
No problem, going to have a client run over night to see if it can catch anymore deadlocks, I think there might be one left, but I am not sure of the code path so I have it printing out a stacktrace now each time we hit RemoveSpawn

Edit: Yet an additional change due to a mem leak with the code changes: http://cutpon.com:3000/devn00b/EQ2EMu/c ... fe756cc40f

Ran over night with no deadlock in frostfang.

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Mon Mar 09, 2020 5:28 pm
by Jabantiz
Code is committed and will be live on the emu server soon. This one took some time because my lazy way of just downloading the file from your git and dropping it over ours backfired on me this time...

EDIT: I did have to make one small change for it to compile on the linux server, just had to change boolean to bool in client.cpp

Re: MSpawnList deadlock in ZoneServer::KillSpawn

Posted: Mon Mar 09, 2020 9:07 pm
by Cadimiom
Thanks for the heads up on the client.cpp boolean setting, haven't gotten my linux environment up but will be soon.