0.7.0 and DB Milestone 1

[John Adams]

Re: http://www.eq2emulator.net/phpBB3/viewt ... 4972#p4972

My final bit of moving is this weekend, after that, I hope to get back into doing some DB work - primarily, getting my act together to release this database chunk to the public. For real this time...

With this in mind, I'd like to pitch that we all come together to agree that what we got now is a good 0.7.0 Emu release (it's been over a year since 0.6.5 came out), or see if we can talk LE into finishing the list (above) and release 0.7.0 by years end.

If I can convince you to release, I'd like to bundle the DB release with 0.7.0 so we can justify nuking the databases (again). LE and I will need to work out the details still of how to best deliver the DB. I'm still pushing for a separate (manual) SVN download of DB/Scripts, making it less work on the LoginServer. LE, what do you think now?

• Note: I'd like to go through DB Updater and remove ALL tables not currently being used by EQ2Emulator -at all-, and only deliver tables that actually are used by Emu... no reason Admins should see "character_house_container_blah" tables if they are not used by the server. Clean up the DB, release it as a full, one-time download, and rejoice.

LE, it makes most sense that Admins have a single-source to download from - but I fear the strain this will put on your download services if anyone keeps screwing up a download and re-doing the entire database, content and all, over and over. DB is currently at about 300mb I think (80MB using bulk inserts). Anticipating many more servers popping up with Content, that could really beat the shit out of your bandwidth.

I'd be willing to host a Public SVN for DB Project, to include the Content Database and Scripts from mmoemulators.com, to relieve some pressure from your systems. I also have a tool written in MS-DOS Batch that admins can use to easily load a complete new DB, then patch to LS to stay current. I would change my dump to include every table in the DB, as opposed to only the content subset.

Let's chat. Something I'd really like to see us do soon.

[John Adams]

k here are a couple of additional thoughts (mostly confusion in my mind).

If we use LoginServer to deliver *all* content, always - it would be the most consistent. Except for those who run Custom servers, right? Or, can those admins still get Table updates, but not Data updates (al la LoginServer.ini)? If so, then maybe that's ok.

If someone is getting table updates but not data updates, this means they will never get new Opcodes or Commands?

If we choose to deliver Content via it's own package/SVN, how would we maintain sync with current development? Say for example an Admin downloads 0.7.0 Binaries, sets up his server, connects to LS and gets the core database. Then, sources in the Content milestone release, thus essentially back-revving his database to the moment when 0.7.0 was released... the next time that admin connected to LS, they would get Table *and* Data updates? (see next)

Regardless of how we release the Content DB, changes that occur to current content (fixes, enhancements) will be submitted to DB Updates as data updates, and thus the download_version is bumped so the next connect, the server will get the content modifications. What happens if the admin downloads our content but decides to modify some things, and thus NEW fixes from us will no longer sync?


I love the table updates automation we got, and the "CORE" data automatically being delivered, because those things I think we need for sure. But Content, I am really starting to lean towards it being 100% optional, and manual. Any other concerns I may not have realized yet?

[LethalEncounter]

Re: http://www.eq2emulator.net/phpBB3/viewt ... 4972#p4972

My final bit of moving is this weekend, after that, I hope to get back into doing some DB work - primarily, getting my act together to release this database chunk to the public. For real this time...

With this in mind, I'd like to pitch that we all come together to agree that what we got now is a good 0.7.0 Emu release (it's been almost 2 years since 0.6.5 came out), or see if we can talk LE into finishing the list (above) and release 0.7.0 by years end.

If I can convince you to release, I'd like to bundle the DB release with 0.7.0 so we can justify nuking the databases (again). LE and I will need to work out the details still of how to best deliver the DB. I'm still pushing for a separate (manual) SVN download of DB/Scripts, making it less work on the LoginServer. LE, what do you think now?

• Note: I'd like to go through DB Updater and remove ALL tables not currently being used by EQ2Emulator -at all-, and only deliver tables that actually are used by Emu... no reason Admins should see "character_house_container_blah" tables if they are not used by the server. Clean up the DB, release it as a full, one-time download, and rejoice.

LE, it makes most sense that Admins have a single-source to download from - but I fear the strain this will put on your download services if anyone keeps screwing up a download and re-doing the entire database, content and all, over and over. DB is currently at about 300mb I think (80MB using bulk inserts). Anticipating many more servers popping up with Content, that could really beat the shit out of your bandwidth.

I'd be willing to host a Public SVN for DB Project, to include the Content Database and Scripts from mmoemulators.com, to relieve some pressure from your systems. I also have a tool written in MS-DOS Batch that admins can use to easily load a complete new DB, then patch to LS to stay current. I would change my dump to include every table in the DB, as opposed to only the content subset.

Let's chat. Something I'd really like to see us do soon.

Yah I agree with you that with all the advances we have made since the last release, 0.7 could definitely be released anytime now. Unfortunately I wouldn't have time to finish the list for 6.0 and 7.0 from your link, especially considering I will be visiting family for Christmas for a few days. As for the downloads, if you can make a nice db package that people can use out of the box, then we can easily add it to the project on SF and they can download it there. Faster than they could get it from me anyways. I am probably going to be increasing my upload capability to 3 Mb, but that isn't much when sending files What do you think about the long term possibility of getting rid of the updater built into World and creating a separate app that could be run via the command line or via a nice GUI? That way we could easily add new file types/data updates without modifying the world code to do so. I think we have outgrown the limited capabilities of the current updater.

[LethalEncounter]

If we use LoginServer to deliver *all* content, always - it would be the most consistent. Except for those who run Custom servers, right? Or, can those admins still get Table updates, but not Data updates (al la LoginServer.ini)? If so, then maybe that's ok.

If someone is getting table updates but not data updates, this means they will never get new Opcodes or Commands?

These situations would be perfect for a nice GUI updater that allowed you to specifically choose which tables you want to update. You could choose to only grab changes for opcodes or commands and it will ignore all the other changes.

If we choose to deliver Content via it's own package/SVN, how would we maintain sync with current development? Say for example an Admin downloads 0.7.0 Binaries, sets up his server, connects to LS and gets the core database. Then, sources in the Content milestone release, thus essentially back-revving his database to the moment when 0.7.0 was released... the next time that admin connected to LS, they would get Table *and* Data updates? (see next)

As long as the your db release included a table_versions table, whenever the world connected to the login it would correctly update to any changes that the world could handle.

Regardless of how we release the Content DB, changes that occur to current content (fixes, enhancements) will be submitted to DB Updates as data updates, and thus the download_version is bumped so the next connect, the server will get the content modifications. What happens if the admin downloads our content but decides to modify some things, and thus NEW fixes from us will no longer sync?


I love the table updates automation we got, and the "CORE" data automatically being delivered, because those things I think we need for sure. But Content, I am really starting to lean towards it being 100% optional, and manual. Any other concerns I may not have realized yet?

That is one of the problems with the current system. We cannot specify which data sources to grab. If we could configure it to download DB_XXXReleaseXX.gz and any updates or grab updates from a db somewhere, we could seamlessly have the players databases updated without manual intervention. The only way I see to do all these great things without a huge development effort would be to make the updater program standalone. This program could be the one stop shop for keeping your world updated. The binaries, .xml files, database, etc. We could even add the capability of it updating itself when necessary as well. What do you think? Have I sold you on it yet?

[John Adams]

Definitely, if you're game for a standalone DB updater, I'm totally cool with that. I wasn't sure how you'd react to me suggesting we not use your very clever, very functional DB updater anymore. Seriously, not even being sarcastic. I love the DB updater, just not sure how it'll handle all this data.

I am not too bad at designing UI's, nothing fancy, but for backend work, I'd definitely need someone more familiar with C# or VB.Net to get it all together.

Since I've had so much time off the DB Project to think, I'm certain the only way to release masses of data and stay consistent is to release an entire DB that can be sourced in and patched-to-current after, as you suggest.

[Image]

You could do C#/VB but keep in mind that shuts out the nix users (I guess they can use wine but most servers dont have a GUI). Can't we just take the existing update code and just make a unique build out of it, say the 'Updater'. Then we would need a simple additional exe which could handle the replacement of the Updater executable (Would probably just handle renaming the file and overwriting).

[John Adams]

Yeah after posting that, LE and I had IRC discussions and came to the conclusion a CLI is best for now, with a configurable XML associated to determine what data the admin wants to retrieve from us.

This way when Zexis wants new opcodes, he won't lose his spawn tables should we DROP/CREATE them again.

[ZexisStryfe]

yeah... that would suck...

[John Adams]

I've been talking to LE about trying to finish this new Patcher product he's working on. He'll have to detail how it'll work, since we haven't gone over all the details yet. In general, I think it's going to allow admins to pick what tables/data they want to pull down... so as to not overwrite their own data, in the case of custom servers.

Meanwhile, I'd like to ask the team (Scatman, Zcoretri, Zexis, Image, Ilythor, and anyone else still listening) to go over the proposed features for 0.6.0, 0.6.5 and 0.7.0 and see if there is anything else we definitely want in our 0.7.0 release. I know Guilds are coming probably this week (Scat?) so we'll scratch that one off. Anything else we should consider adding? I'd like to not wait too much longer to release 0.7.0

As for our database - Scatman, I'm going to need to get in-game (on Dev) with you and walk through every zone, see what condition our zones are in, and make a list of what still needs to be done. I'd like to package Server and DB together for this next release (and I'll get my InnoDB/Constraints delivered finally).

Zcoretri - can you and I decide what we're going to do about Spells for release? Again, we may need to get in-game and go over all the dialogs and make sure there are no structs issues (display) and build a few more scripts to get levels 1-10 spells working completely.

I'd like someone to take on the challenge of finalizing the starting_items, starting_skills tables. We had a few people trying to help, but they have dropped off the map.

Items need work, but that's going to be a continuing effort. What we have works well enough, considering the list is over 112,000 items! (Thanks Bion, wherever you are) But in this, we have some SOE changes to items that I just need to know if current Clients will have an issue with (display, structs, etc) - Again Zcoretri, might need you for that.

Guild Support: Major new feature coming soon, which we're not going to just commit and then release 0.7.0. It needs to be tested for at least a few weeks, by us and by players, to work out any kinks (there will be no kinks, right Scat? )

Zexis, we missing any zones?


I'm going to be building shop cabinets for the next few days, so I'll be on and off the computer while I take breaks from driving screws through my fingers. Because of this, I'm not setting any deadlines to get code or DB stuff done, just asking that we get it done sooner than later. I'll try to get my part done by Friday.

[Zcoretri]

I will post more about spells when I get home from work tonight.

[John Adams]

Thanks, Z. I am not looking for additional functionality at this time, but what we might need to do to fix what is currently supposed to be working, and/or finish levels 1-10 for all classes/races. Some things just cannot be done right now (pet's, illusions) but let's figure out what can work.

Thanks!

[ZexisStryfe]

i will make a pass through the zones tables.

[Scatman]

I know Guilds are coming probably this week (Scat?) so we'll scratch that one off. Anything else we should consider adding? I'd like to not wait too much longer to release 0.7.0

Yep this week for sure.

As for our database - Scatman, I'm going to need to get in-game (on Dev) with you and walk through every zone, see what condition our zones are in, and make a list of what still needs to be done. I'd like to package Server and DB together for this next release (and I'll get my InnoDB/Constraints delivered finally).

That's a good idea because I've too forgotten what exactly we've worked on since I haven't touched content in quite a bit.

[Zcoretri]

Thanks, Z. I am not looking for additional functionality at this time, but what we might need to do to fix what is currently supposed to be working, and/or finish levels 1-10 for all classes/races. Some things just cannot be done right now (pet's, illusions) but let's figure out what can work.

Thanks!

Ok I'm home and forgot what the hell I was gonna say, lol.

Anyway, my first goal will be to get the spell names correct for each class (1 -80), then I will concentrate on getting all classes 1-10 spells hooked up with LUA's and spell data, and If there is time attach correct spell visuals.

I would like to spread out the LUA's just a little more with your blessing john, i.e. have a direct damage only lua, a dot only lua etc.

I can also take on the task of the starting stuff and dabble into items as well as needed.

[John Adams]

Yeah, let's make a few more LUA's, I don't mind. I just didn't want 10,000 of them with like "Righteous_Blow.lua" etc lol

...though that certainly would make things very clear, wouldn't it... hmm.