One of the decisions I'm struggling with is how much time to spend on making it so mods can't break a game.

Here's an example: Right now, if a mod hooks into a manual distribution game and sets every territory to neutral, then nothing will be pickable. The entire map will be neutral, yet players of that game will be asked to pick their starting locations, which is impossible. If it's a multi-player game, then nobody will be able to commit picks and will have to vote-to-end or surrender or be booted. A bad experience all around.

Obviously, a mod should not do that, since it serves no purpose. The question is, should the WarLight framework check for this situation and stop the mod from setting everything to neutral?

Ideally, it should. However, there are a ton of situations like this, and writing checks for every possible bad thing a mod could do would be very time consuming. I'm trying to decide if that work is worth doing. Let's examine the pros and cons.

PRO:

Mod authors will have an easier time coding mods, since if they accidentally do something bad, they get a descriptive error message. In the example above, they'd get a message saying "There must be at least one pickable territory". Without this check, they'd just have to run their mod, realize nothing is clickable, and then go fix the bug.

CON:

If it takes a ton of time to check for every possible bad thing a mod could do, it means we can add fewer hooks to mods. Every time we add a hook, we have to stop and think "How could a mod break the game with this?" and then test the game with each one of those.

What do you think?

I wouldn't recommend checking each edge case. A mod author should be responsible for their edge cases since they know their scenarios best. If a mod is not tested well, it will eventually stop being used by the community due to it's poor quality.

If there are bugs in a mod and the developer can iterate and fix them, I don't see too much of problem with that. I'd rather see time spent enabling more hooks :)

I wouldn't recommend checking each edge case. A mod author should be responsible for their edge cases since they know their scenarios best. If a mod is not tested well, it will eventually stop being used by the community due to it's poor quality.

If there are bugs in a mod and the developer can iterate and fix them, I don't see too much of problem with that. I'd rather see time spent enabling more hooks :)

Yeah, I think you're right. Ultimately it will be a mod author's responsibility to weed out any bugs.

If it really becomes an issue, we can make a "Report a bug in this mod" button that players can click that reports the bug back to the mod author (along with the game link and the mod's printed output and such). This is more effective of a solution than trying to catch every single possible thing a mod could do wrong on its behalf, and will make for a more flexible mod framework (albeit slightly harder to use.)

Edited 4/5/2017 03:54:13

Agree with MotD; if you can come up with some elegant, generalized way to determine whether a game is playable, that would make a nice safeguard. But perhaps only create a noticeable warning rather than banning the mod outright, since it's hard to foresee how mods will be used at this juncture- plus if you try to play it safe, you'll likely end up trimming mods that could be used to make the game unplayable but in practice have some good settings that they were designed to be used with (at least this is what happens in general when people try to avoid all unsafe arbitrary executables- you take out capabilities that would allow an executable to be unsafe but also limit the set of possible safe executables in the process).

Reporting mod bugs would also be a really worthwhile feature; I don't think you can expect mod developers to debug their mods entirely without any Warlight-specific warnings/errors- as mods get more complex, this system will cause more and more of them to be tested in production. But if there's at least a report option for mods, that can be mitigated- say, if the mod creator receives mail about reported problems with their mod.

You could perhaps have some safety features built into Warlight itself- like making sure that the number of pickable territories that actually ends up being used by WL is at least 1 (to use Python here for the sake of example, using something like "max(1, numPickable)" to guarantee that it's at least 0)).

Ultimately though, you've opened a pretty big door by allowing the execution of arbitrary code and I don't think there's any way to make it safe except by stripping features- which could be overkill.

Perhaps, over time, if it becomes an issue, players could create some static analysis tools themselves to weed out mods that are potentially buggy (so like... linting tools for Warlight mods?). But if you don't see a viable and elegant path to keeping mods safe, I would prefer the anarchy. If not having all these safeguards works for assembly languages and C, it would work for Warlight. :P

This'll only make modding tougher, and the development of WL take much longer. Let it be; if it's broken, the modmaker will likely catch it on test to begin with.

Even if not all safeguards are coded, there could still be those for the most common cases.

Edited 4/5/2017 16:20:24

I think this is one of those "time will tell" things. If there are some really bad scenarios that creep up very often, perhaps it'd be good to put some safeguards in place, but I tend to agree that you shouldn't get too hung up on trying to cover every possible scenario. In general, as long as some kind of outcome is possible(such as surrendering/booting as you said, or even just force-ending the game if it's impossible to start), I think you're probably fine in most cases and the "wisdom of the crowds" can weed out the crap mods that don't work.

I think you shouldn't make the checks, but you should implement a rule, that broken games must be voted to end. Cause I already was in such a game and one player didn't want to vote to end and yes... What should I say more to that? Very annoying.

Surrenders should not be dependent on other players' consent in potentially broken games.

Free Modket!

The Lord has a point, though. Modded games should have free surrendering.

Can't you implement a testing grounds, similar to the single player levels?

1) Establish a set of testers (maybe members that opt in?)

2) A game must be played to completion with a winner with that mod script.

3) Develop a set of ground rules for what is acceptable (Sort of like rules for making a map)

4) All of the testers vote if it is an acceptable mod, in accordance with the rules.

5) Once tested and approved, mod goes live.

6) Testers that continually approve mods that are unacceptable are removed from the testes pool.

Edited 4/8/2017 16:49:38