Since my previous posts last year, there have, unfortunately, been numerous instances of people misusing Asphalt. Between our last post and today, the 2nd of May 2023, there have been 184 more instances.
Our previous command flagging system was tedious, in that we were only scanning one command, our most popular command (/play). This system still had a large amount of manual effort of going to the database and running a series of queries to fetch instances of misuse from within other commands, not just /play.
So, in v5.5.1 (released on the 15th of April 2023, so just over 2 weeks from the date of writing this) – our brand-new flagging system went live. During those 2 weeks, Asphalt successfully flagged (successfully being quantified as leading to a valid block put in place on that user) 38 people for misusing Asphalt.
For our new system, and considering the number of commands being run on a daily basis, we ruled out manually analysing every command from every user. We instead opted to use Azure’s Content Moderator (https://azure.microsoft.com/en-gb/products/cognitive-services/content-moderator) to handle this for us (referred to as CM from this point forward).
Now, I know what you may be thinking: “what about false positives?”, which is why we only take what CM returns as a guideline. For example, a response from CM looks like this:
If you notice the “ReviewRecommended” field, that is what we use from CM to determine whether or not we should flag this command to our support team for manual review. This way it ensures that only things that could break our rules will be flagged to the team.
The classification categories that CM provides us with are as follows:
- Classification 1: Sexually Explicit/Adult
- Classification 2: Sexually Suggestive/Mature
- Classification 3: Offensive (e.g. Racism)
If a review is recommended, it will be sent into a channel that all of our support team are able to communicate in and discuss flagged commands. They then have access to see what the input was for this command and who executed it.
We can then either click “Block” to automatically block this user for violation of our rules or if we deem this to not be worth a block (or is a false positive) then we can click “Ignore”, which is self-explanatory.