[S] Featured Trial Re-QA

Post about what you like and dislike on AAO and suggest new features and improvements.

Moderator: EN - Forum Moderators

User avatar
Enthalpy
Community Manager
Posts: 5169
Joined: Wed Jan 04, 2012 4:40 am
Gender: Male
Spoken languages: English, limited Spanish

[S] Featured Trial Re-QA

Post by Enthalpy »

The stickied topic for featured trials identifies featured trials as "a list containing our greatest cases on AAO -- with immersive stories and other great features -- that we recommend most of all out of many thousands of cases made by the English community." Well, either that needs to be removed, or some re-analysis is in order.

The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?

Another sizable problem is broken links. Is the community's top recommendation really going to be a case that links to images and sounds that no longer exist? Everything Has A Price fails this check.

Then there are some trials that are simply of questionable quality. The Borgonian Sugar Turnabout needs quality assurance. Five-liners abound, punctuation and grammar are problematic, the speech bubble for objections are not used, and neither are very necessary hide character commands. The thread showcase gives no indication of any quality assurance ever being performed on it. Turnabout Beginnings is effectively playing through the canonical 3-4 from Edgeworth's perspective, except without the exciting bits; nothing of interest is done with Edgeworth's view of the story, so the case ends up being the player expected to make Edgeworth's arguments from the case, but without any hint of fairness for a person unfamiliar with 3-4. Most of the dialogue is copied directly from the canon case's script. The showcase doesn't clearly explain why it was featured, and the only definitive reason I can find was that it was "hard." But that isn't necessarily feature-worthy material.

My recommendation is that the QA team at least does a cursory re-examination of all featured trials, and a standard examination of all featured trials that don't have records of a full-scale QA review in the showcase. If necessary edits need to be made, but can't be, then the trials should be removed. With featured trials ranging from having the authors disagreeing with their featured status to broken links to outright dubious merit, the featured trials list just isn't the "recommended playing" list that it's supposed to be.
[D]isordered speech is not so much injury to the lips that give it forth, as to the disproportion and incoherence of things in themselves, so negligently expressed. ~ Ben Jonson
User avatar
kwando1313
Posts: 7684
Joined: Tue Jul 22, 2008 6:33 pm
Gender: Male
Spoken languages: English, Français (un peu), Ancient Belkan
Location: Uminari City

Re: [S] Featured Trial Re-QA

Post by kwando1313 »

To add to this, can we finally get miscellaneous things featured? Because, as is, it's quite ridiculous that non-AA style things, despite their high quality, cannot be featured.

(For the record, I completely agree with revamping the featured trials list. I was going to ask about this earlier, but me and laziness. xP)
Avatar made by Rimuu~

Image

"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
User avatar
Unas
Admin / Site programmer
Posts: 8850
Joined: Tue Jul 10, 2007 4:43 pm
Gender: Male
Spoken languages: Français, English, Español
Contact:

Re: [S] Featured Trial Re-QA

Post by Unas »

Enthalpy wrote:The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
Because the decision of being in the list or not is not up to the author.
The author decides whether his case is public or not.
The QA team decides whether it's featured or not, depending on whether they think the trial is of high enough quality to catch the interest of visitors and demonstrate the possibilities of the engine.


As for the need to reevaluate the existing featured trial list, yes it'd be a good idea. Especially as far as missing contents are concerned. (Of course, as I mentioned in http://aceattorney.sparklin.org/forum/v ... =13&t=9351 , at some point in the future it should stop being a problem)
Such a re-evaluation would take a lot of work though. You should discuss that with the Trial Reviewers' team, they might ask you to help in this evaluation so that they can keep working on new trials.


And at last, I'm perfectly fine with non-AA content in the featured trials. It's been the case for ages in the French section, where the reviewers feature pretty much anything they want, provided they agree it's good enough.
Due to it being a lot larger and having a lot more released content, the English section can't really work in such a way. That's why Meph oriented the English QA reviews to be more focused on details, and relying on an "objective" measure of quality : compliance to the style of real AA games. It's mainly thanks to Meph's QA reviews that the quality of AAO trials rose to the current level, so I'd rather not have it called ridiculous.
Sure, it might be good to change that criteria and widen the scope of QA reviews, I guess it should be discussed with Tap and the team.
ImageImageImage
If knowledge can create problems, it is not through ignorance that we can solve them.
Si le savoir peut créer des problèmes, ce n'est pas l'ignorance qui les résoudra. ( Isaac Asimov )
User avatar
Bad Player
Posts: 7228
Joined: Wed May 20, 2009 10:53 pm
Gender: Male
Spoken languages: American
Location: Under a bridge

Re: [S] Featured Trial Re-QA

Post by Bad Player »

Unas wrote:
Enthalpy wrote:The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
Because the decision of being in the list or not is not up to the author.
The author decides whether his case is public or not.
The QA team decides whether it's featured or not, depending on whether they think the trial is of high enough quality to catch the interest of visitors and demonstrate the possibilities of the engine.
This seems kinda weird to me, since it's the author who requests the review in the first place...

I feel like re-evaluating trials can be streamlined a lot; a lot of the trials that would be re-evaluated (or more precisely, just plain evaluated) have authors that don't actually come here anymore. I think instead of a full-fledged QA review, (if it's not good enough to be featured anymore) the reviewer would just need to make a short notice saying as such, and quickly giving the reasons why. (Missing images, grammar mistakes, subpar logic, etc.) At that point the author would be able to request a review, and then a full QA review would be done. I just don't think it's worth it to put in the time and effort necessary to QA review all those cases when most of the authors will probably never return and make the required changes.

As for featuring things that aren't serious trials... Would it be possible to divide the featured trials list into two sections (one for serious trials and one for other things), or put in some sort of tag system to help show what's what? I think I'd feel a lot better about it if that could be done.
User avatar
Hodou Okappa
Posts: 5087
Joined: Tue May 05, 2009 7:19 pm
Gender: Male
Spoken languages: English

Re: [S] Featured Trial Re-QA

Post by Hodou Okappa »

Bad Player wrote:
Unas wrote:
Enthalpy wrote:The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
Because the decision of being in the list or not is not up to the author.
The author decides whether his case is public or not.
The QA team decides whether it's featured or not, depending on whether they think the trial is of high enough quality to catch the interest of visitors and demonstrate the possibilities of the engine.
This seems kinda weird to me, since it's the author who requests the review in the first place...

I feel like re-evaluating trials can be streamlined a lot; a lot of the trials that would be re-evaluated (or more precisely, just plain evaluated) have authors that don't actually come here anymore. I think instead of a full-fledged QA review, (if it's not good enough to be featured anymore) the reviewer would just need to make a short notice saying as such, and quickly giving the reasons why. (Missing images, grammar mistakes, subpar logic, etc.) At that point the author would be able to request a review, and then a full QA review would be done. I just don't think it's worth it to put in the time and effort necessary to QA review all those cases when most of the authors will probably never return and make the required changes.

As for featuring things that aren't serious trials... Would it be possible to divide the featured trials list into two sections (one for serious trials and one for other things), or put in some sort of tag system to help show what's what? I think I'd feel a lot better about it if that could be done.
Agreed on every single thing said here.

To expand on the "authors don't decide" thing, isn't it only appropriate that authors should be allowed to choose whether or not their own work gets (possibly unwanted) additional exposure? It seems unfair to continue highlighting trials that even the author doesn't want anymore. Needless to say, I could have easily removed Bitter on my own, but I'm leaving it so far because there's been no official order to take it down and because it helps make the point imo.
Also known as: okappa, houdou.
User avatar
kwando1313
Posts: 7684
Joined: Tue Jul 22, 2008 6:33 pm
Gender: Male
Spoken languages: English, Français (un peu), Ancient Belkan
Location: Uminari City

Re: [S] Featured Trial Re-QA

Post by kwando1313 »

In addition, doesn't the author themselves decide if they want to be QA Reviewed? So, shouldn't they also have the same power to have their trial un-featured if the want to?
Avatar made by Rimuu~

Image

"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
User avatar
Enthalpy
Community Manager
Posts: 5169
Joined: Wed Jan 04, 2012 4:40 am
Gender: Male
Spoken languages: English, limited Spanish

Re: [S] Featured Trial Re-QA

Post by Enthalpy »

The current QA system starts out with a simple check to determine that "it has an overall story and gameplay of astounding quality; it should be really engaging, have interesting contradictions and be really fun", with the second, detail-oriented check done only if it passes check one. If one of those old trials fails check one, there's no need to do check two. I think that would handle a lot of the streamlining that Bad Player and Hodou want.
[D]isordered speech is not so much injury to the lips that give it forth, as to the disproportion and incoherence of things in themselves, so negligently expressed. ~ Ben Jonson
User avatar
kwando1313
Posts: 7684
Joined: Tue Jul 22, 2008 6:33 pm
Gender: Male
Spoken languages: English, Français (un peu), Ancient Belkan
Location: Uminari City

Re: [S] Featured Trial Re-QA

Post by kwando1313 »

Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
Avatar made by Rimuu~

Image

"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
User avatar
Unas
Admin / Site programmer
Posts: 8850
Joined: Tue Jul 10, 2007 4:43 pm
Gender: Male
Spoken languages: Français, English, Español
Contact:

Re: [S] Featured Trial Re-QA

Post by Unas »

Hodou Okappa wrote:To expand on the "authors don't decide" thing, isn't it only appropriate that authors should be allowed to choose whether or not their own work gets (possibly unwanted) additional exposure?
Erm... No, because I don't see any point for that.
If you don't want exposure, you can remove the "released" status on your trial and make it private again.
If you feel it's ready to be shared publicly, then everyone will be able to play it anyway.
kwando1313 wrote:doesn't the author themselves decide if they want to be QA Reviewed?
That's only because we decided on a heavy QA process for the English section, and we can't run that for all released trials. So we work on a per-request basis, but it's not a definite rule. In the French section for example, anyone can nominate anyone else's trial for QA review if they feel it's awesome. And before that, when we didn't have a QA process, the section admins would just chose trials that they loved and add them to the featured trial list.
The featured trial list is, and has always been, an editorial decision. Basically, I decide what goes to the front pages of my website. I delegated this decision-making power to Meph and then the QA teams because I trust them with this work; I never delegated it to all members.

Enthalpy wrote:The current QA system starts out with a simple check to determine that "it has an overall story and gameplay of astounding quality; it should be really engaging, have interesting contradictions and be really fun", with the second, detail-oriented check done only if it passes check one. If one of those old trials fails check one, there's no need to do check two. I think that would handle a lot of the streamlining that Bad Player and Hodou want.
Except that check one is a very vague statement. You are now requesting to demote trials that, actually, were the most engaging and highest quality on the site at the time of their release, and are often still fun. How do you define an obvious limit ?
Again, I'm all for a cleanup of the list, but I think validating check one is actually a big part of the job :-P
kwando1313 wrote:Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
Of course, I think that's a good idea. The QA team can hardly do that effort alone when they also have new trials to review. :-)
ImageImageImage
If knowledge can create problems, it is not through ignorance that we can solve them.
Si le savoir peut créer des problèmes, ce n'est pas l'ignorance qui les résoudra. ( Isaac Asimov )
Trybien
Posts: 3050
Joined: Sat Oct 17, 2009 12:20 am
Gender: Male
Spoken languages: English

Re: [S] Featured Trial Re-QA

Post by Trybien »

Unas wrote:
kwando1313 wrote:Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
Of course, I think that's a good idea. The QA team can hardly do that effort alone when they also have new trials to review. :-)
It might not be a bad idea to set up a separate, temporary team (or sub-group of the QA team) to go through and re-evaluate the cases that were featured before the standard QA system was put in place. It could be made up of site veterans and/or esteemed trial authors/editors. Of course they'd need to be approved with you and Meph (since he's the head of the QA team).
Image
Winner of the “Broken Commandments” Case Competition
User avatar
Bad Player
Posts: 7228
Joined: Wed May 20, 2009 10:53 pm
Gender: Male
Spoken languages: American
Location: Under a bridge

Re: [S] Featured Trial Re-QA

Post by Bad Player »

Considering there actually hasn't been that much QA activity lately, the trials in question don't tend to be that long, and this isn't really a super-urgent project, I don't really see the need for special temp reviewers for this.
User avatar
kwando1313
Posts: 7684
Joined: Tue Jul 22, 2008 6:33 pm
Gender: Male
Spoken languages: English, Français (un peu), Ancient Belkan
Location: Uminari City

Re: [S] Featured Trial Re-QA

Post by kwando1313 »

But... Considering the activity levels of the other QA reviewers, wouldn't it just essentially be you with a little bit of Hodou...?
Avatar made by Rimuu~

Image

"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
User avatar
E.D.Revolution
Posts: 5743
Joined: Mon Jul 26, 2010 9:00 pm
Gender: Male
Spoken languages: English and decent Spanish
Location: Across dimensions, transcending universes

Re: [S] Featured Trial Re-QA

Post by E.D.Revolution »

Trybien wrote:
Unas wrote:
kwando1313 wrote:Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
Of course, I think that's a good idea. The QA team can hardly do that effort alone when they also have new trials to review. :-)
It might not be a bad idea to set up a separate, temporary team (or sub-group of the QA team) to go through and re-evaluate the cases that were featured before the standard QA system was put in place. It could be made up of site veterans and/or esteemed trial authors/editors. Of course they'd need to be approved with you and Meph (since he's the head of the QA team).
Actually, ShadowEdgeworth is the new head of the QA.
Image
User avatar
Bad Player
Posts: 7228
Joined: Wed May 20, 2009 10:53 pm
Gender: Male
Spoken languages: American
Location: Under a bridge

Re: [S] Featured Trial Re-QA

Post by Bad Player »

kwando1313 wrote:But... Considering the activity levels of the other QA reviewers, wouldn't it just essentially be you with a little bit of Hodou...?
Probs (chew)
User avatar
kwando1313
Posts: 7684
Joined: Tue Jul 22, 2008 6:33 pm
Gender: Male
Spoken languages: English, Français (un peu), Ancient Belkan
Location: Uminari City

Re: [S] Featured Trial Re-QA

Post by kwando1313 »

Bad Player wrote:
kwando1313 wrote:But... Considering the activity levels of the other QA reviewers, wouldn't it just essentially be you with a little bit of Hodou...?
Probs (chew)
BP, you sly dog, you...
Avatar made by Rimuu~

Image

"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
Post Reply