[S] Featured Trial Re-QA
Moderator: EN - Forum Moderators
- Enthalpy
- Community Manager
- Posts: 5172
- Joined: Wed Jan 04, 2012 4:40 am
- Gender: Male
- Spoken languages: English, limited Spanish
[S] Featured Trial Re-QA
The stickied topic for featured trials identifies featured trials as "a list containing our greatest cases on AAO -- with immersive stories and other great features -- that we recommend most of all out of many thousands of cases made by the English community." Well, either that needs to be removed, or some re-analysis is in order.
The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
Another sizable problem is broken links. Is the community's top recommendation really going to be a case that links to images and sounds that no longer exist? Everything Has A Price fails this check.
Then there are some trials that are simply of questionable quality. The Borgonian Sugar Turnabout needs quality assurance. Five-liners abound, punctuation and grammar are problematic, the speech bubble for objections are not used, and neither are very necessary hide character commands. The thread showcase gives no indication of any quality assurance ever being performed on it. Turnabout Beginnings is effectively playing through the canonical 3-4 from Edgeworth's perspective, except without the exciting bits; nothing of interest is done with Edgeworth's view of the story, so the case ends up being the player expected to make Edgeworth's arguments from the case, but without any hint of fairness for a person unfamiliar with 3-4. Most of the dialogue is copied directly from the canon case's script. The showcase doesn't clearly explain why it was featured, and the only definitive reason I can find was that it was "hard." But that isn't necessarily feature-worthy material.
My recommendation is that the QA team at least does a cursory re-examination of all featured trials, and a standard examination of all featured trials that don't have records of a full-scale QA review in the showcase. If necessary edits need to be made, but can't be, then the trials should be removed. With featured trials ranging from having the authors disagreeing with their featured status to broken links to outright dubious merit, the featured trials list just isn't the "recommended playing" list that it's supposed to be.
The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
Another sizable problem is broken links. Is the community's top recommendation really going to be a case that links to images and sounds that no longer exist? Everything Has A Price fails this check.
Then there are some trials that are simply of questionable quality. The Borgonian Sugar Turnabout needs quality assurance. Five-liners abound, punctuation and grammar are problematic, the speech bubble for objections are not used, and neither are very necessary hide character commands. The thread showcase gives no indication of any quality assurance ever being performed on it. Turnabout Beginnings is effectively playing through the canonical 3-4 from Edgeworth's perspective, except without the exciting bits; nothing of interest is done with Edgeworth's view of the story, so the case ends up being the player expected to make Edgeworth's arguments from the case, but without any hint of fairness for a person unfamiliar with 3-4. Most of the dialogue is copied directly from the canon case's script. The showcase doesn't clearly explain why it was featured, and the only definitive reason I can find was that it was "hard." But that isn't necessarily feature-worthy material.
My recommendation is that the QA team at least does a cursory re-examination of all featured trials, and a standard examination of all featured trials that don't have records of a full-scale QA review in the showcase. If necessary edits need to be made, but can't be, then the trials should be removed. With featured trials ranging from having the authors disagreeing with their featured status to broken links to outright dubious merit, the featured trials list just isn't the "recommended playing" list that it's supposed to be.
[D]isordered speech is not so much injury to the lips that give it forth, as to the disproportion and incoherence of things in themselves, so negligently expressed. ~ Ben Jonson
- kwando1313
- Posts: 7684
- Joined: Tue Jul 22, 2008 6:33 pm
- Gender: Male
- Spoken languages: English, Français (un peu), Ancient Belkan
- Location: Uminari City
Re: [S] Featured Trial Re-QA
To add to this, can we finally get miscellaneous things featured? Because, as is, it's quite ridiculous that non-AA style things, despite their high quality, cannot be featured.
(For the record, I completely agree with revamping the featured trials list. I was going to ask about this earlier, but me and laziness. xP)
(For the record, I completely agree with revamping the featured trials list. I was going to ask about this earlier, but me and laziness. xP)
Avatar made by Rimuu~
"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
- Unas
- Admin / Site programmer
- Posts: 8850
- Joined: Tue Jul 10, 2007 4:43 pm
- Gender: Male
- Spoken languages: Français, English, Español
- Contact:
Re: [S] Featured Trial Re-QA
Because the decision of being in the list or not is not up to the author.Enthalpy wrote:The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
The author decides whether his case is public or not.
The QA team decides whether it's featured or not, depending on whether they think the trial is of high enough quality to catch the interest of visitors and demonstrate the possibilities of the engine.
As for the need to reevaluate the existing featured trial list, yes it'd be a good idea. Especially as far as missing contents are concerned. (Of course, as I mentioned in http://aceattorney.sparklin.org/forum/v ... =13&t=9351 , at some point in the future it should stop being a problem)
Such a re-evaluation would take a lot of work though. You should discuss that with the Trial Reviewers' team, they might ask you to help in this evaluation so that they can keep working on new trials.
And at last, I'm perfectly fine with non-AA content in the featured trials. It's been the case for ages in the French section, where the reviewers feature pretty much anything they want, provided they agree it's good enough.
Due to it being a lot larger and having a lot more released content, the English section can't really work in such a way. That's why Meph oriented the English QA reviews to be more focused on details, and relying on an "objective" measure of quality : compliance to the style of real AA games. It's mainly thanks to Meph's QA reviews that the quality of AAO trials rose to the current level, so I'd rather not have it called ridiculous.
Sure, it might be good to change that criteria and widen the scope of QA reviews, I guess it should be discussed with Tap and the team.
- Bad Player
- Posts: 7228
- Joined: Wed May 20, 2009 10:53 pm
- Gender: Male
- Spoken languages: American
- Location: Under a bridge
Re: [S] Featured Trial Re-QA
This seems kinda weird to me, since it's the author who requests the review in the first place...Unas wrote:Because the decision of being in the list or not is not up to the author.Enthalpy wrote:The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
The author decides whether his case is public or not.
The QA team decides whether it's featured or not, depending on whether they think the trial is of high enough quality to catch the interest of visitors and demonstrate the possibilities of the engine.
I feel like re-evaluating trials can be streamlined a lot; a lot of the trials that would be re-evaluated (or more precisely, just plain evaluated) have authors that don't actually come here anymore. I think instead of a full-fledged QA review, (if it's not good enough to be featured anymore) the reviewer would just need to make a short notice saying as such, and quickly giving the reasons why. (Missing images, grammar mistakes, subpar logic, etc.) At that point the author would be able to request a review, and then a full QA review would be done. I just don't think it's worth it to put in the time and effort necessary to QA review all those cases when most of the authors will probably never return and make the required changes.
As for featuring things that aren't serious trials... Would it be possible to divide the featured trials list into two sections (one for serious trials and one for other things), or put in some sort of tag system to help show what's what? I think I'd feel a lot better about it if that could be done.
- Hodou Okappa
- Posts: 5087
- Joined: Tue May 05, 2009 7:19 pm
- Gender: Male
- Spoken languages: English
Re: [S] Featured Trial Re-QA
Agreed on every single thing said here.Bad Player wrote:This seems kinda weird to me, since it's the author who requests the review in the first place...Unas wrote:Because the decision of being in the list or not is not up to the author.Enthalpy wrote:The most basic problem is that some authors want their featured trials taken off the lists. Mimika falls into this category with her Timely Chronograph, as does Hodou Okappa with his The Bitter Turnabout. If not even the authors agree that a trial should be featured, why is it on the list?
The author decides whether his case is public or not.
The QA team decides whether it's featured or not, depending on whether they think the trial is of high enough quality to catch the interest of visitors and demonstrate the possibilities of the engine.
I feel like re-evaluating trials can be streamlined a lot; a lot of the trials that would be re-evaluated (or more precisely, just plain evaluated) have authors that don't actually come here anymore. I think instead of a full-fledged QA review, (if it's not good enough to be featured anymore) the reviewer would just need to make a short notice saying as such, and quickly giving the reasons why. (Missing images, grammar mistakes, subpar logic, etc.) At that point the author would be able to request a review, and then a full QA review would be done. I just don't think it's worth it to put in the time and effort necessary to QA review all those cases when most of the authors will probably never return and make the required changes.
As for featuring things that aren't serious trials... Would it be possible to divide the featured trials list into two sections (one for serious trials and one for other things), or put in some sort of tag system to help show what's what? I think I'd feel a lot better about it if that could be done.
To expand on the "authors don't decide" thing, isn't it only appropriate that authors should be allowed to choose whether or not their own work gets (possibly unwanted) additional exposure? It seems unfair to continue highlighting trials that even the author doesn't want anymore. Needless to say, I could have easily removed Bitter on my own, but I'm leaving it so far because there's been no official order to take it down and because it helps make the point imo.
Also known as: okappa, houdou.
- kwando1313
- Posts: 7684
- Joined: Tue Jul 22, 2008 6:33 pm
- Gender: Male
- Spoken languages: English, Français (un peu), Ancient Belkan
- Location: Uminari City
Re: [S] Featured Trial Re-QA
In addition, doesn't the author themselves decide if they want to be QA Reviewed? So, shouldn't they also have the same power to have their trial un-featured if the want to?
Avatar made by Rimuu~
"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
- Enthalpy
- Community Manager
- Posts: 5172
- Joined: Wed Jan 04, 2012 4:40 am
- Gender: Male
- Spoken languages: English, limited Spanish
Re: [S] Featured Trial Re-QA
The current QA system starts out with a simple check to determine that "it has an overall story and gameplay of astounding quality; it should be really engaging, have interesting contradictions and be really fun", with the second, detail-oriented check done only if it passes check one. If one of those old trials fails check one, there's no need to do check two. I think that would handle a lot of the streamlining that Bad Player and Hodou want.
[D]isordered speech is not so much injury to the lips that give it forth, as to the disproportion and incoherence of things in themselves, so negligently expressed. ~ Ben Jonson
- kwando1313
- Posts: 7684
- Joined: Tue Jul 22, 2008 6:33 pm
- Gender: Male
- Spoken languages: English, Français (un peu), Ancient Belkan
- Location: Uminari City
Re: [S] Featured Trial Re-QA
Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
Avatar made by Rimuu~
"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
- Unas
- Admin / Site programmer
- Posts: 8850
- Joined: Tue Jul 10, 2007 4:43 pm
- Gender: Male
- Spoken languages: Français, English, Español
- Contact:
Re: [S] Featured Trial Re-QA
Erm... No, because I don't see any point for that.Hodou Okappa wrote:To expand on the "authors don't decide" thing, isn't it only appropriate that authors should be allowed to choose whether or not their own work gets (possibly unwanted) additional exposure?
If you don't want exposure, you can remove the "released" status on your trial and make it private again.
If you feel it's ready to be shared publicly, then everyone will be able to play it anyway.
That's only because we decided on a heavy QA process for the English section, and we can't run that for all released trials. So we work on a per-request basis, but it's not a definite rule. In the French section for example, anyone can nominate anyone else's trial for QA review if they feel it's awesome. And before that, when we didn't have a QA process, the section admins would just chose trials that they loved and add them to the featured trial list.kwando1313 wrote:doesn't the author themselves decide if they want to be QA Reviewed?
The featured trial list is, and has always been, an editorial decision. Basically, I decide what goes to the front pages of my website. I delegated this decision-making power to Meph and then the QA teams because I trust them with this work; I never delegated it to all members.
Except that check one is a very vague statement. You are now requesting to demote trials that, actually, were the most engaging and highest quality on the site at the time of their release, and are often still fun. How do you define an obvious limit ?Enthalpy wrote:The current QA system starts out with a simple check to determine that "it has an overall story and gameplay of astounding quality; it should be really engaging, have interesting contradictions and be really fun", with the second, detail-oriented check done only if it passes check one. If one of those old trials fails check one, there's no need to do check two. I think that would handle a lot of the streamlining that Bad Player and Hodou want.
Again, I'm all for a cleanup of the list, but I think validating check one is actually a big part of the job
Of course, I think that's a good idea. The QA team can hardly do that effort alone when they also have new trials to review.kwando1313 wrote:Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
Re: [S] Featured Trial Re-QA
It might not be a bad idea to set up a separate, temporary team (or sub-group of the QA team) to go through and re-evaluate the cases that were featured before the standard QA system was put in place. It could be made up of site veterans and/or esteemed trial authors/editors. Of course they'd need to be approved with you and Meph (since he's the head of the QA team).Unas wrote:Of course, I think that's a good idea. The QA team can hardly do that effort alone when they also have new trials to review.kwando1313 wrote:Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
- Bad Player
- Posts: 7228
- Joined: Wed May 20, 2009 10:53 pm
- Gender: Male
- Spoken languages: American
- Location: Under a bridge
Re: [S] Featured Trial Re-QA
Considering there actually hasn't been that much QA activity lately, the trials in question don't tend to be that long, and this isn't really a super-urgent project, I don't really see the need for special temp reviewers for this.
- kwando1313
- Posts: 7684
- Joined: Tue Jul 22, 2008 6:33 pm
- Gender: Male
- Spoken languages: English, Français (un peu), Ancient Belkan
- Location: Uminari City
Re: [S] Featured Trial Re-QA
But... Considering the activity levels of the other QA reviewers, wouldn't it just essentially be you with a little bit of Hodou...?
Avatar made by Rimuu~
"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."
- E.D.Revolution
- Posts: 5743
- Joined: Mon Jul 26, 2010 9:00 pm
- Gender: Male
- Spoken languages: English and decent Spanish
- Location: Across dimensions, transcending universes
Re: [S] Featured Trial Re-QA
Actually, ShadowEdgeworth is the new head of the QA.Trybien wrote:It might not be a bad idea to set up a separate, temporary team (or sub-group of the QA team) to go through and re-evaluate the cases that were featured before the standard QA system was put in place. It could be made up of site veterans and/or esteemed trial authors/editors. Of course they'd need to be approved with you and Meph (since he's the head of the QA team).Unas wrote:Of course, I think that's a good idea. The QA team can hardly do that effort alone when they also have new trials to review.kwando1313 wrote:Also, if need be, I'm pretty sure you can find lots of people who are willing to re-play featured trials to make sure they meet the first two criteria.
- Bad Player
- Posts: 7228
- Joined: Wed May 20, 2009 10:53 pm
- Gender: Male
- Spoken languages: American
- Location: Under a bridge
Re: [S] Featured Trial Re-QA
Probs (chew)kwando1313 wrote:But... Considering the activity levels of the other QA reviewers, wouldn't it just essentially be you with a little bit of Hodou...?
- kwando1313
- Posts: 7684
- Joined: Tue Jul 22, 2008 6:33 pm
- Gender: Male
- Spoken languages: English, Français (un peu), Ancient Belkan
- Location: Uminari City
Re: [S] Featured Trial Re-QA
BP, you sly dog, you...Bad Player wrote:Probs (chew)kwando1313 wrote:But... Considering the activity levels of the other QA reviewers, wouldn't it just essentially be you with a little bit of Hodou...?
Avatar made by Rimuu~
"The Knight of the Iron Hammer, Vita, and the Steel Count, Graf Eisen. There's nothing in this world we can't destroy."