Code of Ethics for Reviewers and Critics

Find detailed help from the AAO community, or write your own tutorials.

Moderator: EN - Forum Moderators

Code of Ethics for Reviewers and Critics 

Postby E.D.Revolution » Mon Nov 14, 2011 4:35 pm

Another guide co-written by me and Phantom. Some of the stuff in this guide are things discussed in the other guide combined with more content.

As more and more trials are being published via AAO, more and more trials are being subjected to critiques. It's a natural part of publishing: receiving feedback. However, all reviewers and critics are responsible for what they say. They own the review, so they own the consequences linked to that. In order to reduce liability on part of the critics, there is a code of ethics to be established to guide critics and reviewers on their job. Remember, the goal of the review/criticism is to help the trial author improve his/her case and improve his/her writing so that he/she can produce higher quality cases.

But before you decide to write out a critique/review, you need to have a philosophy to guide your writing. The best way to define your philosophy as a critic is to ask yourself the following and ask your self these questions several times deeply:
Spoiler : :
  • What do you think is a good trial? What makes a trial “good?”
  • What do you think is most important in attaining greatness in a trial?
  • What have you noticed about the trials that you’ve played?
  • Why are you critiquing this trial?
  • How open are you to creative licenses?
  • How hard/easy is it to create a trial?
  • Have you created a trial and published one before?
  • What matters more to you? A great, epic story? Memorable characters? Pretty graphics? Or the most authentic case that imitates Ace Attorney at its very finest?
  • What do you consider to be a bad trial? What makes a trial “bad”?


After mulling the questions above, you can start to review cases.

No matter which system of review you use, this code applies to the actual writing of the review.
Spoiler : :
  1. First and foremost, be as honest and complete as you can when you deliver criticisms. Nothing is worse than when a reviewer is missing obvious errors in the review (that are not addressed in any way). If a certain review is clearly lacking sections that others picked up on, it may mean that:
    1. The reviewer doesn't have good eyes.
    2. The critic has extremely low standards.
    3. The critic is holding back his/her true critique.
    4. The critic is being clearly biased (and not admitting to it).
    Your reviews WILL show if you do any of these things listed. As a consequence, your next review will carry less weight due to being perceived as unreliable.
    On the subject of completeness...
  2. Try not to leave your incomplete critique hanging if you have to stop because of life. We all have lives outside of AAO, which is fine. However, you should try to complete your review as soon as you come back. Just like showcasing a trial half finished (outside the context of competitions, as part of an ongoing series of parts, trailers and teasers) will make the author look very unprofessional and sloppy, leaving your critique half-finished will make you sloppy and very unprofessional.
  3. If you play a trial that you want to critique, finish the trial and complete the review. Period. It doesn’t matter if the trial looks very sloppy, half-finished, or just plainly not presentable. The author took the time to work on the Editor to showcase something (even if he/she's trolling). It's very disrespectful to the author to not finish the case all the way through. If you have to take a break from playing for awhile, then do so. But get the job done so the author can look to fixing up his/her case.

    However, if you encounter problems in the trial that don't allow you to continue, you should address it in the thread and not include it in the critique unless you have completed the trial and the problem wasn't resolved by then. Alternatively, you can PM the author yourself (perferred method). These are the only things that need to be addressed before a review (and these should be placed in spoiler tags):
    • A major bug that prevents you from advancing through the trial. Make sure the bug is something that can be fixed and/or Unas already knows about it. If Unas doesn't know about it and the author doesn't know how to fix it, report it to the Bug Reports section.
    • Sudden "Game Over" screens. That's usually when an action that involves "go to x frame" doesn't have a value or is valued at "0." This is more likely to happen in a newbie author's trial than a veteran author's trial.
      As an addendum, do note if you're using save states or advancement links. There is currently a save bug that is messing with Game Over redirects.
    • Parts of the trial you seem to be stuck on and it's perfectly playable.
  4. All criticisms pertaining to trials should stay in the Your Trials section. You should NOT talk about any negative criticism of a trial outside of the section. More likely than not, it's going to look like you're talking crap about the author behind his/her back as a form of slander. Please don't do that. It's unprofessional and never called for. You know the saying: "Whatever happens in Vegas, stays in Vegas." Keep it that way, please.

    On the other side of the token, if you're using a part of your positive criticism to help promote a case, then it's totally fine, so long as you don’t overhype it. If you overhype it, chances are it’ll come off as you seeking attention (including raising expectations that will overrate the trial). It’s just as unprofessional as criticizing outside the trial thread.
  5. Do not let anger fully take over your reviews. At some point, you might get angry over certain sections or aspects of the trial. It’s okay to be angry when playing some parts of the trial. After all, we’re human, not saints. (No, NOT the New Orleans Saints).
    Anger is appropriate in the actual critique when you temper it. Don't let your anger loose while writing a review. It'll look like flaming unless you can solidly back up your reasons for being angry at the parts of the trial.
    As an addendum, don't try to copy That Guy With the Glasses or Angry Video Game Nerd. There's a methodology to their anger in their reviews. When done poorly, it's just cruel.
  6. The critique should contain both positive and negative aspects of a trial. While it's natural to point out the negative aspects of a trial, you should also be looking for things the author has done well. As an author, knowing what you did right is just as important as knowing what you did wrong. Remember: a critique is aimed at helping the author. A review that's full of negative criticisms is not going to help the author in any way.
  7. When you write out the actual review, accentuate the positives first, then bring out the negative. Professional critics will always start their reviews with things that were good about a certain thing before bringing out the bad. That being said, you should try to balance out the time you take to write out the positives with the time you take to write the negatives. Depending on the trial, you might find more things that you can praise or things you can criticize, but in general try to balance it out, with the positives first.
  8. If you do a comparison, make sure that the comparison is fair, and that it makes sense. A comparison is to explain aspects of criticism that can't be said easily or may be complicated to explain in technical terms. Whenever you make a comparison, take into account the context of the thing you're comparing to the thing you're comparing it with/against. For example, if the trial music has unfitting Soul Calibur Music, a fair comparison could be "Soul Calibur is nice to listen to, but it's totally unfitting as court music. Because of the music, I feel as if I'm going to go to war with the cursed sword rather than deliberate my client's case in a court of law."
  9. Try not to compare somebody's work to another person's work if you can't explain it fully. It's just bad form to compare two different pieces of work if there's nothing to note or nothing to justify. As an example, unless a newbie's trial is as good as Blackrune's, don't compare that trial to Blackrune's work.
  10. Choose your vocabulary carefully. When you write the review, be careful on how you word your sentences. You should keep in mind two major concepts: denotation vs. connotation. Denotation is the literal dictionary definition. Connotation is the alignment in terms of positive or negative and how strong a word is compared to a synonym. Connotation can, and often does, lead to misunderstandings. If English is not your strong suit, don’t use synonyms if you don’t know how strong that word is.

    Take for example these two words: dislike and hate. Both are synonyms and mean relatively the same thing. However, hate is a strong word that should be used only in the most serious of situations. There's a difference between hating a book as opposed to disliking a book. Don’t say you hate something when you merely dislike it. Also take into account certain words or phrases may be offensive to the author.


If you’re using a scale or grading system or continuum system to grade the trial, the following code applies only to the actual rating section:
Spoiler : :
  1. Before you start, you must define your baseline averages. What this means is that you should define what is good, what is bad, and what is simply average. In Danielinhoni’s Good point/Bad points system, anything that receives a negative score is considered bad, anything that receives a positive score is considered good and anything that averages out to 0 means it’s average. In a more recent example, E.D.’s grading system listed “C” to “B-” as the baseline average, where anything lower than a “C” means it’s not good/bad and anything above “B” is considered good. If you forget to define your baseline averages, then you must make sure that the author can assume from your review of what your baseline averages are.
    You may look at these examples of how a baseline average is defined.
  2. You need to consistently define how heavy the errors are in relation to your grading system. You should not be all over the place when you consider grading. Example: Two sections, Section A of a trial and Section B of a trial, have exactly one minor spelling error, ceteris paribus (with all other being the same). However, Section A receives an A- and Section B receives a F for the exact error. It makes your review look really unfair and unjustified throughout.
  3. You must make sure you are thorough in your review when you use the grading system. It takes time and effort to review a trial based on a grading system. If you can’t take the time to write up a long report, it’s best you don’t use the grading system at all. See point number 7 for the true reason you need to write up a long report.
  4. You should always tackle the major points that a trial is supposed to succeed on by dividing up the sections (e.g. story, music, presentation, graphics, etc). This makes your review easy to read, easy to follow, and most important of all: easy for the trial author to figure out what he/she needs to work on according to said review.
  5. You must separate a bug report from the review!!! They are not the same thing and should not be combined together. It’s usually convenient for the author if you introduce a bug report FIRST in your post before you post up your review. A bug report consists of obvious errors in a trial, such as spelling errors, improper syntax/grammar, glitches, missing sprites, wrong sounds. This is ONLY supposed to contain minor writing aspects, i.e. things that can be edited.
  6. You should make a “General Observation report.” This report allows you to note what you noticed in the story, use of music, sequence of story, logic. In other words, a “stream of consciousness” report. The only things allowed to be put in this General Observation report are major writing aspects that may need revision.
    • You may separate it from the grading review, or you can combine it with the grade review (so long as the observation comes FIRST before any grading is done). Remember: anything that you note in your General Observation report should guide your review.
  7. The grade needs to be end-result of the review, not the core. What I mean is that the grade is the culmination of each aspect/trial's good and bad points. For this reason, you need to write out objectively what you thought about each aspect/trial and explain it fully before you give the grade.
  8. When you give your grade, do make sure you justify it in some form or way. Even if no one agrees with the grade you give out, giving a justification will paint a clear picture as to why you gave said grade. (It doesn’t matter if it’s an A- to an F).
  9. You may decide to give a mini-review per part to combine it as a whole, OR you may rate the trial overall.
  10. When you think you are finished with your grading, wrap it up with a conclusion of whether or not you, yourself, PERSONALLY think said trial is recommended/feature-worthy. (Either the trial as a whole, or through parts).
  11. When you are completely finished with your grading/review, ALWAYS cross-check your review with at LEAST two different people.This is to ensure you have been completely fair in your grades/justifications throughout your review! It will further your reliability as a critic immensely! However, be careful if one of your cross-checkers is VERY biased regarding the trial you’re reviewing.


I hope you found this code of ethics to be useful. This code should help guide your practice of critiquing and reviewing trials in a responsible manner. Before you start to review, introspect as to what you think a trial should be and ask yourself philosophical questions. Once you have established your philosophy, you can start to review trials. While each person has a different way of reviewing trials, the code of ethics applies to all, in general. As stated before, the grading system has an added responsibility to justify every grade you give. If you follow the code of ethics, you should be within the rules of AAO. And if you do your review correctly, both the author and the critic can benefit from it.
Last edited by E.D.Revolution on Fri Sep 13, 2013 1:16 am, edited 2 times in total.
Image
User avatar
E.D.Revolution
 
Posts: 5721
Joined: Mon Jul 26, 2010 9:00 pm
Location: Across dimensions, transcending universes
Gender: Male
Spoken languages: English and decent Spanish

Re: Code of Ethics for Reviewers and Critics 

Postby Meph » Mon Nov 14, 2011 6:34 pm

E.D.Revolution wrote:I know CardiaX has already written a guide, but to be honest, it doesn't cover enough, and she didn't fix the guide to be more inclusive like I asked her to.

Well to be honest, ED, she's under no obligation to "fix" it with your feedback. I don't want to see you using that attitude, in future.

Anyway, I'm sure everyone will be able to learn something from this. Good job, ED. Cross-checking grades might be a bit difficult, but on the other hand, there are always people willing to express any disagreement they have with grades. ;)
User avatar
Meph
 
Posts: 13439
Joined: Mon Nov 10, 2008 10:07 pm
Location: Probably Disneyland Paris... or the UK
Gender: Male
Spoken languages: English

Re: Code of Ethics for Reviewers and Critics 

Postby E.D.Revolution » Mon Nov 14, 2011 10:37 pm

To be honest, there was a lot of discussion on the other topic, but it wasn't integrated in the main post. And I'm running under the principle that if you write a guide, you should keep your eye on the topic and integrate information related to the topic that was discussed. You can't write a guide - which is designed to help the community - and ignore the activity in it. I don't want to hijack the topic so I'd rather write a new topic about criticism that integrates general criticism as well as graded system. My apologies for poor language.

As for cross-checking grades... It's easier when you have two friends who are also critics/trial authors and have a good sense of what grades are supposed to be. It also helps if they played the case, at the very least.
Image
User avatar
E.D.Revolution
 
Posts: 5721
Joined: Mon Jul 26, 2010 9:00 pm
Location: Across dimensions, transcending universes
Gender: Male
Spoken languages: English and decent Spanish

Re: Code of Ethics for Reviewers and Critics 

Postby Meph » Tue Nov 15, 2011 1:56 am

E.D.Revolution wrote:My apologies for poor language.

It was good of you to apologise, but are you going to remove your comment about it or not?
User avatar
Meph
 
Posts: 13439
Joined: Mon Nov 10, 2008 10:07 pm
Location: Probably Disneyland Paris... or the UK
Gender: Male
Spoken languages: English

Re: Code of Ethics for Reviewers and Critics 

Postby Phantom » Tue Nov 15, 2011 3:03 am

Meph wrote:
E.D.Revolution wrote:I know CardiaX has already written a guide, but to be honest, it doesn't cover enough, and she didn't fix the guide to be more inclusive like I asked her to.

Well to be honest, ED, she's under no obligation to "fix" it with your feedback. I don't want to see you using that attitude, in future.


Meph, you do realize he's just giving his honest motivation as to why he had created the guide in the first place?

Obviously she's under no obligation to "fix" anything, I see what you're saying; but you must consider the fact that he's created a very in-depth guide that paints a clearer picture for critics/authors alike. The more it covers a subject, the better a guide will be.

I've done something similar in a guide I made a long time ago, and you clearly never called me out on the purpose of the guide, so why bust out ED? xP
----------
Also Meph, cross-checking is as ED says: REALLY easier than you think, hence why the reviews are always spot-on fair and justified (and nothing but positive reactions to it, win-win :awesome:)

EDIT:
ED, it's obvious you forgot to include something at the time of this post, SHAME:
Grading system section, number 1...You haven't provided links to examples.
Phantom
 

Re: Code of Ethics for Reviewers and Critics 

Postby Unas » Tue Nov 15, 2011 11:04 am

Phantom wrote:I've done something similar in a guide I made a long time ago, and you clearly never called me out on the purpose of the guide, so why bust out ED? xP

Please, Phantom, I know your love of argumentation, but could you stop questioning our action in matters where you're not involved ? Especially when all the involved parties agree...
ED has agreed that he used "poor language" to express himself (which is the problem here, not the fact that he completed someone else's guide) and reworded it, so your intervention only emphasizes a conflict that doesn't exists :-/
ImageImageImage
If knowledge can create problems, it is not through ignorance that we can solve them.
Si le savoir peut créer des problèmes, ce n'est pas l'ignorance qui les résoudra. ( Isaac Asimov )
User avatar
Unas
Admin / Site programmer
 
Posts: 8801
Joined: Tue Jul 10, 2007 4:43 pm
Gender: Male
Spoken languages: Français, English, Español

Re: Code of Ethics for Reviewers and Critics 

Postby E.D.Revolution » Tue Nov 15, 2011 2:10 pm

Unas wrote:Please, Phantom, I know your love of argumentation, but could you stop questioning our action in matters where you're not involved ?

Unas, I'd have to slightly disagree on that especially when it's on the written on the first line.
E.D.Revolution wrote:Another guide co-written by me and Phantom.

I could actually see where he was getting at. We didn't think it was a problem before we published it and now this happened.

@Phantom, as soon as I find some good examples, I'll post them.

Six replies and none are really on topic. :pearl:
Image
User avatar
E.D.Revolution
 
Posts: 5721
Joined: Mon Jul 26, 2010 9:00 pm
Location: Across dimensions, transcending universes
Gender: Male
Spoken languages: English and decent Spanish

Re: Code of Ethics for Reviewers and Critics 

Postby Unas » Tue Nov 15, 2011 2:57 pm

E.D.Revolution wrote:Unas, I'd have to slightly disagree on that especially when it's on the written on the first line.

Oh, so that first line means that Phantom actually co-wrote your introduction (which was all written with "I", not "we"...) ? :roll:
Again, only your introduction sentence was incriminated. Meph congratulated you for the guide.
So unless you tell me the sentence Meph criticised you about was written by Phantom, then I don't quite see how his post is justified at all - especially after you acknowledged the problem and fixed it anyway.

I'm quite tired of having people (more often than not, Phantom...) questioning everything even when all involved people agree. Phantom, you should remember that our "disagreements" in the past started pretty much the same way, with you criticising Meph's action regarding an event you weren't even involved in, and this after he had already publicly apologised and arranged things with the involved parties...
I'd really prefer if you were more careful about that attitude, I don't want the same story to start again.

But still, now that I'm here, and since ED prefers legitimating the question rather than opening discussion about the guide, let's answer :
Phantom wrote:I've done something similar in a guide I made a long time ago, and you clearly never called me out on the purpose of the guide, so why bust out ED? xP

If you claim that you've been just as rude in the wording and we missed it, then feel free to report your own post and we'll take care of it. Nevertheless, it's not something to be proud about.
ImageImageImage
If knowledge can create problems, it is not through ignorance that we can solve them.
Si le savoir peut créer des problèmes, ce n'est pas l'ignorance qui les résoudra. ( Isaac Asimov )
User avatar
Unas
Admin / Site programmer
 
Posts: 8801
Joined: Tue Jul 10, 2007 4:43 pm
Gender: Male
Spoken languages: Français, English, Español

Re: Code of Ethics for Reviewers and Critics 

Postby E.D.Revolution » Tue Nov 15, 2011 3:55 pm

Unas wrote:
E.D.Revolution wrote:Unas, I'd have to slightly disagree on that especially when it's on the written on the first line.

Oh, so that first line means that Phantom actually co-wrote your introduction (which was all written with "I", not "we"...) ? :roll:
Again, only your introduction sentence was incriminated.
So unless you tell me the sentence Meph criticised you about was written by Phantom, then I don't quite see how his post is justified at all - especially after you acknowledged the problem and fixed it anyway.


Unas, I'm severely offended that you would think that. I'm not completely disagreeing with everything. I'm just saying that we both wrote the guide in general, and that we generally knew the contents. We didn't think there was a problem with the introduction, is all I'm saying. I'm not justifying everything that Phantom is saying about, especially not about where he's disagreeing with Meph. I'm just saying that I can see where he thinks that, but that's that. Period. Do I have to repeat myself again?

And let's not nitpick whether I said "I" vs "we." For all intents and purposes, I = we. After I look for said examples, I'll probably change I -> we.

And the only reason I'm "legitimizing the question" ( :roll: ) is to address any concerns. That is all.

~~~

This matter is closed. Now. If anyone want to continue the above conversation, I can always set up a conference; over tea, crumpets, and cucumber sandwiches. Or over guyere, a bottle of burgundy, and some queen foie gras :P

I want to see what other people think of the code of ethics.
Image
User avatar
E.D.Revolution
 
Posts: 5721
Joined: Mon Jul 26, 2010 9:00 pm
Location: Across dimensions, transcending universes
Gender: Male
Spoken languages: English and decent Spanish

Re: Code of Ethics for Reviewers and Critics 

Postby Phantom » Thu Nov 17, 2011 12:06 am

Find some good examples? Easy, the reviews you did are good enough atm (especially since they are cleaned up and formatted to be easier to read than the reviews I've done).
Phantom
 

Re: Code of Ethics for Reviewers and Critics 

Postby E.D.Revolution » Thu Nov 17, 2011 12:37 am

Besides mine, Phantom :P
After all, I'm trying to find Dan's Good/Bad points somewhere else BESIDES Tap's topic.
Image
User avatar
E.D.Revolution
 
Posts: 5721
Joined: Mon Jul 26, 2010 9:00 pm
Location: Across dimensions, transcending universes
Gender: Male
Spoken languages: English and decent Spanish

Re: Code of Ethics for Reviewers and Critics 

Postby Phantom » Thu Nov 17, 2011 12:49 am

I'm pretty sure on the English section, Dan had only done one review based on that, so you're kinda out of luck xP.

You're still out of luck actually, because obviously I'm the only other person who's done that type of review (and *ahem* the first to do it :P).

To remedy this, you should at least show my review as a "rough draft" w/o proper formatting, vs. yours (which is an actual formatted/completed one that was envisioned).


*PSSSSST*, You should spread the word about me being teh first guy to do it ;P
Phantom
 

Re: Code of Ethics for Reviewers and Critics 

Postby Meph » Thu Nov 17, 2011 12:51 am

I guess we all like to do our reviews differently. These days, after my QA reviews, I like to give a little normal review as well.
User avatar
Meph
 
Posts: 13439
Joined: Mon Nov 10, 2008 10:07 pm
Location: Probably Disneyland Paris... or the UK
Gender: Male
Spoken languages: English

Re: Code of Ethics for Reviewers and Critics 

Postby E.D.Revolution » Thu Nov 17, 2011 1:01 am

Meph wrote:I guess we all like to do our reviews differently.


Hence this is why this is a general code of ethics. :april:

@Phantom, really? He has done only one in the English section? Then why did I hear some stories of bragging if there's only case? [/acting on rumor]
Image
User avatar
E.D.Revolution
 
Posts: 5721
Joined: Mon Jul 26, 2010 9:00 pm
Location: Across dimensions, transcending universes
Gender: Male
Spoken languages: English and decent Spanish

Re: Code of Ethics for Reviewers and Critics 

Postby Phantom » Thu Nov 17, 2011 1:07 am

Indeed Meph, which is a good thing ED separated out the grading review to have its own dedicated section imo.

That's not to say normal reviews are bad- They're really handy for people who don't have time to write an essay's worth (literally!), or just quickly want to get the general gist of the case down for the author.

Also ahh yeah @ED, I don't quite get the last part of your post :P
Phantom
 

Next

Return to Tutorials

Who is online

Users browsing this forum: No registered users and 1 guest