Antialiasing and Image Smoothing [Chapter 1 Posted!]

Read and write tutorials about becoming a great artist.

Moderators: EN - Assistant Moderators, EN - Forum Moderators

Post Reply
Broocevelt
Moderator
Posts: 7319
Joined: Fri Sep 11, 2009 6:06 pm
Gender: Female
Spoken languages: ES/FR/EN/DE
Location: Spain
Contact:

Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by Broocevelt »

Danielinhoni's
Manual Antialiasing Guide
You're not gonna use the Blur tool, right? I mean c'mon...
Hello! This is Dan, and I've heard you needed help with antialiasing :D (Wait... you didn't? What are you doing here then? Whatever, I'll keep going.)

When spriting or browsing sprites, I've encountered a problem many times: How do I make the lines in my sprites and the shades look smoother? Sometimes some lines of pixels look raw, like sawteeth, and pixels are very easily distinguishable. If you don't know what I am talking about, take a look at this picture:
Spoiler : Antialiasing Example :
Image
LEFT: Antialiasing is OFF
RIGHT: Antialiasing is ON

Of course, that kind of nice and good smoothing is obtained thanks to those fancy computer algorithms and powerful graphic cards; in other words, things that are out of our grasp if we consider we are working with 2D images here, and manually. So how do I smooth out my sprites? Can't I just use the blur tool and see how it goes? The answer is not if you want to obtain optimal and believable results. Tools like that only 'blur' as their name say, and automatic antialiasing isn't just there yet, since your computer doesn't really know what you want. Because of that, we'll need to handle the problem by ourselves, working manually pixel by pixel.

"Pixel by pixel? Come on! We're talking about 256x192 images; you don't expect us to work that much! D:

Well, if you have already worked with things like these, you will probably know that it is not really as hard as it sounds. If you haven't, then this is the chance for you to get used to this! Most of good spriters have worked to the scale of a small bunch of pixels on their screens, since it is something that can solve many problems like these. So what do you say, shall we get started?

The objective of this guide is to be able to blend lines and strokes of pixels as well as areas of pixels into another set of pixels, as smoothly as possible, in order to correct sections of sprites that may look raw/ugly to the viewer's eye

Summary
  • Chapter 1: Principle of antialiasing
  • Chapter 2: Colour and antialiasing (Will be posted soon)
  • Chapter 3: Progressive colour blending
  • Chapter 4: Soft surface smoothening
    (Maybe more chapters to come? Request if you need help on a specific topic)
I will periodically update the summary to include links to the corresponding posts
User avatar
Singidava
Posts: 1621
Joined: Fri Jul 23, 2010 6:56 pm
Gender: Female
Spoken languages: Suomi, English & 日本語
Location: Finland
Contact:

Re: Antialiasing and Image Smoothing [Posting Chapter 1]

Post by Singidava »

Reminds me of antialiasing script I wrote in computer graphics course. :D

GIMP does have an automatic antialiasing filter though. When I tried it out with my sprites, it seemed to work just fine. Or you could just resize the image twice.
Then again, sprites have much clearer edges between colors to antialias than most images.
Broocevelt
Moderator
Posts: 7319
Joined: Fri Sep 11, 2009 6:06 pm
Gender: Female
Spoken languages: ES/FR/EN/DE
Location: Spain
Contact:

Re: Antialiasing and Image Smoothing [Posting Chapter 1]

Post by Broocevelt »

Chapter 1: Principle of Antialiasing
Before being able to smooth and soften our sprites, we need some basics. In this chapter we will see how antialiasing works by taking as a reference a previously antialiased image. After the end of this chapter, you should be able to: Understand how antialiasing works, Smooth a black/white line of pixels.

========================================

How does it work?

Let's get started then! If we want to learn how antialias works, we need something to work from ourselves. Let's take this picture for instance.

Image

Beautiful, isn't it? Two oblique lines of pixels in their natural state, untouched. The one at the top appears to be alright, it is an exception: perfectly diagonal lines of pixels do not usually need smoothing. However, one can see the absolute roughness on the bottom one. It litteraly looks like a stair of pixels, which is the main problem we will encounter during spriting when it comes to smoothing our sprites. Usually, these kind of patterns appear after Frankenspriting or Colouring a section of our sprite, affecting the borders of the involved area(s). How does antialias work with that? Well, let's take a look now at an image coming directly from GIMP (a freeware program similar to Photoshop)!

Image

The lines have different thicknesses and some appear to be grey because they are too thin. However, they all look perfectly smooth. One cannot distinguish the pixels in there, it just looks like a line in real life, and not a badly computer-generated line. How does GIMP do this? To answer this question, we are going to zoom in a bit.
Spoiler : Zoomed In :
Image
From now on we will focus on the thicker line in the middle.

First of all, we notice a couple things:
  • We are using a lot more pixels (other than pure white) than we would without antialiasing
  • Lines of pixels seem to be fading out from the line
To have a better understanding of this, we are going to further analyze the patterns appearing on this line, and I will give some theory as well.
Spoiler : Zoomed in :
Image
We can see some darker dominant pixels, which we will call Driving Pixels, and some fading pixels, which we will call Sinking Pixels. Here they are, and these are their roles:

Image

We should think of the Driving Pixels as the skeleton of the line, and of the Sinking Pixels as the Flesh and Skin of the line. Sinking Pixels will be built around and totally depend on the Driving Pixels. (Alright alright, I know bones don't come first, but you get my point: Driving Pixels are static and define where their surroundings will be) Modifying the position of a single Driving Pixel will entirely affect all directly surrounding Sinking Pixels, and slightly less the ones pixels further away. However, Sinking Pixels do not modify each other directly. How do these Driving Pixels modify their surroundings then? We will first look at the actual application and then proceed to jump to the theory.

When selecting a colour from any program's colour palette, usually numbers proper to a certain colour are given and allowed to be input by the user. In this case, a black line over a white background, we will play around with the brightness of the different pixels. (When blending two different colours together, many more factors come into play, but that will be explained on Chapter 2. We will focus on this much simpler situation.) With this in mind, let's take a look at the brightness of the sinking pixels.

Image

In this scale of colours, there are two key things to take into account: The brightness of the driving pixels, and The brightness of the background on which the sinking pixels are drawn.

For our line, the brightness of our driving pixels is 0, and the brightness of the background (White on both sides) is 240 (Max. brightness on MS Paint). We can observe that progressively, the value of the brightness on the Sinking Pixels transitions horizontaly from the Driving Pixels' brightness to the background's value, the Zero Value, that is the value for which we consider the pixels do not belong to the line; thus, we can say they slowly "sink" into the background, hence their name. In any line of this type, the brightness will change in a more or less constant rate. Here is the evolution of the brightnesses:

(0 > +40 > +32 > +36 > +35 > +37 > [+0] > +36 > [+24]) Values between [brackets] are unexpected values or systematicaly needed values.

We find exactly an average of a progression of +36, it is indeed more or less constant, and should be if there is nothing else interfering with the surroundings. However, there is a small detail we have yet to tacke, and that is the number of pixels you should be using for each row of Sinking Pixels. Let's have a look:

Image

We can see the number of Sinking Pixels exceeds the number of Driving Pixels, which is something that should happen most of the time: you must always avoid conflicting colours in an antialiased environment. Having pure white alongside pure black in an antialiased line is not something you will see most of the time: all rows of Driving Pixels must extend along the previous/next row by means of Sinking Pixels. You can choose to add more Sinking Pixels after the final Driving Pixel is reached if you wish for a bit more smoothness, or if the next adjacent pixel is still way too heavy compared to the background. That is the case in the above picture, which is why 2 Driving Pixels are used to smoothen the transition from dark grey to white.

And that is most of what you need to know to smoothen an oblique line! Of course, this only helps us in achieving the task, but we need to know why this works. How is it that by just fading every single of row of Driving Pixels along each other gives us this effect? We are now in the theoric part of antialiasing.

==========

Why does it work?

Image

It kind of works like gravity: Light objects are attracted to heavy objects, and it works both ways! Weak colours are attracted to strong colours, and it also works both ways! The magic is in the fact that, in our minds, the attraction made by the lighter pixel is predominant, since the strong pixel is attracted to an area in which it stands out more, we don't pay attention to the other displacement. This gives us this result:

Image

The displacement of the lighter pixels gets overridden by the displacement of the Driving Pixels, which is why I didn't really represent it. Of course, this representation is extremely unnacurate, but you probably get the point: Antialiasing gives the illusion of a greater resolution by taking advantage of the displacement our brain makes. And that is the main explanation on why this works, I guess. It all comes down to two things:
  • Colour blending
  • Virtual displacement
==========

The x/y axis dependency

When antialiasing, you will always be antialiasing along lines, which is why the previous explanations were crutial to learning how to antialias. However, you will not always be spreading SPx horizontally, that depends on a certain factor:
Spoiler : Different lines :
Image
This section may be very short, and may seem obvious to some of you, but here is the idea:
  • Lines which are more parallel to the x (horizontal) axis will have their SPx spread horizontally.
  • Lines which are more parallel to the y (vertical) axis will have their SPx spread vertically.
  • Lines which are diagonal will only need SPx between their DPx.
And with this last point stated, comes the last section of this first chapter.

==========

General Rules and Tricks to Antialiasing

This may seem like a silly rule, but it is a little trick you can apply almost anywhere.
Whenever a pixel is trapped between two pixels of a very different colour/brightness, said pixel willl transform to get a closer colour to those two pixels.

For instance, let's take a look at this circle:

Image

It is a yet to be antialiased circle. How should we start working on it? It's simple: Everytime you see a corner made by two pixels, if the corner's colour is way too different from those two pixels, then it desperately needs to become closer to the other two pixels. The more pixels, the closer the colour. Take a look at this circle now:

Image

Much better, right? But not quite there yet, it seems we have some unwanted displacement: Notice how 4 lumps appeared around the circle, and how their inside counterpart appears to be too thick. We are going to use a brighter colour to fill in the corners then, exactly at those spots, so that the displacement is minimal.

Image

The outside ring is now perfect. However, the inside still needs some carving. Instead of brightening the SPx even more (that is a mistake if you already achieved smoothness), we will take out some DPx and replace them with SPx over that spot. We know smoothness will be preserved, as once you have found smoothness in your image, adding intermediary pixels will not affect it; however, the shape of the object will be affected, which is what we are looking for. Let us try these spots:

Image

Now we finally have a nice result! But you may ask... "How did you know exactly which pixels to change and which brightness to use?"
Once you will get used to antialiasing manually, you will see it comes down to pure instinct, but if we really are going to analyze it, I will bring your attention upon two key elements here:
  • Placement: Notice how the pixels I chose are adjacent to the pixel on the spot the displacement is being generated from, that is, the one we changed earlier.
  • Brightness: We must take into account all the straight paths of adjacent pixels that cross this one, that is: compare the upper and lower pixels, as well as the ones on the left and on the right. You must choose a colour that works as an intermediary for both crossings. That is why I chose that darker tone, to match these criteria to keep the smoothness going. We will go into more detail about these "intermediary" colours in the next chapter as well as the progressive blending chapter.
Remember that advice though? Do not let two conflicting colours touch each other in an antialiased environment. I see that happening everywhere though! D:

Image

If we apply some SPx over here to transition from pure black to pure white, we will be able to do two things: stop the colour collision, and give the circle a rounder shape thanks to colour displacement. We will do just that:

Image

However, that's not it. There is still some colour collision and some corners which would need some SPx! Can you spot them all on the above circle? See if your antialiasing instincts were correct opening the spoiler tag! (Trust me, don't just open the spoiler tag, I do really recommend that you try to find at least a couple pixels if you want to start getting the antialias mindset :) )
Spoiler : Solution! :
Image

If you were right, congratulations! You're definitely getting something from this!
If you were not, don't worry, we have plenty of lessons to go :D
As indicated in the above picture, a corner pixel is not necessarily created exclusively by 2 or more DPx, but it can also be created by a combination of both. In fact, if two SPx are different enough to a background colour, they will necessarily create a corner pixel which will need to adapt to those two and the background. I will bring your attention upon the pattern created by the SPx we added.

Look at the darker SPx next to them. Don't they remind you of something...? That's right! That is the fading SPx pattern we found when antialiasing a line! That is because Whenever you will try to smoothen a continuous row of pixels, you will get this kind of pattern. So what does this mean in the context of the antialiased line? It means that when creating those SPx, we were doing something more than displacing the DPx. Think you can tell me what we were doing by continuously adding brighter SPx next to the previously created SPx?
Spoiler : Answer :
We were also trying to prevent a colour collision in the direction of the SPx extension, that is, the very first problem we had here:
Image

We solved the colour collision on the edges of each row by adding a SPx, but we created a new collision. So we add another brighter SPx, then another, then another... Until the colour is bright enough to sink into the background, and we have thus eliminated any colour collision!
So what have we learnt from this antialiased/smoothened circle? Let's go through everything:
  • A corner made by two pixels or more which stand out needs to be smoothened
    That is done by changing the corner pixel by a mixture of both the background and the surrounding pixels.
  • Sinking Pixels can be used to modify shapes
    You can either add new SPx to lengthen a shape of DPx (or other SPx), or replace DPx by SPx to carve out some shapes and correct them.
  • Sinking pixels also smoothen the collision of two colours
    Using an intermediary colour between two strongly colliding colours can be a good idea to smoothen that area. Not always, but most of the time
========================================

Now you know everything you need to know to completely antialias Black & White images, and have all the basis you need to know to learn antialiasing with colours! Expect that guide to be up soon. I hope this was useful to understand how antialias works; if you feel brave, try guessing how this will apply to colours, and try antialiasing any kind of sprite, or just a tiny section! Trust me, transposing this B/W knowledge to colours is way too simple when you know all I have explained you.

Good luck and have fun cleaning sprites!
User avatar
mercurialSK
Posts: 297
Joined: Sat Jan 11, 2014 9:26 am
Spoken languages: English
Location: foolishly fooling like a foolish fool
Contact:

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by mercurialSK »

Pretty cool guide. And quick too. One of the best circle explanations I've seen. I guess I don't have to attempt a guide after all.

There's a lot of theory here but it mainly covers multi-colour blending, which is mainly only really useful for lineart. Will you be covering single-pixel blending too?
Image
Backlog: v6 Perceive generator, Apollo, everything else (My thread)
Someday, on AAO™: Chris Tenson: Ace Attorney & A Turnabout Called Justice (pt 2)
Broocevelt
Moderator
Posts: 7319
Joined: Fri Sep 11, 2009 6:06 pm
Gender: Female
Spoken languages: ES/FR/EN/DE
Location: Spain
Contact:

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by Broocevelt »

I'll try covering the topic that has to do with strengh-of-colour = width of a line, which would apply to a single pixel blending with an area, if that's what you mean, although I'm not really sure that really *is* what you mean. Could you explain a bit more what you mean? I probably know what it is but most of this terminology is something I'm just making up for the sake of consistency in this guide xP Thanks for leaving a comment by the way ^^
User avatar
mercurialSK
Posts: 297
Joined: Sat Jan 11, 2014 9:26 am
Spoken languages: English
Location: foolishly fooling like a foolish fool
Contact:

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by mercurialSK »

No problem. And sorry about my bad description.
What I mainly mean is anti-aliasing in shading, so smoothing blocks of colour instead of in smoothing lines. But also there's the use of anti-aliasing darker for making depth as well, though that might fall under shading instead of anti-aliasing specifically.
Image
Backlog: v6 Perceive generator, Apollo, everything else (My thread)
Someday, on AAO™: Chris Tenson: Ace Attorney & A Turnabout Called Justice (pt 2)
Broocevelt
Moderator
Posts: 7319
Joined: Fri Sep 11, 2009 6:06 pm
Gender: Female
Spoken languages: ES/FR/EN/DE
Location: Spain
Contact:

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by Broocevelt »

Ah, that falls under the category of progressive blending, since to do that you need to use multiple areas of colours which progressively blend the 2 main areas together :) Of course, that will be explained after the coloured application of line antialiasing ;) (Although for AA sprites, usually only 1 intermediary colour is used)
User avatar
mercurialSK
Posts: 297
Joined: Sat Jan 11, 2014 9:26 am
Spoken languages: English
Location: foolishly fooling like a foolish fool
Contact:

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by mercurialSK »

Danielinhoni wrote:(Although for AA sprites, usually only 1 intermediary colour is used)
That's the main part I think is important to mention. I found a fantastic guide that explains anti-aliasing in a simple way, but it deals specifically with 2 intermediary colours instead.
Image
Backlog: v6 Perceive generator, Apollo, everything else (My thread)
Someday, on AAO™: Chris Tenson: Ace Attorney & A Turnabout Called Justice (pt 2)
Phantom

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by Phantom »

I need a example of this guide applied to sprites now.
Broocevelt
Moderator
Posts: 7319
Joined: Fri Sep 11, 2009 6:06 pm
Gender: Female
Spoken languages: ES/FR/EN/DE
Location: Spain
Contact:

Re: Antialiasing and Image Smoothing [Chapter 1 Posted!]

Post by Broocevelt »

I'm going to write chapter 2 very soon so this will soon be applied to sprites, don't worry :P

(I mean unless you have B/W sprites that is)
Post Reply