Fixing Disappearing Texture Alpha Details in MipMaps
Posted on:January 22 2017
If you ever programmed a 3D game, you probably came along this problem: You have a texture with alpha channel and lots of small details on them, which disappear when the object is far away or viewed at a sharp angle. This happens for example with foliage, trees, bushes, grass or chain-link fences. Here is an extreme example showing what I mean:
For most objects like the leaves in trees, this isn't a serious problem, most people won't even notice. But for important, fine grained stuff like wire mesh fence as above, this is certainly not wanted. There are some simple workarounds to this problem:
What I did instead for my game was to compute the mip maps slightly different. Instead of calculating the weighted average of the alpha value of each sample as normally when computing the next mip map level, I also compute the maximum alpha value and select a value between the maximum and normal alpha value. And the result was looking a lot nicer:
The result could probably be adjusted a bit, but for me, it looks ok for now. There is no performance impact when rendering this and it is also not slower to generate the mipmaps at all.
For a very simple and badly filtered mipmap, I was originally calculating a pixel for the next level like this:
(Where a, r, g, and b is the total sum for all 4 input samples)
But now I'm doing it this way:
(Where amax is the maximal value alpha had in all samples). refValue is a value which you can adjust to your texture (or even compute it for each texture, if you like), but I figured a value of 0.65 worked nicely for most input images, although it probably is not a perfect value in all cases.
Maybe I'll add this feature into my game engine as well, so that other people will profit from this as well in the future.
For most objects like the leaves in trees, this isn't a serious problem, most people won't even notice. But for important, fine grained stuff like wire mesh fence as above, this is certainly not wanted. There are some simple workarounds to this problem:
- Edit the image and make the alpha channel broader and sharper. But this probably makes your texture look much uglier.
- Adjust the alpha testing reference value for your material. This might work, but if you are unlucky, it doesn't look that nice.
- Disable mipmapping. This looks usually quite nice, but it impacts performance quite a lot
What I did instead for my game was to compute the mip maps slightly different. Instead of calculating the weighted average of the alpha value of each sample as normally when computing the next mip map level, I also compute the maximum alpha value and select a value between the maximum and normal alpha value. And the result was looking a lot nicer:
The result could probably be adjusted a bit, but for me, it looks ok for now. There is no performance impact when rendering this and it is also not slower to generate the mipmaps at all.
For a very simple and badly filtered mipmap, I was originally calculating a pixel for the next level like this:
a /= 4;
r /= 4;
g /= 4;
b /= 4;
newColor.set(a, r, g, b);
(Where a, r, g, and b is the total sum for all 4 input samples)
But now I'm doing it this way:
const float refValue = 0.65f;
a = (int)( (amax*(1.0f - refValue)) + ((a / 4.0f) * refValue) );
r /= 4;
g /= 4;
b /= 4;
newColor.set(a, r, g, b);
(Where amax is the maximal value alpha had in all samples). refValue is a value which you can adjust to your texture (or even compute it for each texture, if you like), but I figured a value of 0.65 worked nicely for most input images, although it probably is not a perfect value in all cases.
Maybe I'll add this feature into my game engine as well, so that other people will profit from this as well in the future.
Comments:
xaos
Quote2017-01-23 10:14:00
and i just realized I should have previewed the comment, as the markup behaves somewhat unusual :-)
xaos
Quote2017-01-23 10:15:00
But with that formatting, it looks quite sophisticated :)
niko
Quote2017-01-23 12:34:00
Unless you actually read it :-)
Just for fun: (33, -54, 164, 164, -54, 33)/256 gives apparently a relatively well-behaved 1d filter (some ringing of course, the truncation not helping, but preserves detail pretty well). It needs clamping of course (and will overshoot somewhat for small transients, but max kind of does that as well)...
Btw. another problem with max is (at least when alpha is premultiplied, which it often is) that mip-levels will tend to become progressively darker when using it solely for alpha. Compensating for that is somewhat cumbersome; I think to stay true to your scheme you would have to basically sort the 4 pixels (or find the one with maximum opacity at least), then blend both color and alpha of the average with the maximum. This then will shift the problem to textures which are inverted forms (so small holes instead of small non-holes), as these would likely become completely opaque in mipmapping (or at least much more opaque than they should be, this is kind of what happens at the mip-level boundary further away).
If building the mipmaps offline it might be worthy to take gamma correction into account (at least usually a value of 128 does not correspond to a real luminance of 0.5, but rather somewhere around 0.75), and use a larger filtering kernel. I would expect your empirical formula to achieve a similar effect in certain cases more by accident, as the problems kind of cancel each other. (Yet another approach would be stochastic sampling maybe, in particular together with alpha-testing, and ideally generating all mip-levels from the source image directly then, with larger and larger sampling areas. You might also check out blue noise and green noise dithering which are roughly related.)
Just for fun: (33, -54, 164, 164, -54, 33)/256 gives apparently a relatively well-behaved 1d filter (some ringing of course, the truncation not helping, but preserves detail pretty well). It needs clamping of course (and will overshoot somewhat for small transients, but max kind of does that as well)...
Btw. another problem with max is (at least when alpha is premultiplied, which it often is) that mip-levels will tend to become progressively darker when using it solely for alpha. Compensating for that is somewhat cumbersome; I think to stay true to your scheme you would have to basically sort the 4 pixels (or find the one with maximum opacity at least), then blend both color and alpha of the average with the maximum. This then will shift the problem to textures which are inverted forms (so small holes instead of small non-holes), as these would likely become completely opaque in mipmapping (or at least much more opaque than they should be, this is kind of what happens at the mip-level boundary further away).
If building the mipmaps offline it might be worthy to take gamma correction into account (at least usually a value of 128 does not correspond to a real luminance of 0.5, but rather somewhere around 0.75), and use a larger filtering kernel. I would expect your empirical formula to achieve a similar effect in certain cases more by accident, as the problems kind of cancel each other. (Yet another approach would be stochastic sampling maybe, in particular together with alpha-testing, and ideally generating all mip-levels from the source image directly then, with larger and larger sampling areas. You might also check out blue noise and green noise dithering which are roughly related.)
xaos
Quote2017-01-23 13:35:00
Add comment:
|
|
Possible Codes
| Feature | Code |
| Link | [url] www.example.com [/url] |
| Bold | [b]bold text[/b] |
| Quote | [quote]quoted text[/quote] |
| Code | [code]source code[/code] |
Emoticons
 |  |  |  |  |  |  |  |
 |  |  |  |  |
Another approach might be using a slightly larger downsampling kernel, kind of like (1 3 3 1)^2, or some other more bicubic-like variant (as this will likely be similar in effect to max, but possibly more well-behaved for iterated application). (1 3 3 1)^2 would have a sum of 64, so could be computed without divisions (and even without multiplications) likewise.