msdfgen parameters

I use the msdf-atlas-gen^[1] utility to generate font bitmaps from MSDFgen^[2]. It has several parameters I’ve previously experimented with:

• size: https://www.redblobgames.com/x/2404-distance-field-effects/#distance-field-resolution^[3]
• pxrange: https://www.redblobgames.com/x/2404-distance-field-effects/#pxrange-setting^[4]
• pxrange vs emrangehttps://www.redblobgames.com/x/2404-distance-field-effects/#debugging-outlines^[5]

However, my earlier experiments were primarily with Fira Sans, and without halos or drop shadows. Since the halos use an extended part of the distance range, I wanted to re-run these experiments to see if I need a different set of parameters. I also want to try the new apxrange / aemrange parameters for asymmetric ranges, as well as other parameters (pxalign , empadding, outerempadding, overlap). And I wanted to update my code to work with MSDFgen 1.12, which changes the glyph coordinate system.

Last time I experimented, it was ad-hoc. I would tweak a parameter and take a screenshot. This time I want to learn how to use a headless rendering library so that I can generate images reproducibly. I had some trouble getting stackgl/headless-gl to work with twgl.js, but just put in workarounds until I got something working. If I’m going to keep using this in the future, I’ll fix it then.

Experiments: display texture size along with rendered output, with outline + halo

• atlas size
• different fonts
• emrange
• aemrange
• empadding
• outerempadding
• overlap

The headless renderer here also is useful for testing parameters like antialiasing width, or gamma values, or different font sizes. I can see all the outputs at once instead of interactively seeing one at a time.

 1  Update msdfgen#

I wanted to document the commands I ran, for my own reference. I had tried vcpkg install msdfgen but that installed the library, and I wanted the command line tool. I already had it checked out in git but needed to updated it:

cd ~/Projects/src/vcpkg
git pull

brew install autoconf automake autoconf-archive

cd ~/Projects/src/msdf-atlas-gen
git pull
git submodule update --recursive --remote

cmake --fresh .
make

The changelog says that the old coordinate system was buggy and they have a new coordinate system. But it looks like the metrics are roughly the same, so I didn’t have to change my parameters.

 2  Bug: interior corners#

While working on this I got annoyed by an outline artifact. I’ve seen this before but didn’t think about it too much. Now that I’m looking closely at individual letters, it bothered me enough to look closely.

Why does this happen? I am using MSDF, which is a multi-channel signed distance field. This handles sharp corners (except for this bug^[6]). But I am also using SDF, a single channel signed distance field, where I want rounded corners. I noticed that all the outline color bleeds are happening with interior corners. I wondered if that correlated with the places msdf and sdf differ, so I added some lines to the shader to override the color:

if (d_msdf > d_sdf * 1.01) gl_FragColor = vec4(1, 0, 1, 1);
if (d_sdf > d_msdf * 1.01) gl_FragColor = vec4(0, 1, 1, 1);

and indeed, the cyan areas are where I want the rounded corners (sdf), and the purple areas are where I want the sharp corners (msdf):

This is easy to fix! I can take the max of the two:

d_sdf = max(d_sdf, d_msdf);

However, I later decided that outlines should always follow msdf and never sdf, so I reverted this change. Halos will always use sdf and outlines always use msdf.

 3  Bug: round outlines#

Fixing this made me think about whether I wanted sdf or msdf for the outlines. In the earlier implementation of outlines, I was using the same code for outlines and halos, and I wanted sdf for halos, so I used sdf for outlines too. But my current implementation uses separate halo and outline code, so I could use msdf for outlines and sdf for halos. I compared the output, and found that using msdf for outlines cleans up some other glitches I had noticed, so I decided to use msdf for outlines.

 4  Bug: drop shadow bleed#

Even setting dropShadow to 0, it’s showing up. I noticed this by setting the drop shadow color to red. Hm. It seems to happen when I have both the drop shadow and halo enabled.

Possible fix: merge drop shadow and halo code

I ended up doing just that — I merged the code, so now you can’t have both drop shadows and halos at the same time.

 5  Test: emrange#

All the tests involve running msdf-atlas-gen and then generating output. This is the original command:

cd ~x/2437-msdfgen-parameters/
cd atlas

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -emrange 0.5 -dimensions 511 511 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json

I want to test a large halo, a thin outline, a thicker font, a thinner font, and a medium sized drop shadow.

This is how I can test the emrange:

for test in 1 2 3 4 5 6 7 8 9
do
    ~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
        -type mtsdf -emrange 0.$test -dimensions 511 511 \
        -font ~/Library/Fonts/FiraSans-Regular.otf \
        -imageout assets/FiraSans-Regular.png \
        -json assets/FiraSans-Regular.json
    node msdfgen-parameters.js
    cp _output.png output/test-emrange-$test.png
done

Wow, I learned a lot from this test.

1. (expected) The halo distance needs emrange to be high
2. (bug) The outline thickness depends on emrange, and shouldn’t
3. (bug) The font thickness depends on emrange, and shouldn’t
4. (bug) The drop shadow distance depends on emrange, and shouldn’t
5. (suspected) The outline quality goes down when emrange goes up
6. (unexpected) There’s a bug that renders the red drop shadow even when the shadow is set to 0

I think the asymmetric emrange feature added to msdfgen will be helpful here, as I can keep the halo range high while not losing as much outline quality. However, I need to fix these bugs to make a better comparison.

I think there may be a precision problem with my intermediate texture, but I can’t easily fix this.

 6  Fix: scaling#

The outline thickness and font thickness depend on emrange but shouldn’t. In particular, columns 3, 4, 5 should look the same in each row:

I thought about this for a while and now think I understand:

The emrange affects the slope of the mapping from distance to the value in the texture. My outline and font thickness are looking at the y value, the encoded sdf. But I want to be looking at the x value, the distance value.

I need to use the distanceRange value stored in the atlas. In the original atlas, with emrange set to 0.5, I had distanceRange=23.828125,size=47.65625. In one with emrange set to 0.9, I have distanceRange=30.881250000000001,size=34.3125. Looking at these ratios, distanceRange/size is equal to emrange. If I want the thresholds to be based on size then I need to divide that by emrange to get the distance range. I think.

To test this:

for test in 2 5 9
do
    ~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
        -type mtsdf -emrange 0.$test -dimensions 511 511 \
        -font ~/Library/Fonts/FiraSans-Regular.otf \
        -imageout assets/FiraSans-Regular.png \
        -json assets/FiraSans-Regular.json
    node msdfgen-parameters.js
    cp _output.png output/test-emrange-fixed-$test.png
done

Columns 3, 4, 5 now look roughly the same in each row:

Drop shadows are still messed up but I may end up reimplementing them a different way, so I don’t want to spend a lot of time tweaking them right now.

 7  Test: atlas size#

I tested emrange of 0.9, which has the worst quality, to see if it would get any better with larger atlas sizes.

for test in 127 128 255 256 511 1023 2047 4095
do
    ~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
        -type mtsdf -emrange 0.9 -dimensions $test $test \
        -font ~/Library/Fonts/FiraSans-Regular.otf \
        -imageout assets/FiraSans-Regular.png \
        -json assets/FiraSans-Regular.json
    node msdfgen-parameters.js
    cp _output.png output/test-atlassize-$test.png
done

1. (unexpected) The drop shadow distance also depends on COMBINED_SCALE, separate from emrange
2. (unexpected) The drop shadow distance also depends on the atlas size
3. I had in my notes that powers of two had some issues with twgl, but at least in this test it didn’t matter (although it may be different for browser webgl vs headless gl, and something I will have to double check later)

It’s clear that the drop shadow implementation needs work.

A higher emrange needs a higher resolution atlas, so I also tried at a more reasonable emrange 0.4 to see how well smaller atlases did:

for test in 127 255 511 1023 2047 4095
do
    ~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
        -type mtsdf -emrange 0.4 -dimensions $test $test \
        -font ~/Library/Fonts/FiraSans-Regular.otf \
        -imageout assets/FiraSans-Regular.png \
        -json assets/FiraSans-Regular.json
    node msdfgen-parameters.js
    cp _output.png output/test-atlassize-$test-emrange-4.png
done

Here 255 was pretty good, and 511 was slightly better. I didn’t see any meaningful improvement past 511.

 8  Fix: drop shadow#

The drop shadow implementation I had put in was a quick & dirty hack, and from the earlier tests it seemed like it didn’t scale well and it didn’t play well with halos.

I decided to remove the drop shadow implementation, and extend halos to support offsets. I changed the halo opacity dropoff to be more like the drop shadow’s. I am not sure this is the right thing to do. But it’s something to investigate later.

I then tested various atlas sizes, COMBINED_SCALE parameters, and emrange parameters, and came up with a way to scale the drop shadow coordinates so that they are independent of those. I tried to make it so that a drop shadow offset of 100 means 1 character width, but right now I don’t know if this worked out because the calculations are correct or just by accident.

In any case, it’s much better than it was before. A drop shadow is a dark halo with an offset. But I think I need to revisit this at some point.

I should have taken before/after screenshots.

 9  Test: aemrange#

The distance field sets 0 at the edge of the font. The interior is <0 and exterior is >0 (or vice versa, depending on convention). I made a visualization of the distance field:

From this we can see that the interior has far fewer contour lines than the exterior. That’s because fonts are relatively thin. If it were a large object like a solid circle, it would have many contour lines on the interior.

The encoding of the distance field maps distances to a value 0.0–1.0, which gets stored in the texture as 0–255. By default, the encoding sets distance=0 to value=0.5 which will be 127.5. Let’s look at how much of the font space is used by each encoded value, with blue being the interior of the font, green the outline, and brown the extended halo:

From this histogram we can see that a large portion of the 0–255 range is wasted. It’d be nice if we could move the distance=0 point to be somewhere else. And in fact, MSDFgen 1.3 supports this, with the aemrange parameter.

Here’s the original command:

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -emrange 0.5 -dimensions 511 511 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json
node msdfgen-parameters.js
cp _distribution.json output/symmetric-distribution.json

and here’s the version that uses aemrange instead of emrange:

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.4 0.1 -dimensions 511 511 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json
node msdfgen-parameters.js
cp _distribution.json output/asymmetric-distribution.json

This uses the available range much better:

In the asset json is distanceRange=19.3875, distanceRangeMiddle=-6.4625. I need to update my code to handle this. The new zero point is distanceRangeMiddle/distanceRange.

Does this improve the quality? I think it's easier to tell with emrange set to 0.8, or aemrange set to -0.7 +0.1 (which preserves the 0.8 distance):

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -emrange 0.8 -dimensions 511 511 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json
node msdfgen-parameters.js
cp _output.png output/test-symmetric-4-4.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.7 0.1 -dimensions 511 511 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json
node msdfgen-parameters.js
cp _output.png output/test-asymmetric-7-1.png

Oh, but it did not improve quality! It's mainly noticeable in the third column. What's going on?

Aha. emrange set to 0.8 is the same as aemrange set to -0.4 +0.4. The first value sets the size in the atlas. When I tried -0.7 +0.1 it made the distance field go out farther, which means less resolution for the letter and more for the halo. I want to keep the halo size constant to make this comparison, so I should only change the second number.

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.4 0.1 -dimensions 511 511 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json
node msdfgen-parameters.js
cp _output.png output/test-asymmetric-4-1.png

The asymmetric version here does look slightly better. The contour lines are closer together, which means we are getting better resolution. It's hard to notice, but the top left curve of the & is a little bit smoother with the asymmetric version. And the outlines on the third column look quite a bit better.

Asymmetry helps by throwing away the range that is unused. But what if it was really needed? The range depends on the font. If it's set too low, we'll "clip" the useful part of the range. I added a line to the shader to detect this:

if (distances.g >= 1.0) gl_FragColor = vec4(1, 0.5, 0, 1);

and here's what it looks like when clipping:

This is something I'll leave in so that when I'm experimenting with different fonts, it'll let me know if something is wrong.

 10  Test: antialiasing edge width#

I reset my atlas for the next experiments.

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.3 0.1 -dimensions 1024 1024 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json

Although my goal for this project was to experiment with msdfgen's parameters, having a headless renderer has been nice for testing the shader parameters too. Previously I was using interactive sliders to see the effect of one value at a time. But now I can generate output for lots of parameters and see them all side by side. For example, the antialiasing edge width parameter:

labels: [1, 2, 5, 10, 15, 20, 25, 30].map((edge_blur, index) => [
    new Label({text: "&", x: -860 + 245*index, y: -125, size: 400, outline: 1.0, edge_blur: edge_blur}),
    new Label({text: (edge_blur/10).toFixed(1), x: -860 + 245*index, y: -170, size: 30, innerColor: "#ffffff", haloColor: "#000000", halo: 200}),
]).flat(),

I had set it to 1.0 by default, and this confirms that 1.0 is a reasonable choice. Lower values from 0.5 down look a little blocky, and higher values from 2.0 up look a little blurry. I decided to zoom into that range:

labels: [9, 10, 11, 12, 13, 14, 15, 16].map((edge_blur, index) => [
    new Label({text: "&", x: -860 + 245*index, y: -125, size: 400, outline: 1.0, edge_blur: edge_blur}),
    new Label({text: (edge_blur/10).toFixed(1), x: -860 + 245*index, y: -170, size: 30, innerColor: "#ffffff", haloColor: "#000000", halo: 200}),
]).flat(),

From eyeballing this, I think maybe 1.2 or 1.3 might be the best choice. I don't have any explanation for this. Could be it 1.25? Could it be sqrt(1.5)? I don't know.

{ tech note : use style="image-rendering: pixelated" when looking at this on the web }

This is a reminder that although it's fun to make things interactive, it's often better to show all the outputs at once instead of requiring interaction to see one output at a time.

TODO also test for small text, including out_bias

 11  Test: other fonts#

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.3 0.1 -dimensions 1024 1024 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular.png \
    -json assets/FiraSans-Regular.json
node msdfgen-parameters.js
cp _output.png output/font-firasans-regular.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.3 0.1 -dimensions 1024 1024 \
    -font ~/Library/Fonts/Lora-Italic\[wght\].ttf \
    -imageout assets/Lora-Italic.png \
    -json assets/Lora-Italic.json
node msdfgen-parameters.js
cp _output.png output/font-lora-italic.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.3 0.1 -dimensions 1024 1024 \
    -font ~/Library/Fonts/Merriweather-Regular.ttf \
    -imageout assets/Merriweather-Regular.png \
    -json assets/Merriweather-Regular.json
node msdfgen-parameters.js
cp _output.png output/font-merriweather-regular.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.3 0.1 -dimensions 1024 1024 \
    -font ~/Library/Fonts/SourceSerifPro-LightIt.otf \
    -imageout assets/SourceSerifPro-LightIt.png \
    -json assets/SourceSerifPro-LightIt.json
node msdfgen-parameters.js
cp _output.png output/font-sourceserifpro-lightit.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.3 0.1 -dimensions 1024 1024 \
    -font ~/Library/Fonts/Vegur-R\ 0.601.otf \
    -imageout assets/Vegur-Regular.png \
    -json assets/Vegur-Regular.json
node msdfgen-parameters.js
cp _output.png output/font-vegur-regular.png

I wanted to compare the quality loss with Fira Sans and with Lora Italic.

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -emrange 0.5 -dimensions 512 512 \
    -font ~/Library/Fonts/FiraSans-Regular.otf \
    -imageout assets/FiraSans-Regular-lo.png \
    -json assets/FiraSans-Regular-lo.json
node msdfgen-parameters.js
cp _output.png output/firasans-regular-lo.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -emrange 0.5 -dimensions 512 512 \
    -font ~/Library/Fonts/Lora-Italic\[wght\].ttf \
    -imageout assets/Lora-Italic-lo.png \
    -json assets/Lora-Italic-lo.json
node msdfgen-parameters.js
cp _output.png output/font-lora-italic-lo.png

~/Projects/src/msdf-atlas-gen/bin/msdf-atlas-gen \
    -type mtsdf -aemrange -0.2 0.05 -dimensions 2048 2048 \
    -font ~/Library/Fonts/Lora-Italic\[wght\].ttf \
    -imageout assets/Lora-Italic-hi.png \
    -json assets/Lora-Italic-hi.json
node msdfgen-parameters.js
cp _output.png output/font-lora-italic-hi.png

I halved resolution and also removed the asymmetric emrange, and it didn't make things much worse with Fira Sans:

But in Lora Italic it made a difference in the third column especially, in this thin section where the two curves come together:

So I think that's something to be aware of — thin curved sections of fonts may require higher resolution atlases.

 12  Test: gamma and out_bias#

In a previous gamma correction test, I had noticed out_bias improved the text rendering too. But this depends on the font size, the bias parameter, and the gamma value. This has been hard for me to figure out.

Comparing gamma with out_bias set to 0:

...Array.from({length: 20}, (_, i) => new Label({x: 600, y: -250 + 24 * i, text: "Live and drink, friend. " + (Math.pow(1.5, (i-10)/10)).toFixed(2), size: 18, gamma: Math.pow(1.5, (i-10)/10)})),
...Array.from({length: 20}, (_, i) => new Label({x: 825, y: -250 + 24 * i, text: "Live and drink, friend. " + (Math.pow(1.5, (i-10)/10)).toFixed(2), size: 18, gamma: Math.pow(1.5, (i-10)/10), innerColor: "#ffffff"})),

Comparing out_bias with gamma set to 1.4:

...Array.from({length: 20}, (_, i) => new Label({x: 600, y: -250 + 24 * i, text: "Live and drink, friend. " + ((i-10)/20).toFixed(2), size: 18, gamma: 1.4, out_bias: (i-10)/20})),
...Array.from({length: 20}, (_, i) => new Label({x: 825, y: -250 + 24 * i, text: "Live and drink, friend. " + ((i-10)/20).toFixed(2), size: 18, gamma: 1.4, out_bias: (i-10)/20, innerColor: "#ffffff"})),

Comparing font sizes with out_bias set to 1/8 and gamma set to 1.4:

...Array.from({length: 20}, (_, i) => new Label({x: 600, y: -250 + 24 * i, text: "Live and drink, friend. " + (8+i), size: 8+i, gamma: 1.4, out_bias: 1/8})),
...Array.from({length: 20}, (_, i) => new Label({x: 825, y: -250 + 24 * i, text: "Live and drink, friend. " + (8+i), size: 8+i, gamma: 1.4, out_bias: 1/8, innerColor: "#ffffff"})),