AlphaPixel often gets called upon to work on legacy codebases, sometimes VERY legacy. We have contact with code from the 80s and 90s on a regular basis, in a variety of dialects and languages, and stored and archived in various difficult containers and mediums.
While NDAs and confidentiality mean we often can’t talk about our paid projects, we recently had an interesting side project that used the same processes, only it was all for fun, so we can talk all about it.
The task: Revive the Amiga source code for a basic educational raytracer written by our friend and early mentor, Brian Wagner.
In August/September 1991, during the era of powerful “big box” Amiga computers, when the Amiga was at the top of its game in the graphics and performance computation world, our friend Brian wrote a couple of articles for a short-lived programming magazine named AmigaWorld Tech Journal (henceforth AWTJ). At the time, I read the article but never compiled and ran the code. Recently, while mentoring some computer graphics Padawans myself, I had the bright idea to revisit that super-simple explanation of ray-tracing in its most basic form. But there were Difficult Problems to Solve.
In Part 1 of this blog series, we located the legacy source code online and assembled a set of tools to bring the code into a modern environment intact. In Part 2, we updated the code and got it running with exactly the same abilities it originally had, using a modern compiler and operating system. In Part 3 we restored the original windowed preview GUI (and made it better!).
We’re starting with the code from the end of Part 3, which has a nice preview UI:
How do we stop our SDL program from freezing when busy or interacting with the window?
To start we should probably add an “event pump” to process UI events coming from SDL, so the program doesn’t freeze when you try to interact with it. This will allow it to be aborted cleanly by closing the window.
Let’s change the GUI code near the bottom of traceimage() in image.c to be
#ifdef WINDOWED_UI
SDL_SetRenderDrawColor(rp, (Uint8)color.r, (Uint8)color.g, (Uint8)color.b, 255);
SDL_RenderDrawPoint(rp, i, j);
// present graphics and run event loop
static unsigned int PresentInterval;
// only do the work every 6400 pixels traced, to minimize overhead
if(!((++PresentInterval + 1) % 6400)) {
SDL_RenderPresent(rp);
if (SDL_PollEvent(&event) && event.type == SDL_QUIT)
return; // kind of an abrupt exit, but the code is simple enough to do it this way
}
#endif // WINDOWED_UI
All this is doing is running an event processing poll every time we Present the rendering (every 6400 pixels traced). This allows blocking UI interactions like window moves to work, and we also check the event queue to see if the user clicked the Window close button. If they did, we bail out of traceimage() abruptly with a return statement, because there are no dynamic resources to take care of anyway. Normally I’d prefer to set an exit flag and let the two for loops cleanly end, allowing any end-of-function processing and cleanup to finish, but I know that there isn’t any, so in the interest of expediency, we just return. I had to actually slow my rendering down in order to keep the window onscreen long enough to test this, but it does work great.
Let’s commit it! 4f859ae7e6d55a65c8088ae77cc4a143ea49c679.
Now, the next BIG task is to add an image saver for some kind of modern image format.
How do we quickly add a PNG saver for our RGB image?
Sometime down the road we could rig up GD, ImageMagick or CImg or similar, but we want quick, cheap and cheerful, and hopefully something really easy to pull in via vcpkg. CImg is tempting because it’s header-only, but it’s also a C++ Template, and we’re just C right now. For now, let’s try libspng. It’s C, header-only, BSD-2-clause, straightforward API surface and supports a very common and well-regarded 24-bit image format, PNG.
Let’s edit vcpkg.json to request libspng:
{
"dependencies": [
"sdl2",
"libspng"
]
}
And then add to CMakeLists.txt in the appropriate places: find_package(SPNG CONFIG REQUIRED) target_link_libraries(tracer PRIVATE $<IF:$<TARGET_EXISTS:spng::spng>,spng::spng,spng::spng_static>)
We’ll also define OUTPUT_PNG to tell the image saving code we’re about to write to use libspng.
I’m using the example.c from libspng ( https://github.com/randy408/libspng/blob/v0.7.3/examples/example.c ) for inspiration, and mimicking the original writeRGB to minimize code changes, even though the structure of writeRGB is not great to begin with. At least my PNG saver resembles the RGB writer.
One complication is that the PNG saver wants the R, G and B pixels interleaved one after another in the memory buffer passed to the image writer. So, we need to temporarily allocate one triple-size buffer the size of the R, G and B buffers output by the tracer, and then pixel by pixel, interleave the R, G, B.
This looks like
// perform interleaving
for(unsigned long int outbyte = 0, yrow = 0; yrow < scrh; yrow++)
for(unsigned long int xcol = 0; xcol < scrw; xcol++)
{
size_t imageoffset = yrow * scrw + xcol;
image[outbyte++] = red[imageoffset];
image[outbyte++] = grn[imageoffset];
image[outbyte++] = blu[imageoffset];
}
The only other thing that confused me was the fmt argument to spng_encode_image. I thought I would need to pass SPNG_FMT_RGBA8 but it turns out that SPNG_FMT_PNG is what I actually want. It basically means “figure it out from the ihdr.color_type = SPNG_COLOR_TYPE_TRUECOLOR in the ihdr structure”.
And now we get a PNG output that looks like
Which is exactly what it looked like before, only now we get it in a PNG file instead of having to convert from three R, G, B files. Awesome! Commit baby! f0f1d469734e863a39145e4e6ca1b05db9478bcf
What’s next? Let’s unhitch ourselves from the old predefined screen resolutions of the Amiga.
Instead of this: scrm = atoi(argv[3]);
We’re going to use argv[3] and a new argv[4] to pull direct pixel width and height from the command-line. I’m not going to explicitly document the code here, it’s super simple, just refer to the Git diff. When it’s done, you can try
tracer.exe pyrs pyrsopts 2048 1536
Or even
tracer.exe pyrs pyrsopts 4096 3072
Dang! Our little raytracer has gone 4k! Look at those gorgeous sharp pixels! Zoom in!
It’s still using the old Amiga aspect ratio code in the beginning of traceimage()
ar = (scrw * ((scrh * 4.0) / (scrw * 3.0))) / scrh;
And that formula is calculating with metrics for an old 4:3 Cathode Ray Tube SD monitor, which isn’t accurate anymore, so things are looking a bit stretched in 4k. We’ll consider revamping the scene and viewport definition in a later update.
I’m going to also commit a change to the pyrsopts file that finally changes the screen-portion scaling factor from 0.25 to 1.0. I’ve been testing all along at 1.0 (fullscreen) but never put it into the repository.
Now we’re at commit 1206172df2a56b2443be66ccb05dd96b985dbe5d.
The last thing I will do is add a unit test. I want to have a way to automatically validate after each change, whether I have diverged from the most recent test-reference (intentionally or unintentionally).
How do we use CTest to add a simple unit test?
We’re going to use CTest for this. CTest may not be the most favorite or sophisticated test environment, BUT it is included with CMake, and it is certainly up to the task before us. All we need to do is run the executable with known test conditions, record the output, and compare it against stored output from a known-good test run. Then, we’ll report on the success or failure.
We’ll add a few new lines to the CMakeLists.txt file:
# Define the command that generates the output file
add_test(NAME pyrs256k-generate
COMMAND ${CMAKE_BINARY_DIR}/tracer ${CMAKE_SOURCE_DIR}/pyrs ${CMAKE_SOURCE_DIR}/pyrsopts 640 400
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
# Compare testdata output and unverified recent output
add_test(NAME pyrs256k-compare
COMMAND ${CMAKE_COMMAND} -E compare_files
${CMAKE_SOURCE_DIR}/pyrs.png
${CMAKE_SOURCE_DIR}/testdata/pyrs.png
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
)
# Ensure that the comparison test only runs after the output is generated
add_custom_target(run_tests
COMMAND ${CMAKE_CTEST_COMMAND} --output-on-failure
DEPENDS pyrs256k-generate pyrs256k-compare
)
What this does is create two new actions:
pyrs256k-generate Runs tracer with known inputs and produces an output (pyrs.png) in the top-level folder. pyrs256k-generate generates the test data output by executing
tracer pyrs pyrsopts 640 400
Which will output a pyrs.png file if successful.
pyrs256k-compare Binary compares the pyrs.png with the stored reference testdata/pyrs.png (which is the output of the exact same command, verified to be good).
I’ve also changed the main (SDL_main, actually) function to always return output values compatible with CTest (0 for success, 1 for failure) so running the pyrs256k-generate test command doesn’t trigger spurious errors.
We make one step dependent on the other so both will run automatically. So now, if you build the project and then execute ctest from within the build folder, you should see something like
\AmigaWorld-Raytracer-Brian-Wagner\build>ctest Test project AmigaWorld-Raytracer-Brian-Wagner/build Start 1: pyrs256k-generate 1/2 Test #1: pyrs256k-generate ................ Passed 1.15 sec Start 2: pyrs256k-compare 2/2 Test #2: pyrs256k-compare ................. Passed 0.02 sec 100% tests passed, 0 tests failed out of 2 Total Test time (real) = 1.20 sec Hey! Success!
To prove that it’s really capable of detecting ANY diversion from the gold standard, let’s drop in a minor breaking change in math.c’s shadesky():
/* Calculate a sky color based on ray direction. */
int oneShot = 1;
VOID shadesky(r, c)
struct Ray *r;
struct Color *c;
{
SHORT zr, zg, zb;
SHORT hr, hg, hb;
FLOAT dp;
dp = r->dx * gnx + r->dy * gny + r->dz * gnz;
if (dp < 0.0) return;
zr = 0;
zg = 0;
zb = 100;
hr = 0;
hg = 40;
hb = 160;
c->r = oneShot + hr + (zr - hr) * dp;
c->g = hg + (zg - hg) * dp;
c->b = hb + (zb - hb) * dp;
oneShot = 0;
return;
}
I’ve added a oneShot variable that starts as 1, and the FIRST time shadesky() is called, it will perturb the output color’s red channel by increasing it imperceptibly by ONE step (out of 255). After that it will clear itself and never alter anything again. So we’ll get ONE pixel changed by ONE level. Let’s build and test again:
\AmigaWorld-Raytracer-Brian-Wagner\build>ctest Test project/AmigaWorld-Raytracer-Brian-Wagner/build Start 1: pyrs256k-generate 1/2 Test #1: pyrs256k-generate ................ Passed 1.01 sec Start 2: pyrs256k-compare 2/2 Test #2: pyrs256k-compare .................***Failed 0.03 sec 50% tests passed, 1 tests failed out of 2 Total Test time (real) = 1.05 sec The following tests FAILED: 2 - pyrs256k-compare (Failed) Errors while running CTest Output from these tests are in:/AmigaWorld-Raytracer-Brian-Wagner/build/Testing/Temporary/LastTest.log Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
BUSTED! The unit test caught even the tiniest digression between the stored file and the new human-visually-indistinguishable file.
I’m happy with where this has gotten to, so I’ll commit what we have and end this post. This is commit hash e10873f1328cb7468db951edacd0fbe3039805a4.
Next time we’ll start some real code cleanup and restructuring now that we know we’ll immediately detect any regressions, and then move on to some new functionality like maybe interpolated animation of the scene, different geometry types, and maybe even realtime ray tracing.
Huge thanks to Jeremy ‘cubicool’ Moles for helping me with the CTest testing!
AlphaPixel solves difficult problems every day for clients around the world. In the past we’ve done extraordinary things, like recovering deleted forensic video evidence from SD cards, having old failed hard drives refurbished to rescue critical data, reverse-engineering proprietary data formats to liberate valuable siloed information, even desoldering surface mount chips and sucking their brains out with hardware exploitation devices. We’ve worked on common and exotic architectures and operating systems and environments from the 8-bit to the 64-bit, Motorola, Intel, ARM, RISC-V, PA-RISC, DEC Alpha, Power, and microcontrollers.
The path to rescuing old code and data and making it useful again is a journey, often akin to building a rope suspension bridge one plank at a time, while suspended in mid-air. Sometimes while being attacked by natives, a la Indiana Jones. Often there are interlocking chicken and egg problems, where the obvious solution won’t work because of one minor issue, which then cascades and snowballs into bigger problems. It’s not ucommon to find a bug or omission in some old tool you need to use, and when you go to fix it, you find the tool can no longer be compiled by modern toolchains, necessitating reconstruction of a side environment just to fix the problem that’s blocking you on the main endeavor. We’ve jokingly used the term “Software Archaeologist” in some contexts like these.
If you have a difficult problem you’re facing and you’re not confident about solving it yourself, give us a call. We Solve Your Difficult Problems.
