AlphaPixel often gets called upon to work on legacy codebases, sometimes VERY legacy. We have contact with code from the 80s and 90s on a regular basis, in a variety of dialects and languages, and stored and archived in various difficult containers and mediums.

 

While NDAs and confidentiality mean we often can’t talk about our paid projects, we recently had an interesting side project that used the same processes, only it was all for fun, so we can talk all about it.

 

The task: Revive the Amiga source code for a basic educational raytracer written by our friend and early mentor, Brian Wagner.

 

In August/September 1991, during the era of powerful “big box” Amiga computers, when the Amiga was at the top of its game in the graphics and performance computation world, our friend Brian wrote a couple of articles for a short-lived programming magazine named AmigaWorld Tech Journal (henceforth AWTJ). At the time, I read the article but never compiled and ran the code. Recently, while mentoring some computer graphics Padawans myself, I had the bright idea to revisit that super-simple explanation of ray-tracing in its most basic form. But there were Difficult Problems to Solve.

 

In Part 1 of this blog series, we located the legacy source code online and assembled a set of tools to bring the code into a modern environment intact. In Part 2, we’ll update the code and get it running with exactly the same abilities it originally had, using a modern compiler and operating system. I’ll try to document every step and tool used.

We now rejoin our courageous heroes with a collection of C source files checked into GitHub. Start by checking out the existing repository to your local computer.

I’m now going to create a new branch on GitHub, named ‘modernization, where I will check in each incremental change.

If you are following along in the home game, make your own fork of the whole repository, and you can create your own modernization branch.

 

 

Now open the checked-out repository in Visual Studio Code.

 

It’ll look something like this:

Many of the files included in the repository (PDF, ADF, LZH, doc, executables, etc)for completeness aren’t actually used during the build process, but they aren’t harming anything. 

 

Since we didn’t launch from a Developer Studio Command Prompt, compiling with the cl.exe compiler won’t work due to paths. We can configure the tasks for this project to make things work properly. Referring to the instructions at https://code.visualstudio.com/docs/cpp/config-msvc#_run-vs-code-outside-the-developer-command-prompt, add the following just after the first open-curly-brace in the tasks.json to set up the necessary paths:

  "version": "2.0.0",
  "windows": {
    "options": {
      "shell": {
        "executable": "cmd.exe",
        "args": [
          "/C",
          // The path to VsDevCmd.bat depends on the version of Visual Studio you have installed.
          "\"C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/Tools/VsDevCmd.bat\"",
          "&&"
        ]
      }
    }
  },

Make sure to customize “C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/Common7/Tools/VsDevCmd.bat\” to your system’s install path for the compiler you’re using if necessary.

 

I’ll double-click tracer.c and then choose Run, and the C++ (Windows) compiler. Let’s see how it compiles!

* Executing task: C/C++: cl.exe build active file 
Starting build...
cmd /c chcp 65001>nul && cl.exe /Zi /EHsc /nologo /FeC:\AmigaWorld-Raytracer-Brian-Wagner\tracer.exe C:\AmigaWorld-Raytracer-Brian-Wagner\tracer.c
tracer.c
C:\AmigaWorld-Raytracer-Brian-Wagner\tracer.c(1): fatal error C1083: Cannot open include file: 'exec/types.h': No such file or directory

 

Yup. About how we expected. Not well. There are Amiga-proprietary header/include files that we don’t have. I’m going to just comment them out to start, which I know will bite me later, but in the interest of seeing what we’re going to be missing, we need to get past this uninformative show-stopper.

 

C:\AmigaWorld-Raytracer-Brian-Wagner\tracer.h(7): error C2061: syntax error: identifier 'SHORT'
C:\AmigaWorld-Raytracer-Brian-Wagner\tracer.h(8): error C2143: syntax error: missing '{' before '*'
C:\\AmigaWorld-Raytracer-Brian-Wagner\tracer.h(9): error C2061: syntax error: identifier 'nx'
C:\\AmigaWorld-Raytracer-Brian-Wagner\tracer.h(9): error C2059: syntax error: ';'

 

Ahh, there we go. Let’s dissect what’s going on and why.

This code is using a common Amiga convention of using artificial typedefs like SHORT instead of the compiler native types like short. These types are defined in one of the include files we commented out, “exec/types.h”. So we’ll need to address that.

 

Why did the Amiga do this?

 

Well, the Amiga developer tools came about in the mid 80s, a time when 8-bit, 16-bit and 32-bit architectures were all in play simultaneously. The C language had internal types like char, short, int and long, but didn’t exactly nail down how big all of the types were (Rationale for International Standard — Programming Languages — C section 6.2.5). On a 16-bit architecture, short and int might be both 16-bit and on a 32-bit architecture int might be 32-bit. Some compilers on the same architecture might interpret the type size rules differently too. So, the creators of the Amiga OS decided they’d lay the foundation for consistent types by defining an OS-level type collection that was agreed upon by everyone. So an INT was always the same:

typedef long            LONG;       /* signed 32-bit quantity */

 

It only made sense to canonize ALL of the basic types in the same way to eliminate any confusion, therefore we have SHORT types like

typedef short           SHORT;      /* signed 16-bit quantity (use WORD) */

typedef unsigned short  USHORT;     /* unsigned 16-bit quantity (use UWORD) */

So, we can either search and replace all the Amiga-specific typedefs and change them, or we could just bring in compatible typedefs from exec/types.h. In the interest of minimizing code changes, I’m going to do the latter, by creating my own local types.h with only the types we need and replacing the exec/types.h with my local types.h. Strangely, I can’t find the location of the definition of CHAR in the AmigaOS includes, but I’m recreating it here because it obviously was utilized.

// types imitating the Amiga's exec/types.h

#ifndef VOID
#define VOID            void
#endif

typedef unsigned char   UBYTE;      /* unsigned 8-bit quantity */
typedef char            CHAR;      /* signed 8-bit quantity */
typedef short           SHORT;      /* signed 16-bit quantity (use WORD) */
typedef unsigned short  USHORT;     /* unsigned 16-bit quantity (use UWORD) */
typedef long            LONG;       /* signed 32-bit quantity */
typedef unsigned long   ULONG;      /* unsigned 32-bit quantity */
typedef float           FLOAT;
typedef double          DOUBLE;
typedef unsigned char   UBYTE;      /* unsigned 8-bit quantity */

#ifndef TRUE
#define TRUE            1
#endif

#ifndef FALSE
#define FALSE           0
#endif

#ifndef NULL
#define NULL            0L
#endif

And change the includes project-wide to

#include "types.h"

 

While we’re at it, we’re going to update all the source files to refer to our types.h and remove references to missing headers. We’ll also remove unnecessary files from the modernization branch such as the ADF, PDF and LZH files.

 

Right now we’re at git revision hash 88886da4301318ddef11be85a3f8a75eb7fb9662.

 

This toy project used the Amiga native file IO and memory APIs, because at one time these were more performant than the included portable standard library functions. Of course those functions aren’t available. Let’s write some siple stubs to get the code working with minimal changes across the codebase. Maybe later we’ll refactor to switch projectwide once we can do some unit testing to make sure we’re not breaking things accidentally.

 

I’ll create a simple platformstub.c and platformstub.h combination, put stub implementations there and include it anywhere it’s needed. Here’s what I needed to implement to start:

 

void *AllocMem(ULONG byteSize, ULONG attributes);
void FreeMem(void *memoryBlock, ULONG byteSize);
FILE *Open(const char *filename, LONG accessMode );
bool Close( FILE *file );
LONG Write (FILE *file , void *buffer, LONG length);

Those, plus include fixups and a CMakeLists.txt comprise GitHub commit hash 8cc7295b1d70120d75ff8a7ed0574ac6aeb21327.

 

Now we’re down to only 86 errors/warnings across two files, tracer.c and load.c. A lot of them tend to be security warning from using the old-skool C stdio functions that can overrun easily. For the moment, we’re going to just suppress those warnings so we can see the real problems by adding the _CRT_SECURE_NO_WARNINGS definition. We’ll do this by slapping

 

add_definitions(-D_CRT_SECURE_NO_WARNINGS)

 

Into the CMakeLists.txt.

 

This cuts down the errors and warnings a lot. There are some of these:

[build] C:\AmigaWorld-Raytracer-Brian-Wagner\image.c(83): warning C4013: 'unitvector' undefined; assuming extern returning int
[build] C:\AmigaWorld-Raytracer-Brian-Wagner\image.c(94): warning C4013: 'polygonhit' undefined; assuming extern returning int
[build] C:\AmigaWorld-Raytracer-Brian-Wagner\image.c(99): warning C4013: 'groundhit' undefined; assuming extern returning int
[build] C:\AmigaWorld-Raytracer-Brian-Wagner\image.c(100): warning C4013: 'shadowchk' undefined; assuming extern returning int
[build] C:\AmigaWorld-Raytracer-Brian-Wagner\image.c(117): warning C4013: 'shadepoint' undefined; assuming extern returning int
[build] C:\AmigaWorld-Raytracer-Brian-Wagner\image.c(123): warning C4013: 'shadesky' undefined; assuming extern returning int

Seems like there never were files of function prototypes, or at least I don’t have them. I’m not sure how the compiler and linker would have dealt with this as, for example, unitvector returns float. I’ll just whip up a math.h, free.h, write.h, etc. and include them in the various .c files that invoke the corresponding functions.

 

With the completion of that we’re at commit f5ab2d0c2d72cab3bd64b0bb60bdb3e762bc1530.

 

Now, we mostly have missing UI API stuff, warning about loss of precision going to float, some misuse of pointers to the wrong type, and a weird compiler warning about a SYSTEM HEADER FILE. Let’s dive in and try to eliminate the rest of the actual errors. First, I’m going to put #ifdef WINDOWED_UI around any UI code, without removing it. I’m going to replace it at some point, so I want it around as a reference, but I want it out of the way while debugging actual non-UI functionality. Later I can search WINDOWED_UI to find all the bits I need to replace.

 

Ok, that cut down the errors a lot. Now let’s look at this one:
[build] C:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt\corecrt_math.h(984): error C2059: syntax error: 'constant'

 

Um, wut?

 

It appears that some rocket scientist made a function named y1 (a Bessel function) and put it in the C math headers so it’s included by default.

 

        _CRT_NONSTDC_DEPRECATE(_y1) _Check_return_ _ACRTIMP double __cdecl y1(_In_ double _X);

 

 If you’re a person doing any coordinate space math and you ever name a variable y1, it will cryptically clash. Fortunately, StackOverflow has the fix, just predefine and it goes away. So we’ll add that define to the CmakeLists.txt.

 

add_definitions(-D_CRT_SECURE_NO_WARNINGS -D_CRT_DECLARE_NONSTDC_NAMES=0)

 

That solves that. Now let’s address the fscanf precision issues by changing %d to %hd because we’re loading into short instead of long.

<         fscanf(fp, "%d ", &polys[npoly].vtx[i]);

>         fscanf(fp, "%hd ", &polys[npoly].vtx[i]);

 

At this point (commit 626133930520072f6de01008f85e43560ea16b42) we are able to compile and link without errors. The remaining compiler warnings are all about loss of precision from double to float, or from a larger to a smaller type. I’m going to leave those as-is, because I believe they are representative of the original program intent, and possibly refactor them in the future when switching everything to larger types.

 

Ok, let’s see what we get by running it (I merged the .red, .grn and .blu files into a PNG):

Hey! Looks great, but why is it only quarter-screen? Ah, I need to RTFM. The article text describes the lines of the options file, pyrsopts:

 

The scl field describes which fraction of the screen to use. We’ll use this value to scale the width and height of the screen mode specified in the command line, letting us create images that cover, for example, only 1/2, 1/4, or 1/8 of the screen. This feature allows the user to generate quick test images.

 

The scl line defaults to 0.25. Edit the third line of the pyrsopts file to read 1.0 and re-run:

 

Wow! Raytracing from 1991, resurrected!

 

I’m going to stop right there, with the code as represented by GitHub commit 08545faee0a5bf1e15b8b6c668d4b2f1cb20ce17 compiling and running without error, with the most minimal changes possible.

 

In our next episode, we’ll refactor and clean some things up to make it more flexible, and then maybe add a UI.

 

Footnote: I booted up an emulator pretending to be an 8MHz 68000 Amiga 500 and ran the tracerFFP in screen mode 1, with scale 0.25, which I believe is a total traced pixel size of 80×50 pixels (320×200 from screen mode 1, quarter-sized, so 80×50). It took approximately 6 minutes 31 seconds (391 seconds) to complete. This averages out to 10.23 pixels traced per second. Testing the same scene on an emulated Amiga 4000/040 (with 68882-compatible FPU) takes only a second, so the benchmark accuracy is poor. We can change to screenmode 4 (640×400) and scale 1.0 (render all pixels) to find it can render 256,000 pixels in 57 seconds, a rate of 4491 pixels per second. Later we’ll do some similar benchmarking on modern CPUs.

 

AlphaPixel solves difficult problems every day for clients around the world. In the past we’ve done extraordinary things, like recovering deleted forensic video evidence from SD cards, having old failed hard drives refurbished to rescue critical data, reverse-engineering proprietary data formats to liberate valuable siloed information, even desoldering surface mount chips and sucking their brains out with hardware exploitation devices. We’ve worked on common and exotic architectures and operating systems and environments from the 8-bit to the 64-bit, Motorola, Intel, ARM, RISC-V, PA-RISC, DEC Alpha, Power, and microcontrollers.

 

The path to rescuing old code and data and making it useful again is a journey, often akin to building a rope suspension bridge one plank at a time, while suspended in mid-air. Sometimes while being attacked by natives, a la Indiana Jones. Often there are interlocking chicken and egg problems, where the obvious solution won’t work because of one minor issue, which then cascades and snowballs into bigger problems. It’s not ucommon to find a bug or omission in some old tool you need to use, and when you go to fix it, you find the tool can no longer be compiled by modern toolchains, necessitating reconstruction of a side environment just to fix the problem that’s blocking you on the main endeavor. We’ve jokingly used the term “Software Archaeologist” in some contexts like these.

 

If you have a difficult problem you’re facing and you’re not confident about solving it yourself, give us a call. We Solve Your Difficult Problems.