HypoThermia's
Q3A: Tutorial - Compiling without MS Visual C++
Welcome!
| This tutorial is aimed at the Quake3 Arena mod developer who
can't or won't work with Microsoft Visual C++. In addition it helps
lift the restriction that all coders working on a project use the same
tools in the same environment.
Summarized here is the experience I had in building the Q3Source
for the Borland compilers. Hopefully you can use this to create
the tools required to develop Q3Source using the compiler you're
familiar with. Perhaps someone has done the
hard work for you already!
Feedback is welcome
and encouraged, particularly if you have ported Q3Source to another
compiler. Examples are provided based on the work I did. There's no substitute
for real world experience.
| It's my hope this site will act as a focus for developing compiler
solutions for building Q3Source. Mod makers can then choose the
platform and compiler they want to work on. |
Contact me if you do
make a successful port to another compiler, at the very least I'll put
a link on my page to your hard work.
If you have any questions about porting to another compiler then I'll
try to help. No binaries or attachments over 50K without my permission
please!
Sit back, kick off your shoes, and try not to think of cheese!
Cheers,
HypoThermia
|
Contents
What's new
Background
| The Quake3 Source released by id Software builds part
of the game code, allowing dedicated enthusiasts to add to and improve
Quake
3 for the gaming community at large. As the game already ships on several
platforms it's clear that the Q3Source needed to be as platform
independent as possible. It was written using portable ANSI C and compiled
to a bytecode that runs on any machine with a Q3 executable.
Mod makers can finally develop game enhancements: on a single platform,
for multiple platforms.
One beneficial side effect of using portable ANSI C is that a large
number of programmers are already familiar with C as a programming language.
The other main benefit is the ability to build binaries that only work
on the one platform for development and debugging.
Contents
Top of page |
Setting up and distributing your project
In setting up your work for distribution I would encourage the minimum
amount of modification to the Q3Source. You shouldn't need to go
in with a butcher's knife and make huge amounts of changes all over the
place.
Use a separate directory for your work
To help organize the changes you make I strongly recommend that you make
a directory under quake3\source to hold the files you create to
build the Q3Source successfully.
This allows more than one compiler to work from the same source distribution,
and the source\cgame, source\game and source\ui
directories don't get cluttered up.
The one exception to this is the project file you create to build binaries
for your platform. This should go in quake3\source and have a
unique and relevant name.
Document your changes
It is better for you to document your changes for someone else to apply
than redistribute modified source files. They can be integrated with an
existing project much more cleanly.
Example:
Open the header file game\q_shared.h and move to line 424, it should
be:
float Q_crandom(int* seed);
Insert after it the following:
#ifdef __BORLANDC__
#ifdef random
#undef random
#endif
#endif
You might also consider distributing DIFF files.
Make your work available on the Internet
Others can then benefit from your hard work. Notify me so I can update
this site. Notify portals so others in the community can get the benefit
too.
Have a look at my distributed work on porting to the Borland compilers
for an example.
Contents
Top of page |
The objectives
The objective can be split into three parts:
-
Building the bytecode for use with Quake3 (and redistribution)
-
Building binaries that can be tested and debugged on your system
-
Releasing your work for others to use
When you release your work for others to use it should make the minimal
number of changes to the Q3Source installation. Preferably you should
provide your own batch files or scripts, making sure they don't overwrite
the ones supplied with Q3Source. Id Software might release
an updated source, overwriting your files. Ideally a mod developer should
only need to re-apply your necessary changes to the Q3Source
codebase.
Read 'Distribute your project' for ideas on
how to do this, so you can start as you mean to go on.
In the first two cases you'll be using the header files supplied with
your compiler. The main issue for the bytecode is making your headers look
like ANSI C. See 'Getting started on the bytecode'
for more information.
Building binaries for your system needs an understanding of how to modify
portable code in a way that keeps it portable. In other words, another
compiler should be able to use the Q3Source code you've modifed
without running into problems of its own. See 'Compiling
the binaries' for details.
Contents
Top of page |
Getting started on the bytecode
| The bytecode that runs on the Quake3 Virtual Machine (QVM) is platform
independent. It's compiled using lcc.exe, a tool supplied by id
Software, and will use the header files from your compiler. Each of
the compiled files is then assembled and linked using q3asm.exe.
There are three separate QVM files you'll be compiling:
| qagame.qvm |
Contains the code needed to run a game server. In single player
this also controls the bot AI. |
| cgame.qvm |
Handles the events and screen display on your local (client) machine. |
| ui.qvm |
Provides the User Interface and menu front end to the single player
game. |
To get the bytecode to compile you'll need to work out how to make your
header files appear as platform independent ANSI C.
Automating compilation using scripts
Provided as part of Q3Source are 4 batch files that run from the DOS prompt.
Three of the files are concerned with building each of bytecode modules
qagame,
cgame,
and ui, called game.bat, cgame.bat and ui.bat.
Each calls the fourth batch file compile.bat with the location
of a source file needed to build that QVM module.
Once compiled, the last job of each script is to assemble and link the
files to make the distributable. q3asm.exe uses a response file
for each module: game.q3asm, cgame.q3asm and ui.q3asm.
Copy these files to your compiler directory under Quake3\source.
Noticing that they actually do their work in a subdirectory called vm,
modify them in the following way:
-
Adapt the batch file to work using the script language on your system,
making sure that you still use compile to call lcc.exe.
-
change the relative paths to each of the source files
-
change the relative paths to the executables lcc.exe and q3asm.exe,
or
add them to your executable path (document this!)
-
change the relative paths to ..\cgame, ..\game, and ..\ui
in compile.
-
in each of the .q3asm files: modify the path to xx_syscalls
only, so it uses the right .asm file in the Q3Source subdirectories
cgame,
game,
and ui.
If you run each of there files they should now *try* to compile the source,
bombing out with an error about not finding some header files.
Understanding your header files
From now on the changes you need to make are in the compile script
only.
The first modification to compile is to tell it where your header files
are. Make sure the following argument is passed to lcc.exe:
-I<path to header files>
where <path to header files> is an absolute path correct
for your system.
Now that your header files can be found you'll start running into platform
and compiler specific issues. Most of these should be solved by passing
the equivalent of a #define xxxxxx to lcc.exe. You can
do this by adding the argument -Dxxxxxx or -Dxxxxxx=""
in the compile script.
If your header files can be used on more than one compilation model
then you need to work out a path through them that gives a "pure" ANSI
C definition of all functions. One way to do this is find the header file
that defines the compiler specific information, bypass it, and supply your
own definitions to lcc.exe.
You may also have to define some compiler specific values to help control
the route through the header files. Check that these aren't used in the
Q3Source already, and if they are that they won't cause problems.
| Borland C++ uses header files that can build
for executables or DLLs in Win32, Win16, and executables for DOS in 6 different
memory models.
I choose to force the Win32 executable path onto the Borland
header files by defining __FLAT__. I had to avoid the Win32 references
in Q3Source (we're not compiling for that platform!) so I didn't
define WIN32, _WIN32 or __WIN32__. These variables
are defined and used by either Borland or Microsoft tools.
I also defined the compiler specific value __BORLANDC__, further
controlling the route through the headers.
With these defined I started getting errors from lcc.exe about
_RTLENTRY
and similar constants, so I had to remove the header file that supplied
these definitions <_defs.h> (defining ___DEFS_H did
this as the header was protected from repeated inclusion by this value).
A typical Borland definition looks like:
int _RTLENTRYF atoi(const char _FAR *__s);
and applying these definitions reduced it to:
int atoi(const char *__s); |
Contents
Top of page |
Keeping the code portable (and how to make
necessary changes)
| Avoid making modifications to the Q3Source, unless you can absolutely
have to. Try and make changes through the command line options in the compile
script instead.
When you have make modifications to Q3Source, do so after you've
determined that the use of -Dxxxxxx can't solve your problem.
Make the changes so that they are controlled by a constant defined only
by your compiler, make sure this constant is defined in the compile
script as well.
Make your necessary changes to files that already have a compiler specific
component in them (game\q_shared.h for example). You should only
need to touch a few header files.
If you need to edit a C source file, think again! Look at the
section on 'Expected errors', there should
be no need to fix these.
If a route through the Q3Source header files is already available
for your platform, but you're using a different compiler, then take advantage
of it.
Document your changes and allow the recipient of your work to incorporate
them, understanding the benefit themselves. Remember: you're helping people
who are already exerienced C coders.
| Using the Borland header files there was a clash with the definition
of random(). As the Q3Source definition needed to take
precedence, the following code was inserted:
#ifdef __BORLANDC__
#ifdef random
#undef random
#endif
#endif
and was added after game\q_shared.h line 424.
Notice that this also works when building binaries, as the same problem
arises. |
| The only other change I had to make to a source file was in a Borland
header file. lcc.exe was generating an error while parsing a #error
directive (even though it wasn't executed). The change made to the Borland
header put the error message into quotes.
#error "Can't include both STDARG.H and VARARGS.H"
in <stdarg.h> line 20. |
Contents
Top of page |
Expected errors
Despite the portablilty of the Q3Source, warnings are generated
by the source. You might also get a few warnings from your header files.
Work out why and decide whether any change is needed in your header files.
ui/ui_ingame.c:103
ui/ui_ingame.c:107
ui/ui_atoms.c:739
ui/ui_atoms.c:742 |
"Warning: Conversion of 'pointer to void'... ...is compiler
dependent"
Not a problem for the QVM, though it might be for your compiler. |
You might also find that your bytecode modules are of a different size
to those released with Quake3. This is probably caused by differences
between how the header files use static data for their implementation.
| The Borland header files generate another warning:
limits.h:31 Character constant taken as not signed
Paradoxically this is warning about the method employed by Borland
to find out if a char type is signed or unsigned. It can be ignored. |
Contents
Top of page |
Testing the bytecode
| Follow the instructions in makeamod_readme.txt for modifying
the source code to produce the "Slow rocket mod". Don't forget to change
things back!
Try playing back the demos using "timedemo 1". This is not
a performance test, but ensures that the same frames are drawn each time.
Play a few games against bots by compiling the bytecode into a directory
other than baseq3. With or without the slow rocket mod!
Go on! You've earned it!
Contents
Top of page |
Compiling the binaries
| Being able to compile the bytecode for the QVM is the minimum amount
of work that needs to be done with your header files. To make further development
work easier using your compiler you need to develop the means to compile
binaries from the Q3Source.
This is dictated by your compiler and platform. For command line tools
you should provide a makefile that defines the relationship between
the Q3Source and the compiled binaries. If your platform has a graphical
development environment then a project file would be desirable.
Project files
The project file is the only thing that should be placed outside your compiler
directory (discussed in 'Distribute your project').
As the Microsoft Visual C++ project went in the \Quake3\source
directory, so should yours. If it is a makefile then try and give
it a meaningful name like makefile.gcc.linux.
The work you've put in to understand the header files should now be
of benefit to you. If you needed to do any modifications to the Q3Source
then you shouldn't get error messages from these. Again, try to use existing
defined constants (related to your platform) within the Q3Source
to get the code compiling.
There is no substitute for familiarity with your compiler tools.
Take note that there are several files in source\game that
are also used to build the client (cgame) and user interface (ui).
It would also be beneficial to have any intermediate files created by
your compiler placed in your custom compiler directory.
Using C_ONLY in the project
It is almost certainly necessary to include the following definition within
the project for each binary:
#define C_ONLY
Id wrote some Intel optimized assembler routines for some of
the performance critical math. If you can't compile that then define this
constant. Try and do it in a way that doesn't interfere with the Q3Source.
Defining constants
If the Q3Source can already be compiled for your platform using
another compiler, then you might find it beneficial to define some or all
of the constants used by that compiler. Any modifications you need to make
might be built upon this.
Handling system calls to Quake3
Each of the three binary files you need to build are equivalent to the
three bytecode files for the QVM. For the Wintel platform they are called
qagamex86.dll,
cgamex86.dll,
and uix86.dll.
Each requires that a file with a name like xx_syscalls.c is
included. It allows the binary to access optimized code within the Quake3
executable. You must ensure that QDECL is defined so that the
binary and executable can interface. The definition of QDECL is
in game\q_shared.h.
Get it wrong and the game will crash.
The function dllEntry() accepts the callback function in the
Quake3
executable. Despite the similar name to DllEntryPoint() used in
Win32 DLLs, it shouldn't be confused.
Making sure Quake3 can use your binaries
Only two functions need to be visible to the Quake3 executable:
vmMain()
and dllEntry(). Make sure these are exported from your binaries
in a way that Quake3 can see.
Development only
Building the server, client, and user interface code using binaries is
for debugging and testing purposes only. It is not for redistribution.
| Borland tools use an Integrated Development
Environment (IDE). Each of the three modules needs to be compiled to a
DLL for Win32.
As most of the work is already done for Microsoft Visual C++
on the same platform it is beneficial to define _MSC_VER=0x800.
The explicit value is used within the Borland header files themselves.
Microsoft also uses WIN32 to indicate compilation for
Win32. The Q3Source header files respond to this, so it needs to
be defined.
The Borland IDE allows these values to be defined as part of the project
without modifying the Q3Source code. This is equivalent to the -Dxxxxxx
used with lcc.exe earlier.
C_ONLY is required as I don't have Turbo Assembler.
Borland compilers prepend an underscore to the name of exported
functions. This conflicts with the Microsoft way of things, so each
of the exported definitions in the *.def files needs to be modified
to:
[EXPORTS]
vmMain=_vmMain
dllEntry=_dllEntry
Again, these files (one for each binary) are in the compiler specific
directory I'd created. |
Contents
Top of page |
Making modifications
| You will find that the Q3Source will generate quite a few warnings
when compiled. You should look at them to determine whether they are inconsequential.
For the most part they should be.
As you've already got Q3Source compiling to the bytecode you
shouldn't find that there are major issues.
If you should need to make modifications then consult the guidelines
'Keeping the code portable'.
If you need to introduce platform specific code that fixes problems
or issues that you discover, then try and keep the source code in a separate
file in your custom compiler directory.
Contents
Top of page |
Some possible errors
Some compilers have a slightly stricter interpretation of the C language
than others. If possible you should determine whether your compiler behaves
in the expected fashion.
Some warnings in Q3Source
Id have used several coding styles that can trigger warning messages.
Some can be ignored, others should at least be checked.
Errors like 'variable assigned a value but not used in function
xxxx' can be safely ignored. Whereas an error like 'Conversion
may lose significant digits' should be checked.
One error that you might see is 'Possibly incorrect assignment'. This
is caused by the following type of code:
for( j = 0; control = controls[j]; j++ )
At some point in the array controls[] an element is zero, this is guaranteed
by the usage of the array elsewhere. It also acts as a terminating condition,
the for-loop will terminate at the first zero value. It is equivalent to:
for( j = 0; (control = controls[j])==0; j++ )
and can be ignored.
Check your math library implementation
Depending on how your math library is implemented, you may find you are
getting sqrt:DOMAIN errors when using your build of cgame.
This arises from what appears to be a bug in the Quake3 executable
(build 1.15c) that results in the Q3Source trying to find a vector
perpendicular to {0,0,0}.
Fixing this is compiler dependent. You need to find a way to intercept
and ignore this error: check the help files for your compiler and its math
library. Your solution should be placed among your compiler dependent files,
The bug manifests itself when viewing demo001.dm3 with "timedemo
1".
| This error is generated by the Borland math library implementation.
The solution is to include this file in the build of the client game code
(cgame):
// borland_hack.c
#include <math.h>
int _RTLENTRY _EXPFUNC _matherr(struct exception* e)
{
return 1;
} |
Contents
Top of page |
Testing the binaries
| Move the binaries into the same directory as your Quake3 executable,
and fire up Quake3.
Try the same things used to test the bytecode.
Particularly important is the testing of demo001.dm3 using
"timedemo 1". If you get a sqrt:DOMAIN error then look
here for the solution.
Contents
Top of page |
|