- What You Need For This Tutorial
This article is about how to extend the Dungeon Siege engine with new
functions that you create. Even though there are thousands of functions
available to be called from skrit, sometimes you just have to write
your own function to get what you want done. For example, say you are
creating a mod that needs to talk to a database to register player
stats, or you want to do something simple like query the system time in
order to make your mod "real time". To do either of these things you
will need to create a set of functions in an engine extension that can
be called from within your mod's skrit code.
In a nutshell, to extend Dungeon Siege with new engine functions for skrit to use:
- Create a dynamically linked library (DLL) and set its output extension to be .dsdll rather than .dll.
- Export the functions using the compiler's "declspec( dllexport )" tag that you want the game to see.
- Put the resulting .dsdll file in the same directory as DungeonSiege.exe.
It's pretty simple! Most of this doc is about the rules
governing what you can and cannot export, and some of the quirks in the
system.
What You Need For This Tutorial
- Dungeon Siege v1.1 or above
The .dsdll features were
added after Dungeon Siege originally shipped, so make sure to get the
latest version through your ZoneMatch AutoUpdate feature.
- A Microsoft C++ compiler
This is not absolutely
required, as Delphi and other languages are perfectly capable of
creating .dsdll's, however it is easiest with a Microsoft compiler so
that the name mangling format is guaranteed to be compatible.
Visual C++ 6 and Visual C++.NET are guaranteed to work for creating
.dsdll's, and it's been reported that Visual C++ 5 works fine as well
though I have not verified this myself.
- The WebMod sample
This is a little mod I put together that contains a sample .dsdll, mod,
and map. It demonstrates how to download a web page, process it a bit,
and put the results up onscreen. Not terribly exciting, but the most
important part of the sample is the dsdll.h file, which is a set of
macros, docs, and some helper templates. Be sure to check this file out!
The mod can be downloaded by clicking here.
- The DbgHelp.dll redistributable
This is a DLL that is required by the game when using .dsdll's to
decode the mangled names that the C++ compiler exports. It's only
required in Win98 and WinME, as Win2K and WinXP both have the file
already.
The latest version of DbgHelp.dll can be found in Microsoft's
"Debugging Tools for Windows" (freely available on their web page) or
in the System32 directory of Win2k/XP, plus I've also included it in
the WebMod sample for convenience. To make sure that your .dsdll-based
mod runs on all versions of Windows, DbgHelp.dll should be installed to
the same directory as DungeonSiege.exe.
- FuBi documentation on the web
This is not required reading, but if you want to know how the system
works underneath, I have written some detailed documentation on FuBi,
our function binder that drives much of the low level systems of
Dungeon Siege. You can find information including much of the source
code to the system here:
http://www.drizzle.com/~scottb/gdc/ (see GDC 2001 material)
As this was written nearly two years ago, some of it may
be out of date (FuBi has progressed considerably since then) but it's
probably about 90% accurate. For the original paper on FuBi, you can
see my "gem" from the first Game Programming Gems book:
http://www.drizzle.com/~scottb/publish/gpgems1_fubi.htm
This paper is even older, but it provides a good
walkthrough of the problem that FuBi was intended to solve: the lack of
reflection capability in C++.
Here is how to build a basic .dsdll with a single function:
- Start a new DLL project in Visual C++.
- Modify the linker settings to change the name of the output file to have an extension of .dsdll.
- Create a .cpp file and add it to the project - it must be compiled as C++, not C, for this to work. Here is a basic sample:
// dlltest.cpp
#include "windows.h"
#include "dsdll.h"
BOOL APIENTRY DllMain( HANDLE, DWORD, LPVOID )
{
return TRUE;
}
class DllTest
{
FEX static const char* GetTestString( void )
{
return ( "Hi there!!!\n" );
}
};
The FEX keyword is actually a macro that comes from the dsdll.h
file. More detailed information on the macros available in this file is
included in a later section of this article.
- Compile and link the .dsdll and then copy it to the same
directory as DungeonSiege.exe. You may find it convenient to just
modify the project settings to output to that directory directly so you
don't need to copy the file manually (or you can set a post-link build
step to do so). Another option is to run Dungeon Siege with a shortcut
that sets the dsdll_path command-line option (or in the INI file) to
point to the location of your .dsdll. For example:
dsdll_path=c:\mystuff\dsdll\debug
You can verify that your functions have been exported by running
the "Depends.exe" utility that is distributed with the Microsoft
Platform SDK and looking at the export table.
- The next time you launch the game your new functions will be
available. Here's an example of how to use the function created in #3
above:
Report.ScreenF( "Your dsdll says: %s", DllTest.TestString );
This ought to print the text "Your dsdll says: Hi there!!!" to
the top of the screen. You can insert this code in any skrit in the
game and it will work. You can do other things like assign it to a
string:
string s$ = DllTest.TestString;
In short, it's just like any other function in the game, the engine doesn't know the difference!
The .dsdll is simply a plain DLL renamed to .dsdll and loaded
with exported functions. As such, it can do whatever it likes in the
system, calling out to the Internet, playing sounds, reading the
keyboard, using files on the hard drive, etc. - there are no
limitations placed on what DLL's can do in Windows.
Distributing your .dsdll-based mod is about the same as
distributing any other mod, with two exceptions. First, you must
install a special file DbgHelp.dll into the same directory as
DungeonSiege.exe ; second, you must create a shortcut or batch file
that launches your mod. See the FAQ entry below called "I can't play
multiplayer games with my friends any more! What's wrong?" for more
information on this.
During
system initialization, the game looks for all .dsdll files found in the
dsdll_path directory (which defaults to the game's EXE dir), then sorts
them alphabetically and iterates through them for import. Importing is
done by iterating through the export table on each .dsdll and
de-mangling each entry found there (this is what the DbgHelp.dll is
needed for), then mapping the function and its entry point into the
system.
There are a lot of rules and oddities about how the system
works, so let's just go through those as a set of randomly ordered
notes. Note that most of the following is pulled straight from the
documentation at the top of dsdll.h.
These are specifically supported in exports:
There are also some important restrictions:
- __fastcall is not supported as a calling convention.
- Don't prefix any names with FUBI_ or allow any names
to contain '$' in the middle of the name - these are reserved as
special tags that FuBi uses for documentation and internal functions
attached to exports. If you look closely at the macros in dsdll.h
you'll see how this works.
- Skrit is case-insensitive and FuBi has to accommodate
this. Do not export two functions with the same name but different case
(for example, Test() and TEST()).
- For consistency reasons, do not use underscores in
any exported names - some_function_call() is bad, SomeFunctionCall() is
good.
- If a SetXXX and GetXXX pair exists but the parameters do not match as described above, it will print an error.
- "Complicated" exports won't even parse properly and
you'll get an error from FuBi about how it isn't able to understand
some function you're trying to export. This will typically happen with
templates or functions that take function pointers as parameters. Not
supported.
- Unions are not supported. Never will be.
- Passing complex types by value is not supported. If
you export a function that takes a GUID by value, you'll get an error.
Pass it by reference and it'll be okay. Specific support can be added
to allow passing these by value if you tag the type as POD (and it
really has to be plain old data! no ctor/dtors called!).
- Virtual functions are supported but are very unsafe.
A virtual function exported to FuBi will get called directly. The vtbl
is not consulted ('cause I don't know how to query its contents without
a PDB) to figure out which version of the function to call. To prevent
Bad Things happening from this I issue a warning for exported virtual
functions. Work around this by exporting a non-virtual function that
just redirects the call to a virtual function. Then the compiler will
do the proper vtbl work. Use FUBI_RENAME (documented below) to give it
the same name as the virtual function from Skrit's point of view.
- Sometimes the DbgHelp.dll symbol undecoration code
just bugs out. This happens more often on Win9x for some reason. Tweak
the prototype until it stops.
- Exports of constructors, destructors and overloaded operators are not supported.
- Exporting functions from nested classes and namespaces
is not supported (though documentation and singleton exporting of
nested types is supported), mainly because Skrit doesn't need this
complication.
- Data exports (exporting a variable directly rather
than using FUBI_VARIABLE and function-based Get/Set exports) will
always get scary warnings. Don't export data.
The
dsdll.h file (included in the WebMod sample) contains a set of macros
and templates that will help make it easier to export functions and
types via .dsdll into the game engine. This section documents each of
them.
FEX
Put this in front of any function in order to export it. It just resolves to a dllexport tag.
FUBI_DOC( NAME, PARAMS, DOCS )
FUBI_MEMBER_DOC( NAME, PARAMS, DOCS )
These macros will insert documentation about a specific
function into the type system, which can be retrieved using the various
commands in the Help namespace. Be sure to use the MEMBER version when
documenting a member function of a class, and the non-MEMBER version
when documenting globals. Here is an example from the game's C++
SkritSupport.h file:
FEX vector_3& MakeVector( float x, float y, float z );
FUBI_DOC( MakeVector, "x,y,z", "Returns a vector composed of x, y, and z.
This is a temporary and will change the next time this function is
called so do not store the results." );
Put the macro next to the function that it is documenting - it is
a standalone statement that exports a function which is meant to
document another function. NAME is the unquoted name of the function,
PARAMS is a quoted string containing a comma-delimited list of
parameter names, and DOCS is a quoted string containing documentation
on the function.
FUBI_CLASS_DOC( NAME )
This macro is similar to the FUBI_DOC except that it is
intended to doc a class or type. Just put it inside of a class
definition, where DOCS is a quoted string containing documentation on
the type.
FUBI_RENAME( NAME )
This can be used to rename a function from how C++ sees it to
how skrit sees it. This is most useful for wrapping virtual functions.
Because skrit does not have access to the vtable, the compiler needs to
provide the necessary lookup code for virtual functions. To use this
macro, wrap it around the function name that is being declared. Here is
an example:
virtual void SetVisible( bool bVisible );
FEX void FUBI_RENAME( SetVisible )( bool bVisible )
{ SetVisible( bVisible ); }
This sample is pulled from the code for the UIWindow class. The
first function SetVisible() is virtual and for a call from skrit to
call the proper dynamically bound method, the compiler must generate a
function to select it. The second function that is wrapped with
FUBI_RENAME is that function, and will be the one that is called from
skrit.
FUBI_VARIABLE ( T, PREFIX, NAME, DOCS )
FUBI_VARIABLE_BYREF( T, PREFIX, NAME, DOCS )
FUBI_VARIABLE_READONLY( T, PREFIX, NAME, DOCS )
FUBI_VARIABLE_BYREF_READONLY( T, PREFIX, NAME, DOCS )
FUBI_VARIABLE_WRITEONLY( T, PREFIX, NAME, DOCS )
FUBI_VARIABLE_BYREF_WRITEONLY( T, PREFIX, NAME, DOCS )
These macros are all variations on a simple concept of
simultaneously declaring and exporting functions to set/get members of
a class. The READONLY postfix means that the variable cannot be
modified from skrit (i.e. no "Set" function), the WRITEONLY postfix
means that the variable cannot be read from skrit (i.e. no "Get"
function), and the BYREF tag means that set/get is to be done by
reference, not by value (needed for non-builtins). Here is an example
from the game's code:
struct JobResult
{
FUBI_VARIABLE_BYREF( double, m_, TimeFinished, "Time when job was finished.");
FUBI_VARIABLE( eJobResult, m_, Result, "Result at job finish.");
FUBI_VARIABLE( DWORD, m_, Count, "Times this job was attempted");
FUBI_VARIABLE( eJobAbstractType, m_, Jat, "Job Abstract Type");
FUBI_VARIABLE( eJobTraits, m_, Traits, "Job Traits");
FUBI_VARIABLE( eActionOrigin, m_, Origin, "Job assignment origin");
FUBI_VARIABLE( Goid, m_, GoalObject, "Job param - goal object");
FUBI_VARIABLE( Goid, m_, GoalModifier, "Job param - goal modifier");
FUBI_VARIABLE_BYREF( SiegePos, m_, GoalPosition, "Job param - goal position");
};
The parameters to feed the macros are: T is the type of the
variable, PREFIX is the C++ prefix to put on the variable (our
convention is m_VarName so m_ is our prefix), NAME is the name of the
variable, and DOCS is a quoted string describing the variable.
FUBI_CLASS_INHERIT( T, BASE )
FUBI_CLASS_INHERIT2( T, BASE1, BASE2 )
FUBI_CLASS_INHERIT3( T, BASE1, BASE2, BASE3 )
FUBI_CLASS_INHERIT4( T, BASE1, BASE2, BASE3, BASE4 )
These macros tell the type system about class inheritance. This
is useful in skrit so that a pointer to a derived class can be
implicitly upcasted to a base class by the skrit compiler. This is a
convenience for skrit so that a pointer to a derived class can have
base class methods called on it. The T, BASE1?BASE4 parameters are all
just the unquoted names of your types, where up to four base classes
are supported per type. Put the macro inside of the class definition to
export the relationship. Here is an example from the UIEditBox type to
say how it is derived from UIWindow:
class UIEditBox : public UIWindow
{
FUBI_CLASS_INHERIT( UIEditBox, UIWindow );
...
Note that skrit does not provide downcasting capabilities,
because it has no way to verify if a pointer to a base actually points
to a derived type instead. For this you should export your own
functions to do the casting, and verify that the cast will work
properly before returning the casted pointer. Here is an example from
the UI again, for casting a pointer to a UIWindow to the derived
UIEditBox:
FEX UIEditBox* QueryDerivedEditBox( UIWindow* base )
{
if ( (base != NULL) && (base->GetType() == UI_TYPE_EDITBOX) )
{
return ( (UIEditBox*)base );
}
return ( NULL );
}
FUBI_SINGLETON_CLASS( T, DOCS )
FUBI_SINGLETON_NESTED_CLASS( T, NAME, DOCS )
These macros tell the engine about your singleton types
(singletons are objects that have one and only one instance in the
system). They are designed to be used with the Singleton templates
provided in dsdll.h but basically require that you declare a static
function GetSingleton() in your class that returns a reference to the
class's singleton.
Put these macros inside of the class definition. The
parameters to pass into the macro are "T", the name of the type, and
"DOCS", a quoted string documenting the class. These macros actually
contain the FUBI_CLASS_DOC macro and pass the DOCS parameter through to
it.
The NESTED version of this macro is meant for nested types,
where NAME is the name of the nested type combined with the name of its
outer type. For example a nested class Tel inside outer class Baz would
be named BazTel for the NAME parameter, but left as Tel for the T
parameter.
FUBI_POD_CLASS( T )
This macro tells the engine that the given type is POD, or
"plain ol' data", meaning that it does not contain any advanced types
(such as std::strings) or pointers, and does not require a constructor
or destructor to set up or shut down instances of the type. This makes
it easy to save/load instances of the class, send them over the
network, or pass them by value to local functions.
Note that .dsdll's will probably not find this macro useful
currently because savegame and network functions are not exportable,
but the macro will remain for future mod capability expansion.
Singleton <T>
AutoSingleton <T>
OnDemandSingleton <T>
These are all variations on a theme of automating
registration of class singletons. Just inherit your type from one of
them in order to have it automatically registered. Singleton is for
where you allocate/deallocate the singleton yourself, AutoSingleton is
used for the compiler to automatically allocate it (statically), and
OnDemandSingleton is used to tell the compiler to generate an instance
on demand (i.e. the first time it is referenced). For more details, see
the documentation inside the dsdll.h file. Background info on the
concept of singleton registration via this recursive template trick is
documented in another Gem at this address:
http://www.drizzle.com/~scottb/publish/gpgems1_singleton.htm
Here
are some questions I've been asked since .dsdll's were introduced to
the community, plus a few other miscellaneous things that didn't really
fit anywhere else in this article.
Coding is too difficult for me, but I need .dsdll functions for my mod! What can I do?
While writing a .dsdll may be difficult for some, it's not a
problem for programmers! Why not grab a spot on SourceForge and start a
"public .dsdll" project that contains all kinds of functions that
anyone might need? Set up a process where people can request features
(like "I need a standard deviation function!!") and programmers quickly
write and export them, then rebuild a new version of the .dsdll for
people to download. One uber-.dsdll could easily serve the vast
majority of extra-engine needs for the community.
Can I call engine functions from my .dsdll?
The answer to this is currently "no" for security reasons,
although community members have already discovered and published at
least one safe workaround that does not break the security model. Be
sure to check the forums on the Dungeon Siege community sites for more
information.
Can I create my own networked (RPC) functions?
No. FuBi requires direct access to a number of internal Dungeon
Siege systems that we simply cannot export. If you want to pass
messages over the network, then you will need to set up and communicate
through your own sockets via .dsdll functions.
Can I override engine functions with my own?
Yes, mostly! FuBi is built in such a way that it doesn't care
where its functions come from, and when it finds duplicate functions it
will just ignore the second version. So to override game functions, you
just need to make sure that your functions are scanned before the game
imports its own functions. Dungeon Siege supports this directly if you
rename the DLL's extension to .dsdl0 (that's a zero at the end). The
system scans functions in this order: .dsdl0 files alphabetically,
DungeonSiege.exe, then .dsdll files alphabetically.
Important: the signatures for your override functions must be
identical to the functions they're trying to override, otherwise they
will either get ignored, or you'll get a crash.
Unlike normal .dsdll exports, a .dsdl0 that overrides game
functions can have crazy unexpected behavior. Your overrides will be
called for skrit and incoming RPC's, but not for in-game C++ code
(those will still call the old versions). If messing with .dsdll's is
considered deep modding, then using .dsdl0 overrides is extreme level
150 burrowing-to-the-Earth's-core modding. So use them with extreme
caution, or you may shoot your eye out!
Can I extend the "dev console" with .dsdll's?
Yes! The console lets you type skrit code in directly by
prefixing it with a forward slash. So to call any .dsdll functions of
yours from the console, just put a slash at the front before typing in
the console. You won't have the fancy auto completion, of course, but
you'll be able to type in whatever you like, which can be very
powerful. To speed your development efforts even further, you may find
it useful to create a special debug.dsdll file that contains all kinds
of reporting and analysis functions. Then keep a text file open that is
filled with commonly used skrit code to access those functions. You can
copy commands as needed to the clipboard, then alt-tab back into the
game and paste the command directly into the console. It helps to run
the game in windowed mode (fullscreen=false on the command line or in
the INI file).
What about viruses??
This question comes up a lot, and comes from the basic fact
that a DLL has full access to the entire system, and can infect it with
a virus, or even reformat the hard drive. This fear is not unique to
Dungeon Siege, in fact most games (notably the first person shooters)
require you to download and run DLL's as part of any mod that is
created. A Dungeon Siege mod that does not use .dsdll's is only capable
of "playing in the sandbox" of the engine, and will be safe.
If you would like to learn more about what exactly is involved
here, I suggest searching through the Half-Life forums to see how their
community has handled this. It is the biggest moddable game on the
market, with millions of players.
I can't play multiplayer games with my friends any more! What's wrong?
Simply having a .dsdll available directly modifies the type
system of the game, which is nearly guaranteed to cause a latent crash
or unexpected behavior in multiplayer if it's not identical to other
games joining in. So for security and compatibility reasons, the game
takes a checksum of all its .dsdll files on startup, and includes this
in the multiplayer info packet, which is used to determine when it's
possible for two games to join each other. If one person has a .dsdll
installed, then either everyone else must have the same file, or that
one person must delete it (or move it elsewhere).
The recommended way to use a .dsdll is demonstrated in the
WebMod sample, which is intended to be launched using the webmod.bat
file. This will choose the proper .dsdll file, and install the mod and
map into the game's resource space during startup. A shortcut can be
created by an installer to accomplish similar results (our Yesterhaven
map did this as another example).
I love Delphi and hate C++, what can I do?
Have no fear! It's still possible to make .dsdll's, it's just
going to be more work. There are a couple options. First is to
reverse-engineer the name-mangling format of C++ and modify the
function exports from your Delphi DLL to match your function
prototypes, or perhaps write a postprocessor to rename your functions
directly in the export table to match how the game is expecting. This
is a ton of work, of course.
Another option is to learn enough of C++ to write a proxy
.dsdll that routes calls from the game straight through to your
Delphi-based DLL. This will let you write 99% of your code in your
favorite language without having to get your hands too dirty with C++.
How do I export a class?
The most important thing to do is make sure that the .dsdll
takes care of all the memory management for your class. To do this,
export a function to create an instance, and another to destroy it. The
skrit can call the create function to acquire an instance of the class,
and then call member functions on it or pass it around to other
systems. When it's done with it, it can call the destroy function to
delete it. It's also worthwhile including some automatic garbage
collection code to clean up as a safety measure.
Conclusion
Well
that's it for this article! Hopefully this has provided you with enough
information to create your own extensions to Dungeon Siege. Be sure to
check out the WebMod sample and the dsdll.h in detail before diving into your own project.
303: Skrit
Dungeon Siege II
