Sunday, May 31, 2009

Change of plans

Sometime when developing you come across an unexpected twist and new motivations. My latest "twist" is that for all intensive purpose the server part of my program is being put on hold. Trying to justify this with myself was hard because I feel that my only complete projects were ones I was forced to develop on the job. The new project I'm working on has several goals which I believe are more realistic then a "generic" mmo server.

Some of you may know about my 2d engine tutorial series on gpwiki.org . Since it has been released I've continued to get emails about it, more specifically how to compile/use the code rather than design. The whole purpose of those tutorials was to give people an idea of how to structure an engine rather than something to use. Some people have requested feature enchantments and others want to know when the next version is coming out. About a year ago I started on a new version of the engine. After awhile I was too busy at work and life in general and forgot about then abandoned it.

Since creating that project I've done a lot more engine design and coding and feel like it can be done better. Some of the bigger issues in my mind is that it uses immediate mode in OpenGL, which is slow and generally not that great to work with compared to VBOs and VAOs. The engine also isn't well documented nor structured in a consistent way and worst of all it carries some of my bad habits of using Get/Set in front of things where it isn't need and hungarian notation for private variables. I did delete the project but then the next day I decided that I will leave it up as a code sample and change the front page and license to reflect it's status.

So why the new engine? Besides the things listed in the last paragraph, I'm becoming more interested in modern graphics programming and 3D in general. I've done 3D and even a 3D engine for my current job however I haven't messed with modern features much such as VBOs, VAOs, shaders, FBOs which seem to be all the rage these days. Another big pushing factor for a new engine is to back-port what I do to my companies engine to make my life easier. Since my company is a startup I can really push the direct that the engine goes quite easily. The last point about the new engine is that it doesn't meaning that I have to give up the mmo idea, it just means I'm switching work from the server to the client. I still plan on making the server in scala as I feel it's suited well for that task but I also feel that C++ is better suited for the client (lack of game related libraries on java/scala).

Right now I've been working on the engine in my free time during the last week, I've put about ~15 hours into the project and feel it's coming along rather nicely. So far I've implemented input (keyboard, mouse), windowing, some math (vector2 & 3), vertex buffer formats, textures and vertex buffers. So far I feel that the VBO related stuff is the most interesting.

Right now the formats I support are:
  1. VertexNormalTexture
  2. VertexTexture
  3. VertexNormalColor
  4. VertexColor
They seem to be the common formats out there and if needed you can always plug-in your own formats if you want a custom one.

Here is the code for the vertex buffer:

/*
* File: vertexBuffer.hpp
* Author: Sean Chapel
*
* Created on May 30, 2009, 7:56 PM
* Copyright 2009 Seoushi Games. All rights reserved.
*/


#ifndef _VERTEXBUFFER_HPP
#define _VERTEXBUFFER_HPP


#include "vertexFormats.hpp"
#include "resource.hpp"
#include
#include "types.hpp"
#include "platform.hpp"


namespace k2e
{



/**
* A vertex buffer object that stores a fixed size of elements
*/

template <class T>
class VertexBuffer : public Resource
{

public:

/**
* A default constructor
*/

VertexBuffer()
{
}

/**
* A constructor with the number of elements and thier types
* @param numElements the number of elements to store
* @param type the type of vertex buffer
*/

VertexBuffer(unsigned int numElements)
{
SetSize(numElements);
}


/**
* Default destructor
*/

~VertexBuffer()
{
Dispose();
}

/**
* Sets the number of elements for the VBO
* @param numElements the number of elements to store
*/

void SetSize(unsigned int numElements)
{
buffer.clear();
buffer.reserve(numElements);
}

/**
* Adds an element
* @param v the element to add
*/

void AddElement(const T &element)
{
buffer.push_back(element);
}


/**
* Generates the buffer for use
*/

void Generate()
{
if(buffer.size() == 0)
{
return;
}

Dispose();

glGenBuffers( 1, &glId );
glBindBuffer( GL_ARRAY_BUFFER_ARB, glId );

glBufferData(GL_ARRAY_BUFFER_ARB,
buffer.size() * sizeof(buffer[0]),
&buffer[0], GL_STATIC_DRAW_ARB);

glUnmapBuffer( GL_ARRAY_BUFFER_ARB );
glBindBuffer( GL_ARRAY_BUFFER_ARB, 0 );

isResourceLoaded = true;
}


/**
* binds the buffer for use
*/

void Bind()
{
glBindBuffer(GL_ARRAY_BUFFER_ARB, glId);

T::SetupDraw();
}


/**
* Unbinds the buffer
*/

void Unbind()
{
T::TeardownDraw();
}


/**
* Implements the resource's dispose method
*/

virtual void Dispose()
{
if(isResourceLoaded)
{
glDeleteBuffers(1, &glId);
isResourceLoaded = false;
}
}

private:

u32 glId;
std::vector buffer;
};


} /* k2e */


#endif /* _VERTEXBUFFER_HPP */




If you noticed, every method is commented in doxygen format. It takes a lot long to code when adding all the comments up front but it also means that everything is documented as it progresses so people can actually use the code. All of my code so far has been commented like this and I feel it makes the code much nicer to work with and it makes me think of design before implementing something. When making my code I first design out all of the header file and make sure I'm not forgetting something then I'll comment the header then code. I find this method good pratice but it is more time consuming, fortunatly I don't have to hit crazy deadlines with this code.

Anyways about the code itself, you will notice it's templated which is something the kore2engine made little use of even though there are places that could definatly use it. It's also defined as a resource which means that I can easily have it managed by a resource manager (that's a topic for another blog post however).

Lets take a look at one of the texture formats that plug into the vertex buffer.


struct VertexTexture
{
/**
* A constructor that initializes all components
* @param position the verts position
* @param textureCoords the verts texture coordinatess
*/

VertexTexture(Vector3 position, Vector2 textureCoords)
{
this->position = position;
this->textureCoords = textureCoords;
}


/**
* Used for setuping up drawing with this format used by VertexBuffers
*/

static void SetupDraw()
{
int size = sizeof(Vector3) + sizeof(Vector2);

glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);

glVertexPointer(3, GL_FLOAT, size, (char*)NULL + 0);
glTexCoordPointer(2, GL_FLOAT, size, (char*)NULL + sizeof(Vector3));
}


/**
* Used for tearing down drawing with this format used by VertexBuffers
*/

static void TeardownDraw()
{
glDisableClientState(GL_VERTEX_ARRAY);
glDisableClientState(GL_TEXTURE_COORD_ARRAY);
}


Vector3 position; /**<>
Vector2 textureCoords; /**<>
};


Nothing really too interesting, but it just goes to show how easy it is to make your own format.

So lets show some vertex buffers in use, in a simple application I'm using to test functionality and develop with I needed a hexagon grid. Here is the code to generate and draw a hexagon in a vbo.


HexGeometry::HexGeometry(float radius)
{
vbo.SetSize(8);

vbo.AddElement( k2e::VertexTexture(k2e::Vector3(0.0f, 0.0f, 0.0f),
k2e::Vector2(124.0f / 256.0f, 112.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(-radius/2.0f, radius, 0.0f),
k2e::Vector2(67.0f / 256.0f, 8.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(radius/2.0f, radius, 0.0f),
k2e::Vector2(189.0f / 256.0f, 8.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(radius, 0.0f, 0.0f),
k2e::Vector2(249.0f / 256.0f, 112.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(radius/2.0f, -radius, 0.0f),
k2e::Vector2(189.0f / 256.0f, 218.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(-radius/2.0f, -radius, 0.0f),
k2e::Vector2(67.0f / 256.0f, 218.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(-radius, 0.0f, 0.0f),
k2e::Vector2(7.0f / 256.0f, 112.0f / 256.0f)));
vbo.AddElement( k2e::VertexTexture(k2e::Vector3(-radius/2.0f, radius, 0.0f),
k2e::Vector2(67.0f / 256.0f, 8.0f / 256.0f)));

vbo.Generate();
}

HexGeometry::~HexGeometry()
{
}

void HexGeometry::Draw()
{
vbo.Bind();

glDrawArrays(GL_TRIANGLE_FAN, 0, 8);

vbo.Unbind();
}


The engine I wrote for my company would have been atleast three time as long because in that engine I decided to separate each component into their own buffer and force using an index array even if they were in order like above. Also at the moment that engine can only use GL_TRIANGLES, which made it easier to implement and use but less efficient along longer to setup. Overall I think this is pretty nice. Here is a screen shot from my test app with a grid of hexagons drawn with the above code.



Next time I'm unsure of what I will share and talk about, perhaps the resource manager would be a nice next step or possibly 2d graphics (My test app needs a gui).

Also I'm sorry about the code formating, since html ignores whitespace it makes it a pain to mess with. I've been using http://quickhighlighter.com/ and it looks great on their servers but this blog seems to mess it up for whatever reason, if someone has an idea on how to fix this let me know. Soon enough I'll publish all my code somewhere and I can just give links to a well formated html version.

Thursday, May 14, 2009

A Simple Server in Scala

Surprisingly enough there isn't much information regarding using sockets, scala and actors together. I figured that something like this would be quite wide spread however I found it straight forward to create. I took my knowledge of actors and then followed some sample code for Java sockets.

Right now all the server does is listen for connections and print out what clients send to it, nothing special. There are some downsides to the way I'm doing things. The actor class by default spawns it's own thread which can get ugly fast however it is easy enough to convert over to coroutine like behavior. The reason I haven't done this yet is because I'm not using select to see if there is data so waiting for one receive or in this case line would block the whole thread and that wouldn't be nice. Anyways here is my code.


  1. package stdServer
  2.  
  3. import java.io._
  4. import java.net.{InetAddress,ServerSocket,Socket,SocketException}
  5. import scala.actors.Actor
  6. import scala.actors.Actor._
  7.  
  8.  
  9. object Server
  10. {
  11.   def main(args : Array[String])
  12.   {
  13.     val port = 6669
  14.    
  15.     try
  16.     {
  17.         val listener = new ServerSocket(port)
  18.         var numClients = 1
  19.  
  20.         println("Listening on port " + port)
  21.        
  22.         while (true)
  23.         {
  24.             new ClientHandler(listener.accept(), numClients).start()
  25.             numClients += 1
  26.         }
  27.  
  28.         listener.close()
  29.     }
  30.     catch
  31.     {
  32.         case e: IOException =>
  33.             System.err.println("Could not listen on port: " + port + ".")
  34.             System.exit(-1)
  35.     }
  36.  
  37.   }
  38.  
  39. }
  40.  
  41.  
  42. class ClientHandler(socket : Socket, clientId : Int) extends Actor
  43. {
  44.   def act
  45.   {
  46.     try
  47.     {
  48.         val out = new PrintWriter(socket.getOutputStream(), true)
  49.         val in = new BufferedReader( new InputStreamReader(socket.getInputStream()))
  50.      
  51.         print("Client connected from " + socket.getInetAddress() + ":" + socket.getPort)
  52.         println(" assigning id " + clientId)
  53.      
  54.         var inputLine = in.readLine()
  55.         while (inputLine != null)
  56.         {  
  57.             println(clientId + ") " + inputLine)
  58.        
  59.             inputLine = in.readLine()
  60.         }
  61.  
  62.         socket.close()
  63.      
  64.         println("Client " + clientId + " quit")
  65.     }
  66.     catch
  67.     {
  68.         case e: SocketException =>
  69.             System.err.println(e)
  70.      
  71.         case e: IOException =>
  72.             System.err.println(e.printStackTrace())
  73.          
  74.         case e =>
  75.             System.err.println("Unknown error " + e)
  76.     }
  77.   }
  78. }
  79.  
  80.  


One of things I like about this code is that it shows off mixing Java code with Scala. If you haven't noticed it fits right in and you can't really tell that there is Java embedded.


I also wrote a version of this code with Scala's remote actors however it seems limiting. When I tried to connect to telnet and typed something my server instantly crashed with an out of memory error. I'm guessing that remote actors only can only talk to each other which means that I would have to learn the protocol that it is using and mimic it in my client thats not written in Scala and then hope no one tried to use telnet on it. I'd rather just use normal TCP/IP sockets and know that it will work with pretty much anything.

Well thats it for now, next time I hope to have an non-blocking version of this code with concurrent actors and perhaps some basic functionality.

Tuesday, May 12, 2009

Prototyping the MMO

So five days or so have gone by since I last posted on the MMO server development. Like (almost) all personal projects things are going slow but some progress has been made. I started making a server in python and using telnet for testing. I got as far as having multiple clients being able to connect, clients sending a user name/password (simple text strings) and the server telling the client if the authentication was correct by using a text file as my database. It worked well enough even though the sockets were blocking and synchronous, I didn't really care about scalability nor performance as it was just a prototype.

Being the language nazi I am, I found python not to my liking. First I will state some of the things I do like just to be fair.
  1. Indentation instead of brackets.
  2. No semicolons at the end of lines.
  3. Very feature rich set of common functions/tasks.
  4. No compile times.
  5. An interactive console (I miss this in most languages ever since I started playing with lisp/scheme).
  6. Good documentation and beginners guide.
So now for the things I didn't like.
  1. Classes are littered with self because they aren't real classes but rather namespaces.
  2. Global variables have nasty names __init__, __main__ for example.
  3. The state of flux between 2.5, 2.6 and 3.0. Most libraries are still only usable with 2.5 and then some of the bigger ones work with 2.6 but 3.0 is "virtually" unsupported which is shame.
  4. Lack of type annotations when wanted, for example I would love to be able to require that functions only accept and return certain types. This is just a drawback of dynamic languages for the most part however.
  5. Spaces and not tabs. Why people prefer spaces over tabs alludes me, tabs allow you to move around faster with the arrow keys and allow you to setup your preferred spacing. Mixing tabs and spaces is silly however as it will never look right between different computers. (let the religious wars begin)
Now most of these "cons" are just my opinion and I could probably live with them however #3 really annoys me. Once I got up my simple server I wanted a graphical interface for logging on my server and I decided to use wxWidgets except it only supports 2.5 and 2.6 (sort of). The reason I said sort of is because it actually doesn't work on 2.6 on windows because of some gui dll conflict, at least thats what google had to say. It ended up crashing anytime I moved my mouse over a very simple window. Hopefully this will be fixed in the future but it really makes me wonder about the state of python on windows in general. Windows seems to be a second class citizen in general when it comes to programming languages that are either open source or non-mainstream.

I decided to not take the risk and I started writing equivalent programs in other languages. I made a server in C# with async sockets and I really just didn't like the feel of it. I also made a version in C++ and boost.asio which I liked better than the C# version but I know that C++ will just lead to headaches later on and isn't a good prototyping language. Not being satisfied with either solutions I decided to re-look at some of the languages I skimmed over for one reason or another. In conclusion Scala is my next choice for prototyping a server with.

I looked over some of the beginning docs and I liked what I saw so I picked up "Programming in Scala". So far I'm only at chapter 7 however I really like the way the book is written and how material is presented. Most computer science or tech books bore the crap out of me but for whatever reason I like PinS. So what drove me to taking a look at Scala?
  1. The mix of function, imperative and object oriented paradigms.
  2. Having an actors library for concurrent processes in the same thread.
  3. Statically typed with type inference and the option to declare types if you want.
  4. Runs on the JVM and mixes well with Java (cross-platform goodness).
  5. Short concise programs, much like python is compared to C like languages
  6. The syntax is extensible and makes DSLs easy to create
  7. A familiar syntax without many odd operator symbols for everything.
In feel like Scala is mostly related to OCaml however with an object system that people actually use, mixed with Haskell except without forcing you to be purely functional and odd syntax choices. Of course it gets its influences from other languages as well.


Next time I hope to be able to post some code and my experiences with Scala.

Monday, May 4, 2009

New Project

So I haven't updated my blog in awhile, I'll talk a little bit about languages I've been messing with then I'll move into my new pet project.

Haskell was the last language I posted about and pretty much nothing has changed in my mind about it. It is a neat language and I hope to use it more some day however monads have really turned me off for the moment. Why have monads turned me away? Well it's actually quite simple, being a game programmer in C like languages I'm use to state and monads just makes it either more difficult, painful or ugly compared to the "C" way of doing it. There is always functional reactive programing which is suppose to be the new "cool thing" for programming in Haskell however I just don't feel like learning them as the concept is quite academic (like a lot of things in Haskell) and not really proven for game development yet. So to conclude I like Haskell as a language and it brings on some neat ideas however for game programming I'm having a hard time accepting it.

After I looked at Haskell I stumbled upon factor. Factor is a neat stack based language derived from Forth. I really liked how simple it was, push stuff on the stack then push a function that read the previous stack items and put the result on the stack. It seemed a lot like functional programming but a bit different. I really loved the idea of an image based programming language, basically what that means is that the programming language, libraries and programs are contained in an image file. What this means to the user is that it's very simple to import/include other modules as the programming language knows about everything in it's image. It also makes it painless to add new modules to the image, no need to worry about system paths and the such. As I worked with the language I just couldn't get my brain trained to think in the way factor wants you to (basically backwards or postfix). Because the language is stack based it depends what is already on the stack meaning you really don't pass arguments to a function it just reads from the stack. Also it reads the variables on the stack from back to front meaning it seems backwards in that way as well "push 3; push 4; +"(pseudo code). you would think that this translates to "3 + 4" if it was converted to infix but it's really "4 + 3". This seemed to cause me pain when coding, it made comprehending the code harder. I thought it would get better with time but I gave it a week of coding in my spare time and I felt no better than when I started. Curiously I looked around about factor and according to other blogs some guys had been programming in it for over a year and still are having the same problem I am. I decided that I will look at the language at another time.

Given all the languages I've looked at I got somewhat burnt out on looking for something to replace C, I decided I'd rather just make something. I decided up front that I wanted to make something that deals with networking as I haven't explored that area like I have graphics, sound and general game programming. Networking is becoming more and more important for game developers, it's getting harder to sell single player games and not having much experience with it I feel lacking.

So what did I decide on? I decided I want to design an mmo server. I'm unsure how far I will get with this but I do know I will take away some good knowledge from it. Right now I'm in the planning stages, designing protocols, server layouts and road maps/feature lists on how I will develop it. If your interested in this then I'm keeping all the information at my site here. Being a game programmer and having used C/C++ extensively, I naturally decided to use C++ for my language of choice without really considering my options. Upon doing research I came across Eve Online and surprisingly they use (stackless) python of all languages, this really surprised me because of how slow it is compared to C and how it had to handle tens of thousands of connections at a time (I believe peak is arround 50-60k players all on the same world). Also while I heard of python before I hadn't heard of stackless python. Apparently stackless is a modification to regular python for continuations, coroutines and talking between coroutines.

Now I saw why Eve could handle so many players at once. Still I wasn't exactly convinced python was the right choice so I started looking at other languages like erlang, boo, scala, clojure and even C#. I'm sure all those languages would work for an mmo server as well but besides from C# those are relatively obscure programming languages and if I ever wanted help (on the project, not in general) I probably shouldn't choose them. I then looked into the state of mono on the mac as that is my work machine and if it didn't work well at home and work then I didn't want to use it. I have to say mono has matured quite a bit since I last looked at it (over a year ago) however monodevelop still lacks a debugger and the coroutines that were recently hacked in were a hack that only mono has (.net doesn't support them) and the memory usage was almost double that of stackless python. So I've come to the conclusion that stackless python while it may not be the end result it will be good enough for prototyping and if I really need speed I can always extend with C.

I also must thank Richard Tew (for his posts on gamedev), the Eve Online developers (for their video presentations and developers blog) and lastly the other peoples that responded to my thread at Gamedev.net for helping me come to my conclusions on the subject so far.

In conclusion I've started work designing an MMO server/client and hope to have something to show for it in a couple of months. Lastly I've decided to start using twitter again for the sole purpose of letting people know about my latest developments on the MMO and this blog. You can follow me as Sean Chapel if your interested.