Skip to main content

FlatBuffers for Unity, part 1 (overview)

Seems every game development faces a problem: how to serialize and deserialize data? This question occurs in two main scenarios: saving/loading and network communication (how to send data to server or computer). In both cases you want to save some game data (items, players or perhaps game state or action) and restore this data later (might be on other server).

Most common solutions

What approaches does Unity suggest for that? I have found these variants:
  • PlayerPrefs (link);
  • String serializers (JSON or XML);
  • Binary serilization;
Let me take short overview of each of them.

PlayerPrefs

 It is one of the most simple way to store data in Unity. It uses built-in system PlayerPrefs that allows to save data in system key-value storage. It has only three functions to store base types of data (int, float and string). PlayerPrefs is not about performance or data transferring. Pros and cons:
+ Simple to use
+ Built-in
- Does not support data transferring
- Does not support complicated data types (as classes)
- Pure performance

String serialization (json, xml)

String serialization has an advantage over other methods: it can be easily read by human. So you can see how does your data serialize. Unity supports several helpers to work with these format (XMLSerializer is Microsoft's wrapper, but Unity supports it). You can combine JSON/XML with previous PlayerPrefs to store object (serialize your object and store it as string). But unfortunately these serializers have bad performance and serialized data has a large size. Pros and cons:
+ Simple to use
+ Human-readable formats
+ Support complicated object serialization
+ Open standards that have a lot of realizations (cross-language support)
- Large serialized size
- Pure performance

Binary serialization

Using of binary serialization allows to avoid problems with performance and large size of serialized data. It is also easy to use (similar to json serializing). There is a class BinaryFormatter that helps to serialize data. But unfortunately it does not provide cross-language support. It means that you are limited to C# and if your game requires for a example a server you have to write a server on C#. If it is not a problem binary serialization is a good choice.  Pros and cons:
+ Simple to use
+ Well performance (in comparison with previous approaches)
+ Support complicated object serialization
+ Small size of serialized data (in comparison with string serialization)
- No multi-language support

Other ways...

Well... what if none of mentioned variants suits you. What can you do else? Google introduced their approaches to data exchange with messages. There are two protocols developed by Google: protobuf and flatbuffers. Flatbuffers is more later protocol and it has some advantages over protobuf. Both these protocols are much more efficient than previously mentioned approaches. You can find performance tests on this page.

To my astonishment, there is pure support of these protocols in Unity. You can find some articles about using protobuf in Unity but mostly they use old-built release version of protobuf-net and it is frustrating. Also it is not as convenient as previously mentioned approaches.

About flatbuffers I have found one good article (this one) but the problem with using flatbuffers still stays on. For example Unity Editor does not support creating flatbuffers-files (*.fbs). You have to create them in other ways (for example from console or by Windows Explorer). Another inconvenience is that for compiling (translating from .fbs messages to C# or other language code) you have to run a console app (flatc) with a number of arguments. For example typical command to compile Message.fbs to Message.cs with selecting include directory is:

I have decided to use flatbuffers in my project because of these advantages:
+ Great performance
+ Small size of serialized data
+ Cross-language support

To make my life easier I have decided to develop a tool/plugin for Unity to facilitate using flatbuffers. Next part of this article contains detail overview of this tool (I called it UnityFbs). A few screenshots how it looks:

Comments

Popular posts from this blog

FlatBuffers for Unity, part 2

In previous part we have considered different ways to serialize and deserialize data. In this chapter I tell about tool that facilitates using of flatbuffers in Unity. What should it do? I have highlighted three main points. First of all it should enable to create .fbs messages from Unity Editor. Just as right-click action. Next, it should allow to compile .fbs messages to C# classes (and some other languages). And finally, it should be flexible and allow user to tune tool for his needs. Creating .fbs messages This point is quite easy: we need to add a new menu item "Create .fbs" to Assets menu. You can do it easily by adding a MenuItem attribute to your method. Additionally you can add a validation method that enables/disables menu item. I used this opportunity to check that there is a selected folder (when we need to create a new file). More information you can find here . [MenuItem("Assets/Create/FlatBuffers message")] [MenuItem("Assets/UnityFbs/C

Unity: procedure-generated levels

To begin with, let me give some explanation what was the original goal. As a weekend project I was writing a 2d game. And there was a need for a dungeon map: number of rooms somehow connected with roads. I think that it should be auto-generator, that will create various levels by input parameters (like number of rooms). At the very beginning I decided to divide the functionality into separate classes: one for generating level and another for rendering. Generator I have decided that generator has take maze X-length and  number of rooms to generate as input parameters. Additionally it has to try keep the Y-limit, but it is not so strictly as X-limit.  Well, generator will take these parameters in constructor: What about internals? I decided to use a container that can hold existing nodes and generate new one. I named it MapGrid. Inside it has a grid of nodes (List of List). First List - list of levels. It contains list of Nodes that have the same X-coordinate. On adding new N