FlatBuffers for Unity, part 2

In previous part we have considered different ways to serialize and deserialize data. In this chapter I tell about tool that facilitates using of flatbuffers in Unity.

What should it do? I have highlighted three main points. First of all it should enable to create .fbs messages from Unity Editor. Just as right-click action. Next, it should allow to compile .fbs messages to C# classes (and some other languages). And finally, it should be flexible and allow user to tune tool for his needs.

Creating .fbs messages

This point is quite easy: we need to add a new menu item "Create .fbs" to Assets menu. You can do it easily by adding a MenuItem attribute to your method. Additionally you can add a validation method that enables/disables menu item. I used this opportunity to check that there is a selected folder (when we need to create a new file). More information you can find here.

[MenuItem("Assets/Create/FlatBuffers message")]
[MenuItem("Assets/UnityFbs/Create flatbuffers message here")]
static void CreateFbsMessage() {
    CreateFbsFile();
}

[MenuItem("Assets/UnityFbs/Create flatbuffers message here", true)]
static bool CreateFbsMessageValidation() {
    return GetSelectedDirectory() != null;
}

I think there is nothing else worth to be mentioned. Just take a look at how it looks in the Unity Editor:

Compiling .fbs-message to C#

That is one of the most interesting part. How should it work? Do we need to download a compiler sources and integrate it into Unity? What should we do on new release? How can we provide the user with latest update?

Facing these questions I have chosen another approach. I have decided to use a compiled binary file (.exe for  Windows) and allow user to provide a path to binary to use (it will be discussed in chapter about flexibility).

Of cause this approach also has some disadvantages (for example delegating work to external executable file is not brilliant thought) but it is easy to implement, easy to configure, does not require to integrate to existing code, code has small size (especially as it is executed as Unity Editor plugin - it should compile fast), and it performs well.

So the plan is:

1. On right-button click on file check whether this file or files (you can select multiple files) could be compiled (check its extension it should be .fbs)
2. Suggest to compile to different languages (C# at least)
3. Call a compiler program with arguments defined at previous step (and additional user-defined arguments if there are)
4. Profit!

First and second step could be easily done with the same method as previously:

[MenuItem("Assets/UnityFbs/Generate C# definition")]
private static void UnityFbsGenerateCSDefinition() {
    GenerateDefinition(GeneratedOutputEnum.cs);
}

[MenuItem("Assets/UnityFbs/Generate C# definition", true)]
private static bool UnityFbsGenerateCSDefinitionValidation() {
    return CheckFbsMessageSelected();
}

Seems there is no other variant than to add same methods for each supporting languages. But all these methods are calling internal method GenerateDefinition that creates class ExeRunner and pass to it files to be compiled and target language.

ExeRunner itself gets additional arguments for compiler depending on selected target. These arguments are taken from Settings that can be changed by user (how user can set these arguments will be discussed later). ExeRunner puts together all arguments, compiler path and files need to be compiled and form a command which is running as a new process:

string arguments = MakeArguments(inputFiles, output);
var process = new Process {
    StartInfo = new ProcessStartInfo {
        FileName = Path.GetFullPath(flatcPath),
        Arguments = MakeArguments(inputFiles, output),
        UseShellExecute = false,
        RedirectStandardOutput = true,
        CreateNoWindow = true
    }
};
var started = process.Start();

Flexibility

Flatbuffers compiler has various parameters. You can find list of its parameters here. Compiler is named flatc and further I also will use this term. As you can see there are a lot of options. Some of them make sense only for specific language (for example --gen-onefile means generating single output file for C# and Go).

To make my solution flexible I have decided to make possible at least:

1. choose a compiler (set path of flatc)
2. set additional arguments for each supported language
3. change these settings user-friendly

First point is very important because of two aspects. First of all security: you may not trust me and suspect me of being spoofed an original flatc by a malicious file. The second point is supporting new releases of compiler: you just need to download and compile new flatc file instead of previous one. Of cause for convinience reasons I have added compiler for different platform (Windows, MacOS and Linux) to package but it fine if you will your binary.

I think that it is discussed enough about user-defined arguments for compiler. So let jump to the third point.

How can we allow user to change Settings of plugin (and thus change behaviour of plugin in Editor)? It became clear to me that such settings have to be somewhere in main menu: Edit > Project Setting (read more). And Unity gives us that opportunity with SettingsProvider! It remains only to use the this tool:

One small feature... 

Compiler can take an inlude directory path as one of argument. This is path to folder where included .fbs messages stored (you can read about include more detaily on flatbuffers page). In short, if you include into your .fbs messages other .fbs messages (for example warrior.fbs includes weapon.fbs) you have to inform compiler where to find these includes files.

I have thought that it would be convenient to set include directory just as right-click on folder. For example:

You also can choose a flatc-compiler with the same way

---

You can find sources on github: https://github.com/Wunder9l/UnityFbs/