qRPC

The Videogame

I started working on a video game at the beginning of quarantine. Initially, I used the Unity game engine to support my goals because it seemed like such an industry leader, but quickly found that C# was not going to support my needs. When I looked at libraries for making network calls in game, I discovered that you usually use some protocol build on UDP to avoid long round trips, head of line blocking in TCP, and quicker handshake times. I didn’t want to directly implement a network protocol on it so I looked around and found Quic.

Quic

Quic is a new network protocol built over UDP. Quic is a little faster to initiate connections. In TLS connections, the TCP Syn’s and ACKs have to do their traditional handshake then the TLS protocol begins it’s Diffie-Helmen, and once the client and server have both arrived at their keys, the connections start.

Quic on the other hand “combines” the TCP introduction and the TLS handshake so there is only one round trip before starting to send application data. There are a ton of resources on what is is and what it isn’t, but the main reason I decided to use it was to

1. learn something new
2. reduce time to connection (although in hindsight, I mostly open a long-lived connection as opposed to a lot of short connections)
3. use UDP. I read in game software engineering blogs that this would be the way to go. I don’t know how much it’ll affect me though.

Now that I knew I wanted to use Quick, I began looking for a C# library that would allow me to use it in Unity. I found Quic.NET, but when I tried to get it to compile on my Mac, I ran into all sorts of issues. I also didn’t have the same IDE I’d wanted to use. Microsoft’s Visual Studio (not Code), low key sucks on Mac. Even worse, when I tried to call it from my Unity code, basically nothing worked. I fiddled around with it for a few days before giving up entirely on C#, and therefor on Unity.

I looked at Unreal Engine, but also found the same problems. There are too many abstractions around the networking layer of the engine that would prevent me from working at that level of the stack. Unreal has a graphical coding language that basically gives you a nice way to build out a state machine for game logic. If you really needed to get your hands dirty, you could write some C++ to get to the lower bits. Since I work with C++ at work, I decided not to do this because I didn’t even want to bother with setting up a build tool for it. I wouldn’t be able to set up all the Bazel scripts and keep my sanity.

Eventually I found Godot. Godot has the same kind of abstractions in the game engine for indie developers, not engineers. They have a DSL called GodotScript which feels a lot like Python, but also doesn’t let me touch the lower levels of the network stack. If you really wanted to get your hands dirty in Godot, you can write some C++, produce a dylib and give that to the game engine. Again, I wasn’t about to start writing C++ for this. The lord answered my prayers with a Rust library. Rust can build to a dylib as well, Cargo is amazing, and the community is really helpful. I’ve gotten awesome responses from r/rust.

qRPC

I was able to find the Godot bindings in Rust and a Quic implementation. The two allowed me to build the RPC framework I wanted. I got my “hello world” examples set up by building my dylib and pointing Godot to it. I took a lot of inspiration from gRPC when building my new network protocol. I’m using protobuf to serialize my data on the wire and generate the coded needed to define my schemas. It’s been a really good experience. The rust proto compiler plugin isn’t officially supported by any company, but the one I found is fine because I don’t make any network calls to an implementation in another language. All the code that I’m going to write for this video game is going to be done in Rust.

I’m calling my new network protocol qRPC and it’s deeply constrained to my current application code, but eventually, I’ll split them apart and use it in other games I make.

I’ve implemented streaming in my qRPC library and took a lot of lessons from the gRPC implementation. Through it, I learned that before bytes actually get written on the wire, gRPC will keep a 64kb buffer on the client and server side to provide a non blocking experience. I’ve done something similar with a tiny buffer stream on both the server and the client. I’ve also tried the make the application’s API with the library similar but putting the same terminology around streams and making my function headers similar.

This also forced me to think a lot about how I thread the library. I spawn greenlets pretty frequently within tokio. When I started working on this project, I very little understanding on the difference between OS threads and greenlets. I’m not going to be able to find it, but Rachel Kroll has a really good blog post about some of the shitty stuff that goes on with gunicorn and python web services. That was the first time I’d even though about OS threads since my OS class at UCLA.

I’m still working on this rust module that I might turn into a crate once it’s all polished. I’ll post another blog post eventually. Maybe I’ll even do some code generation the same way the gRPC plugin works for protoc.