Chapter 2: Work work...

One of the people here at ICT was already working on a similar project where he needed to record audio and/or video while people were playing a simple game. He provided me with the framework that he had built so far and gave me some valuable tips in what to look out for. This framework was built in Java using a Tomcat server, with access to a MySQL database and it used a WebSocket connection to send game events and the SimpleWebRTC framework to send video and audio to the other player. In order to analyze the video and/or audio, these fragments were also manually sent to the Tomcat server for storage.

Using the information that this person had given me, I started building the back-end for my poker game in Java for use on the Tomcat server. I submitted a request to our IT department to set up a Tomcat server for me, which they supplied a couple days later. It took me about two weeks to get a working prototype, where two people could connect to the server and the server would connect these players together in a game, where their webcam images would be sent using the SimpleWebRTC framework. This prototype did not store any information yet, however.

Example of poker game in action

I sincerely disliked the fact that the webcam video was essentially going to be sent twice; once to our Tomcat server for storage, and once to the other player. I wanted to find a way to only send the webcam images to the Tomcat server. This server would then store the image and send a copy to the other player. The SimpleWebRTC framework only allows you to send the images to another WebRTC-enabled client, which is usually only available in JavaScript. There is a WebRTC framework available in Java (called IceLink), but this seemed quite complicated to set up properly. Eventually I decided to use a more basic approach. I will capture just about every webcam video frame and send this frame in base64 encoding to our Tomcat server (using POST). The Tomcat server will store this image as a JPEG file and send the encoded image to the other player using the already existing WebSocket connection. The receiving client would simply display the incoming image on screen.

Surprise!

This solution seemed to work fine for a while, but after about 10-15 seconds the WebSocket connection would crash, which meant that both the video and the game events would not work anymore. After a more thorough investigation I found that the connection would only crash whenever the client tried to send a game event to the server, such as placing a bet. I've never conclusively found what the reason for the crash was, but I expect that the crash happened because the WebSocket connection was busy receiving an image file, while the game is trying to send a bet placement message, which cannot be done concurrently, so it simply throws an error and stops altogether. By using a seperate WebSocket connection that is only to be used for video, this problem seemed to be resolved and the connection does not crash anymore.