Optimizing H.265 Real-Time Streaming

In this thesis, we tried to optimize the current State-of-the-Art H.265 live streaming especially for panoramic video captures. The idea was to utilize the tiling technique used in the H.265 codec, which allows the video to be divided into independent regions. By transmitting only the currently seen part of the panoramic video in high quality, and reducing the quality the further away the Tiles are from the current viewport, we can reduce the bandwidth requirements while still maintaining a seamless, low latency, high-quality live experience.

To realize this, we set up a server environment that is capable of receiving a H.265 encoded panoramic live video stream, which is then converted into multiple quality levels while splitting the entire panoramic video into individual regions, which can be downloaded separately. To obtain those tiles in downloadable format, we utilized the DASH protocol, which is capable of creating a media presentation description file that includes all the information about the position, time, and quality level of each tile and can be interpreted by a client.

For the client side, we decided to use the Unity Engine to create an Android application which can be run on VR devices as well as smartphones. There we implemented a tile selection method that can divide the tiles into four different groups depending on their position relative to the current view of the user. With that information, we download the correct files from the server and recreate a playable video containing multiple quality-level tiles, which drastically reduces the network requirements. In addition, we implemented a custom video player using the native Android Media library to ensure seamless video playback without stutter or delay and provide the user with the best possible viewing experience.