I graduated with honours degree in Computer science from University of Edinburgh. You can see some of my photos from that time here.

Final year project - Deep RL

My final year project was to learn walk-like behaviour on simulated dog-like robot with end to end deep reinforcement learning.

Read my final year project report here (received an A). Or watch some funny videos of learned behaviour below:

• Walking on a mesh - somewhat surprisingly adding uneven terrain improved learning performance. My theory is that the uneven surface filters undersired movement that the agent exhibits initially.
• Walking on flat surface - demonstrating that even small networks learned to perform forward motion. Before working on this project I have mostly studied image recognition where the networks can be gigantic. Therefore it was quite surprising for me to find that smaller networks even outperformed the larger ones.
• Stepping over obstacles
• Swing-like motion - here the agent learned to receive the reward without moving forward. This is an example of classic problem with RL where the the reward does not capture intended behaviour properly.
• Agent that learned to walk around obstacles, again I did not set up the environment properly and the agent learned to simply walk around the obstacles. I am quite happy about these results as it shows that the agent is responding to the environment.

Machine learning for weather prediction

For our Machine Learning Practical course we tried out various deep learning techniques to predict weather. We used Weather Bench dataset that made it very easy to evaluate our results and also included pre-processing of the data. Read the report.

Some other stuff I have done

Doing this degree I have spent hundreds of nights working at Appleton Tower on

• delivery robot
• hand writing assembly code
• custom compiler for C language, and separately a compiler for subset of Haskell
• Android game written in Kotlin
• A*, Minmax and other algorithms in Haskell
• question / answering system
• subset of TCP protocol
• machine learning activation layers
• working with Map-Reduce for no good reason