Collaborations
Currently collaborating with Prof. Ansamma John and Prof. Jini Raju @ Image Processing Research Lab to incorporate deep learning ideas into the Medical Imaging field focussing on faster MRI signal acuqistion, b-value optimisations, etc., especially for the IVIM MRI technology.
I have taken the courses below to improve my independent research in RL, Medical Imaging + Deep Learning and other areas.
Courses
RL Specialisation by University of Alberta and Amii
Learnt about the first principles of RL and its applications.
Things I’ve Learnt ::
Decision-making problem with Bandits, Markov Decision Processes
Exploitation-Exploration tradeoff and handling the balance with confidence bounds
Defining the RL problem and Design choices
Choosing the right algorithm and agent architecture
Constructing good experiments and analysing the parameter study
CS285 - Reinforcement Learning UC Berkley
Learnt about the fundamental theoretical concepts of Deep RL and its applications.
Things I’ve Learnt ::
Policy Gradient (on and off-policy using Importance Sampling)
Actor-Critic (Improved baseline version of policy gradient)
Q-Learning (Evolving from Value Iteration, works best for tabular deterministic environments)
PPO (Proximal Policy Optimisation) - based on TRPO
Game Theory (Coursera - Stanford University)
This course mainly focussed on how to represent and define a game environment with the help of mathematics, so that we can analyse the strategies associated with the game in a much better way. It helps to view the rational and irrational interaction between agents/players in a game. The games can range from perfect logic games like chess,tic-tac-toe; to imperfect information games like rock-paper-scissors, poker. Beyond games, it can be used to model competition between firms, conflict between nations and so on…
Things I’ve Learnt ::
Nash - Equillibrium (You reach an impasse favourable to both,and maintain it)
Prisoner’s Dilemma
Maxmin Strategies (Maximise payoff when opponent tries to minimise it)
Strictly Dominant Strategies (Your dominant strategy will give max reward no matter what others do)
Repeated Games (Playing games repeatedly changes your notion about others gameplay)
Perfect & Imperfect Information Games
— Subgame perfection (try to win small games to win big!)
— Backward Induction (Think backwards to win games!)
Coalition Games (come together to win big)
— Shapley Value (When each person contributes in unequal way)
— Core (Person’s contributions cannot be improved further)
Pareto Optimality (Social good, a state which favours all players)
Machine Learning (Coursera - deeplearning.ai)
The famous course by Andrew Ng introduces machine learning with focus on various fundamentals of this field which include algorithms for different types of problems like Regression, Classification, Clustering, Dimensionality reduction,etc. It also encourages to follow certain principles in data cleaning, feature extraction and normalisation, usage of evaluation metrics to make practical implementation of ML projects accurate and robust.