IEEE Access (Jan 2020)
Simulating the Evolution of Homeless Populations in Canada Using Modified Deep Q-Learning (MDQL) and Modified Neural Fitted Q-Iteration (MNFQ) Algorithms
Abstract
It is estimated that over 235,000 Canadians experience homelessness at some point each year. With the emergence of smart cities, it would be beneficial to leverage the processing power of deep learning to assist in the planning and testing of different policies to address this issue. When examining a population of homeless individuals, one can view them as being distributed, at any one point in time, among several possible states: for example, the street or an emergency shelter. Our work aims to provide a means of simulating across these states, including no longer homeless, over time. The probability that an individual will transition from one state to another is called a transition probability. Thus, by creating a matrix of transition probabilities between all of the states, we have a transition probability matrix. If we simply approached this problem by using a mathematical model such as a Markov decision process, we run into the issue of how to accurately adjust the probabilities to produce realistic results. Ideally, we would have a model that can reasonably modify them based on real-life data. To do this, we introduce two modified deep learning algorithms; modified deep q-learning (MDQL) and modified neural fitted q-iteration (MNFQ). These algorithms dynamically produce a set of transition probability matrices for each week of the year. We discuss the modifications we made to these algorithms to adapt to the homelessness problem and create our simulation. After training our model on high resolution, weekly data, we will show that when running it on a low resolution data set that spans 3 years, our model is able to achieve a relative percent difference from the final population of 12.5%. The end result is a model that can be further improved over time with real world data to provide realistic results.
Keywords
