|
Abstract:
|
Reinforcement learning (RL ) methods have become popular in recent years because of their ability to solve complex tasks with minimal feedback . While these methods have had experimental successes and have been shown to exhibit some desirable properties in theory , the basic learning algorithms have often been found slow in practice . Therefore , much of the current RL research focuses on speeding up learning by taking advantage of domain knowledge , or by better utilizing agents’ experience . The ambitious goal of transfer learning , when applied to RL tasks , is to accelerate learning on some target task after training on a different , but related , source task . This dissertation demonstrates that transfer learning methods can successfully improve learning in RL tasks via experience from previously learned tasks . Transfer learning can increase RL’s applicability to difficult tasks by allowing agents to generalize their experience across learning problems . This dissertation presents inter -task mappings , the first transfer mechanism in this area to successfully enable transfer between tasks with different state variables and actions . Inter -task mappings have subsequently been used by a number of transfer researchers . A set of six transfer learning algorithms are then introduced . While these transfer methods differ in terms of what base RL algorithms they are compatible with , what type of knowledge they transfer , and what their strengths are , all utilize the same inter -task mapping mechanism . These transfer methods can all successfully use mappings constructed by a human from domain knowledge , but there may be situations in which domain knowledge is unavailable , or insufficient , to describe how two given tasks are related . We therefore also study how inter -task mappings can be learned autonomously by leveraging existing machine learning algorithms . Our methods use classification and regression techniques to successfully discover similarities between data gathered in pairs of tasks , culminating in what is currently one of the most robust mapping -learning algorithms for RL transfer . Combining transfer methods with these similarity -learning algorithms allows us to empirically demonstrate the plausibility of autonomous transfer . We fully implement these methods in four domains (each with different salient characteristics ) , show that transfer can significantly improve an agent’s ability to learn in each domain , and explore the limits of transfer’s applicability . |