The UF Method

Introduction

The UF Method is a method for behavioral organization (selection) in autonomous robots. The method is an arbitration method, i.e. a method in which a single behavior is active at each time, and it uses the ethologically inspired concept of utility functions for selecting which behavior to activate in any given situation.

Background on behavior-based robotics and behavioral organization

Behavior-based robotics  In the mid-1980s, a new approach to machine intelligence, known as behavior-based robotics (BBR) was introduced by Rodney Brooks and others. In contrast with traditional artificial intelligence (AI), BBR employs a bottom-up approach in which complete robotic brains are built from simple behaviors. Unlike AI, BBR does not make use of explicit world models. Instead, the information available is that which can be obtained from the sensors of the intelligent machine. This implies that situatedness is important in BBR, i.e. intelligent behavior cannot be defined separately from the system in which it is implemented.

 

The bottom-up approach used by BBR resembles, to a great extent, the processes that have led to intelligent behavior in biological organisms. In evolution, the first organisms were extremely simple. They were followed by unicellular organisms such as bacteria, and later by multicellular organisms with nervous systems capable of more complex behavior. Thus, it is natural to take inspiration from biological systems in the development of behavior-based robotic brains (note that we use the term "robotic brain" rather than "control system", as the latter term implies a more limited system based completely on classical control theory). 

 

Individual basic behaviors, such as avoid obstacle, follow wall, pursue target etc. can be generated and implemented in a variety of ways. The simplest behaviors can often be generated by hand, even though machine learning methods such as neural network methods and evolutionary algorithms often achieve better results.

 

Behavioral organization While it is relatively easy to generate individual basic behaviors, combining them into a complete behavioral repertoire (i.e. a robotic brain) is much more complex. This problem, known as the problem of behavioral organization (also known as behavioral selection or action selection) is, in my view, one of the most important problems in the field of BBR, and a problem that must be solved, if BBR is to generate robots capable of truly intelligent behavior, thus allowing reliable performance in a variety of real-world situations.

 

Several methods for behavioral organization have been suggested in the literature, e.g. subsumption, the potential fields method, DAMN, activation networks etc. However, most of these methods require that the experimenter be able to set several important parameters by hand, often in ad hoc structures that differ from application to application. I believe that this is not the right way to go: in any realistic real-world environment, the number of different situations and events that a robot may encounter is so large that it is extremely difficult to try to set the parameter of its brain by hand. Instead, I strongly believe that the system for behavioral organization

should be generated in a unified and general manner which the experimenter should not have to be concerned with, once a few initial specifications have been made. Furthermore, taking a clue from ethology, one may note that animals generally divide their time between different activities in highly efficient ways, i.e. they are able to solve the problem of behavioral selection very well. Thus, it is not far-fetched to use biologically inspired methods when generating behavioral organization systems for autonomous robots. Two important aspects in this context are (1) evolution and (2) utility functions. Evolution is evidently a very efficient method for optimization, and thus an important factor in the ability of animals to select behaviors appropriately. Furthermore, artificial evolution (i.e. evolutionary algorithms), once initialized, can proceed with a minimum of interference from the experimenter, thus limiting the amount of hand-coding needed. Utility functions also play an important role in behavioral selection in animals. It is invariably so that behavioral selection involves a tradeoff where less important tasks must be sacrificed in favor of more urgent tasks. One can show that, a rational agent, acts so as to maximize a quantity that can be called utility. Clearly, utility functions are an abstact concept, and the animal will, in general, not be aware that it is maximizing a certain quantity. Nevertheless, the concept of utility is a useful abstraction and functions as a common currency, allowing an animal (or a robot) to assess the relative importance of different behaviors in any given situation.

Brief description

Here, only a very brief description of the method will be given. For a more thorough description, see e.g. refs. [1],´and [2]  below*. The main purpose of a behavioral organizer based on the UF method is to handle the activation of appropriate behaviors at all times, based on the utility functions associated with each behavior. In the UF method, the utility functions depend on three types of variables: external variables (measuring e.g. IR sensor readings), internal physical variables (measuring e.g. battery levels) and internal abstract variables (measuring the variation of internal signaling variables, known as hormones), and are obtained using an evolutionary algorithm (EA). Simplifying somewhat, in the UF method, the active behavior is the one with the largest utility (even though behaviors are normally arranged in hiearchies, as described in [5]). One of the main aims of the method is to make possible the generation of robots exhibiting complex (and useful) overall behavior, without requiring that the user should solve the difficult problems of writing complex behaviors and specifying their relative merits in different situations.

An example of the dynamics of evolved utility functions is shown in Fig. 1 below. Note that the utility functions are here shown as functions of time. However, each function depends on several variables, such as sensory readings, hormone levels etc. The figure shows the resulting dynamics for one particular sequence of situations.

Fig. 1 An example utility functions in action. In this case, the robot was equipped with four behaviors. The behavior with maximum utility is active Switching between behaviors takes place when utility curves intersect. Thus, for example, a swtich from B4 (dot-dashed) to B3 (solid) occurs at time t = 135.

In a typical application of the UF method, the user would write software (classes) for the individual, basic behaviors, specify the fitness functions for the task behaviors (for all other behaviors (auxiliary behaviors) no fitness functions need be specified), specify an arena and a robot type and, finally, specify an ansatz for the utility functions and the hormone functions (describing the variation of hormones). Typically, the user would employ the UF Library software library developed by Mattias Wahde and Jimmy Pettersson,  for writing individual behaviors, as well as for the specification of the arena and the robot type.

* Note that the method was originally (e.g. in [2]) called the utility manifold method. The name was changed to the utility function method in order to better emphasize the roles of those functions in the behavioral selection procedure.

Publications dealing with the utility function method

  1. Wahde, M. (2003) A method for behavioural organization for autonomous robots based on evolutionary optimization of utility functions, Journal of Systems and Control Engineering, 217, pp. 249-258, 2003

  2. Wahde, M. (2004) Evolutionary Robotics: The Use of Artificial Evolution in Robotics , pp.63-69 (tutorial presented at IROS 2004, Sendai, Japan)

  3. Pettersson, J. and Wahde, M. (2005) Investigation of a balancing behavior using recurrent neural networks and biologically inspired computation methods, IEEE Trans. Evol. Comp., 9(5), pp. 506-521, 2005

  4. Wahde, M., Pettersson, J., Sandholt, H. and Wolff, K. Behavioral Selection Using the Utility Function Method: A Case Study Involving a Simple Guard Robot, In Proc. of the 3rd International Symposium on Autonomous Minirobots for Research and Edutainment (AMIRE 2005), pp. 261-266, 2005

  5. Wahde, M., and Pettersson, J. A General-purpose Transportation Robot: An Outline of Work in Progress, In: Proc. of the 15th International Symposium on Robot and Human Interactive Communication (RO-MAN 06), Hatfield, United Kingdom, pp. 722–726, 2006

  6. Pettersson, J., and Wahde, M. Improving generalization in a behavioral selection problem using multiple simulations, In: Proc. of the Joint 3rd International Conference on Soft Computing and Intelligent Systems and the 7th International Symposium on advanced Intelligent Systems (SCIS & ISIS 2006), Tokyo, Japan, pp. 989–994, 2006

  7. Pettersson, J., Sandberg, D., Wolff, K., and Wahde, M.  Behavioral selection in domestic assistance robots: A comparison of different methods for optimization of utility functions, In: Proc. of the 2006 IEEE Conference on Systems, Man, and Cybernetics (SMC 2006), Taipei, Taiwan, pp. 4904-4909, 2006