Genetic Algorithm Based System for Patient Scheduling in

Highly Constrained Situations

Vili Podgorelec, Peter Kokol

University of Maribor Faculty of Electrical Engineering and Computer Science

Smetanova 17, 2000 Maribor Slovenia

vili.podgorelec@uni-mb.si, kokol@uni-mb.si

Abstract

In medicine and health care there are a lot of situations when patients have to be scheduled

on different devices and/or to different physicians or therapists. May it concern preventive

examinations, laboratory tests or convalescent therapies, we are always looking for an

optimal schedule that would result in finishing all the activities scheduled as soon as

possible, with the least patient waiting time and maximum device utilization. Since patient

scheduling is highly complex problem, it is impossible to make a qualitative schedule by

hand or even with exact heuristic methods. Therefore we developed a powerful automated

scheduling method for highly constrained situations based on genetic algorithms and

machine learning. In this paper we present the method, together with the whole process of

schedule generation, the important parameters to direct the evolution and how the algorithm

is guaranteed to produce only feasible solutions, not breaking any of the required

constraints. We applied the described method to a problem of scheduling patients with

different therapy needs to a limited number of therapeutic devices, but the algorithm can be

easily modified to be used in similar situations. The results are quite encouraging and since

all the solutions are feasible, the method can be easily incorporated into an interactive user

interface, which can be of major importance when scheduling of patients, and human

resources in general, is considered.

Citation Reference: V. Podgorelec, P. Kokol, Genetic Algorithm Based System for Patient Scheduling in Highly Constrained Situations, Journal of Medical Systems, Plenum Press, vol. 21, num. 6, pp. 417-427, December 1997.

1. Introduction

The problem of constructing an automated scheduling system is know to be a very complex,

especially for those situations when the human resources are involved. But it is also well

known that to do it by hand takes a lot of effort and associated administrative work, in many

cases it is even impossible. Therefore the application of computers to scheduling problems

has a long and varied history. As the first generation of computer timetabling problems in

the early 1960’s were presented programs to produce school timetables with the aim of

fitting classes and teachers to periods. Soon after that numerous different heuristic

approaches to timetabling have been introduced, including simulated annealing, constraint

logic programming, linear programming and graph coloring heuristics. But it soon became

clear that the exact methods are useless for more complex problems due to their

ineffectiveness. Therefore a lot of different non-exact or soft methods are used lately, which

do not give optimal solutions, but reasonably good solutions are obtained in a relative short

time. What kind of solution is good enough and how long are we prepared to wait for it

depends on the problem given, but all the research has one objective in common: how to find

as good solution as possible in as short time as possible.

Regardless of the used method, there are some basic rules that have to be fulfilled in order to

construct a qualitative and effective automated scheduling system. First of all, we have to

guarantee that all obtained solutions will be feasible; we have to fulfill all specific

constraints of the given problem. Moreover, the system has to be efficient enough to find an

adequate solution in a limited time to be still useful for the practical use. Beside these basic

constraints there are even more non-obligatory constraints, that improve the quality of a

scheduling system, when they are fulfilled; and in the case of patient scheduling they are

even indispensable. One such property is generality or independence of the problem. It

guarantees the use of the system for different kind of scheduling problems, not only for one

very specific situation. Second non-obligatory property is the possibility of user interaction

in the phase of solution development; when scheduling of patients is considered, this option

definitely becomes necessary. Also very useful is the ability of the system to continue the

search for the solution when one (or more) of the activities already scheduled is canceled

and removed from the schedule, or when the execution of scheduled activities starts almost

simultaneously with the scheduling.

Although some recent scheduling methods are able to provide an adequate solution in a

reasonable time, there are very little of them that include at least one of the before mentioned

non-obligatory properties. A kind of hybrid genetic algorithms have shown some prosperous

results lately [2-4], and yet they have quite a number of disadvantages. They have been to

problem dependent, which is exactly the opposite of the commonly known properties of

genetic algorithms, and could have easily evolved to a suboptimal solution. And instead of

finding the schedule itself they were searching for the set of rules of how to produce a good

schedule, what disallows user to interact with the system.

In the manner to overcome above weaknesses we developed a scheduling algorithm based on

the genetic algorithms [1,6,7,9] and machine learning [5,8]. With the introduction of actors,

resources and activities the method became problem independent. Because of the diversity

kept in a population and “the best survive” principle we avoid the premature convergence to

a suboptimal solution. In addition machine learning abolishes possible negative

consequences of badly chosen parameter values. With the adequate internal representation of

the individuals we guarantee all temporary solutions to be feasible, which assures the user to

interact with the system. Also it is very important that the user can directly influence the

direction of the evolution by weighting properly all the parameters that influence the quality

of evolved solution. And only when the method fulfills all of the above it can be considered

as a possible solution to a patient scheduling problem.

We applied the described algorithm to a problem of scheduling patients with different

physical therapy needs to a limited number of therapeutic devices and a limited number of

therapists. In this case, a solution is a schedule, which we can consider to be an assignment

of patients to time intervals on the specific therapeutic devices that are operated by physical

therapists, trained for specific devices. Obtained results turned out to be very promising and

because of the algorithm’s effectiveness and low computing resources consumption it can be

used, in our opinion, also for very complex scheduling problems.

The basic information on genetic algorithms and machine learning together with the choices

made to hit the requirements of our situation are given in Section 2. Section 3 describes the

generation of schedules with the use of our system. Section 4 presents the results obtained by

scheduling a test problem, after which the paper ends with some conclusions in Section 5.

2. Genetic algorithms and machine learning

Genetic algorithms are adaptive heuristic search methods which may be used to solve all

kinds of complex search and optimization problems. They are based on the evolutionary

ideas of natural selection and genetic processes of biological organisms. As the natural

populations evolve according to the principles of natural selection and “survival of the

fittest”, first laid down by Charles Darwin, so by simulating this process, genetic algorithms

are able to evolve solutions to real-world problems, if they have been suitably encoded. They

are often capable of finding optimal solutions even in the most complex of search spaces or

at least they offer significant benefits over other search and optimization techniques.

The variety and complexity of learning systems makes it difficult to formulate a universally

accepted definition of learning. However, the common denominator of most learning

systems is their capability for making structural changes to themselves over time with the

intent of getting more efficient in performing given tasks. One of the most important means

for understanding the strengths and limitations of a particular learning system is a clear

definition of knowledge structures, possible structural changes and the legal operators for

selecting and making those changes. There are several different approaches to changing

knowledge structures. The simplest one is the changing of parameters that influence the

system's behavior; we used this approach in our scheduling system.

Let's take a look at simple learning system model (Figure 1). Such a system is performance

oriented. We have some tasks that have to be performed (in our case a schedule generation)

and learning consists of both knowledge acquisition and refinement. System is separated into

two subsystems:

• a task component whose performance-oriented behavior is to be improved; in our case

it is a genetic algorithm that generates schedules, and

• a learning component charged with making appropriate structural changes; in our case

we try to find optimal parameters' values that affect the execution of genetic algorithm.

Figure 1. A performance-oriented learning system.

Since the execution of genetic algorithm depends only on a few parameters (three in our

case), it is beside of setting the appropriate initial values, also very important how and when

in the process of evolution those values are modified. Therefore we presented knowledge as

a set of simple rules indicating whether the parameter's value should be increased or

decreased and when this action has to be done (Figure 2). Knowledge is acquisited and

improved on the base of solved problems as we observe how the solution evolves depending

on the parameters' values. As the solution evolves we randomly choose rules and reward or

punish them (increase or decrease the probability of a rule to be executed) accordingly to

their effect. By executing different problems we can decide which rules are useful in an

actual situation.

IF (generationNum>150)AND (generationNum<200) AND (overallWaitingTime>100)

THEN IncreaseMutationProb BY 0,001%

WITH PROBABILITY 62,17% ;

Figure 2. An example of a rule. Numbers in italic indicate the values that are updated through machine learning.

3. Schedule generation

Every scheduling problem can be described with three categories: activities, actors, and

resources. Let’s have a number of actors, each of them having one or more activities to

perform. Each activity needs some number of resources in order to be performed. Scheduling

is then a process of assigning all activities to the particular time slots and to the particular

resources.

Considering the scheduling of patients, patients are the actors, therapies are the activities and

therapeutic devices and therapists are two types of resources that are necessary for a therapy

to be performed.

As we try to construct a schedule, there are some fundamental constraints that could not be

broken in order to produce feasible solutions (a solution is feasible if the problem is

executable by the given schedule). These are:

• no actor (patient) can perform more than one activity at a time,

• every resource (therapeutic device or physical therapist) can be used by only one actor

at a time (it can perform only one activity simultaneously),

• each activity (therapy) has to be performed in only one continuous time interval,

• every activity can be performed only with a specific resource, and

• some activities have to be performed in an exact time order.

A schedule that satisfies these constraints is a feasible schedule. But just because a schedule

is feasible, it does not mean it is good enough to be used. Many other criteria exist which

influence the quality of a schedule. Any of these criteria can be included into the evaluation

function, that helps us select the fittest individuals for the future evolution, as we will see

when the selection operator will be discussed later on.

3.1. Internal representation of the individuals

Internal representation of the individuals within a population has to be defined in such a

way, that it represents feasible schedule and simultaneously leaves enough space to derive

the genetic operators. It has to guarantee that all the solutions will still be feasible after the

selection and mutation process, and we should be able to randomly select the crossover point

and the influence of the mutation.

In our case, we represent the individuals with a multi-dimensional model. First dimension is

always the time, whereas all other dimensions represent all needed types of resources. In the

case of patients scheduling the model is three-dimensional with the last two dimensions

representing therapeutic devices and therapists. Objects in the model are therapies that have

to be scheduled. Their positions within the model show resources needed for them to

perform and the time order in which they perform.

And how do we guarantee that all the obtained solutions are feasible? As we construct new

solutions, only the time order is defined, not absolute time intervals, and upon this structure

genetic algorithm operates. Then, in the second phase, the exact time intervals are defined

for all the activities; as good as possible according to the time order given (see example on

Figure 3).

Figure 3. An example of internal representation of the individuals with only one resource type. First the time order is defined (1st phase), then absolute time intervals are calculated (2nd phase). Activities with

the same color belong to the same actor - therefore they can not intersect.

The internal representation of individuals is actually a multi-dimensional array, its elements

are the consecutive numbers of activities. Indexes in the first dimension represent the time

order, and other the allocation of resources.

3.2. Seeding of the initial population

One of the parameters that influences the evolution in the genetic algorithm is the size of

population. The size can change during the execution of the algorithm, or it can remain

constant. We used the later approach; in this manner the number of individuals remains

constant all the time. The actual size is determined upon knowledge gathered in the learning

subsystem.

Before we can start with the evolution, we have to seed the initial population of individuals.

They are usually generated randomly, but it is very important to create needed diversity. In

our case, individuals are created as we fill the table with the therapies in a random time

order.

3.3. Selection

For the selection scheme we used modified exponential ranking selection method. After the

evaluation of all individuals, they are sorted accordingly to their fitness score. Then we

replace existing individuals from the worst to the best by creating new ones with crossover

from two selected individuals, that still exist from the old population. When all the

individuals are replaced, the new population is generated (there is still mutation to be

applied).

Actually we never replace all the individuals in the population. In this manner we guarantee

that in every new population there will be a solution at least as good as the best one in the

previous population. The number of the preserved individuals is the second parameter that

influences the evolution. Both its initial value and its change during the execution are

controlled by the learning subsystem.

For effective selection we have to define an adequate evaluation function, that determines

the fitness score of each individual. For this case we implemented the method of negative

points given to the individuals by the evaluation function based on the values of parameters

that, for the given problem, determine the quality of the obtained solution. Less negative

points an individual has, better is its fitness score, and more chances it has to be selected for

the crossover.

For each specific scheduling problem there are some parameters that determine the quality

of the obtained solutions. Regardless of the problem given, there are some general criteria of

the quality:

• overall duration of all activities,

• time of the idleness of resources,

• overall waiting time of all actors (time when an actor has to wait between two of its

activities),

• average waiting time of the actors, and

• maximum waiting time of an actor.

For all the parameters it is good that their values are as low as possible. Therefore we can

use their values as the negative points - evaluation function is then simply the sum of

parameters’ values. But all the parameters are not equally important, therefore we introduce

weights, that increase or decrease their importance. User can select these weights and in this

way direct the searching. For example, if the highest weight is put to the parameter of

average waiting time, the algorithm will prioritize the solutions where the actors on the

average do not wait long.

3.4. Crossover

When we select two individuals from the current population, with crossover we construct a

new solution, that is placed into a growing new population (Figure 4). Both selected

individuals are divided into several parts by cutting the multi-dimensional model of internal

representation of the individuals along the timeline (first dimension) on all of the resource

types. New individual is then constructed by randomly choosing parts from both parents and

putting them together.

Figure 4. An example of two-dimensional model. Shaded fields from both parents construct new individual.

3.5. Mutation

After the new population is fulfilled with individuals, we still have to apply the mutation

operator (Figure 5). For all the individuals, except for some number of preserved ones,

mutation operator with some probability exchanges two randomly chosen activities.

Figure 5. An example of mutation. Shaded fields are exchanged.

Mutation probability is the third and last parameter that influences the evolution and it is

again determined upon the knowledge gathered by the learning subsystem. It turned out by

running different tests, that the appropriate mutation probability for described method is

quite higher than usual. It is mainly the consequence of the fact, that offsprings inherit by

crossover more information from their parents as usual. Also preserving some number of

individuals unchanged has an important role in this situation.

3.6. The process of schedule generation

For every given problem, we first generate some random test problems, execute them and in

this manner the initial knowledge needed for the adequate parameters setting is acquisited.

Also the actual problem can be executed several times to refine gathered knowledge. The

evolution process starts with the seeding of the initial population by generating a number of

schedules randomly, taking just the care that all individuals are feasible. Each individual is

then evaluated for its fitness score. According to the fitness score, better individuals have

more chances to be selected to produce new ones by crossover. The crossover phase is

repeated until new individuals fulfill the complete population (with the exception of some

number of the best individuals that are preserved for the next generation). As the population

size is constant, every time a new individual is created, one old individual (with the lowest

fitness score) is eliminated. When all the new individuals are generated, the mutation

operator is applied with some probability. Next the parameters are modified based on the

gathered knowledge. All the phases, from evaluation of each solution’s fitness score to the

creation of a new population, are repeated until an acceptable solution is evolved.

4. Application of the algorithm and the results

We applied the described algorithm to a problem of scheduling patients with different

physical therapy needs to a limited number of therapeutic devices and a limited number of

therapists. In this case, a solution is a schedule which we can consider to be an assignment of

patients to time intervals on the specific therapeutic devices that are operated by therapists,

trained for a specific device (or more of them). Patients are now the actors, therapies are the

activities and devices and therapists are two types of resources, that are necessary for a

therapy to be performed.

In this case, we have to assure the following constraints to be fulfilled in order to obtain a

feasible solution:

• every patient can perform only one therapy at a time,

• every therapeutic device can be used by only one patient at a time,

• all therapies have to be performed in only one continuous time interval and can not be

broken in several parts,

• each therapy can be performed only on a specific prescribed therapeutic device, that

can be operated only by trained physical therapists, and

• some therapies have to be performed in an exact time order.

The criteria for the quality of the obtained solutions are the following:

• overall duration of all therapies,

• time when devices are idle,

• overall waiting time of patients,

• maximum waiting time of a single patient,

• average waiting time of patients who have more than one therapy, and

• average waiting time only for those patients who are actually waiting.

Upon these parameters we construct an evaluation function that determines the fitness score

of each individual solution. Such application of the described algorithm is intended for little

specialized physical therapeutic studios. The patients in these studios are mostly children.

Therefore it is more important to consider that all the children should finish their therapies as

soon as possible, not waiting long between them, rather than the idle time of devices is very

low. By weighting the parameters in evaluation function properly we were able to get almost

an optimal solution.

Another very important feature in this kind of applications is the handling of so called late

cancellations. This situation occurs when one (or more) of the activities are canceled and

others have already begun. For example a patient got problems after a therapy and therefore

he cancels some of the following ones. In this case we try to reschedule the remaining

activities to reduce the overall duration. This is a great problem with almost all other

scheduling methods, but our algorithm performed quite well due to the diversity kept in a

population.

As an example let’s take a look at the results obtained with the described algorithm for the

situation with 5 different therapeutic devices and 22 patients, each of them with more than 2

prescribed therapies on average. Based on the gathered knowledge of learning subsystem we

set the parameters: population size was 67 individuals, mutation probability varied from

0,003 up to 2,7 %, and in each population between 5 and 6 individuals were unchanged

preserved for the next generation. There were 5 physical therapists available, each of them

for one therapeutic device. One special constraint was, that all the patients have to perform

their therapies on the device number 5 before the therapies on the device number 1. We were

looking for the solutions with the lowest possible overall duration and small waiting times

between therapies. You can see one possible solution, evolved after only 165 generations, on

Figure 6.

Figure 6. Results obtained with the described algorithm. Upper part shows the schedule for all

therapeutic devices (number in the box means patient number); lower part shows schedules for each of the patients.

The duration of all therapies in the obtained solution is 335 minutes and it is also the shortest

possible solution (look at the device number 2). Devices 1, 3 and 4 are idle for 10, 5 and 30

minutes, but that does not affect the overall duration. The maximum overall waiting time is

15 minutes for patient number 21, who has 4 different therapies. The average waiting time

for all patients with more than one therapy is 5 minutes. Patients 1, 5 and 17, who had have

to perform therapies on both devices 1 and 5 are scheduled correctly (therapy on device 5 is

always scheduled before the therapy on device 1 for the same patient).

5. Conclusion

In the paper we have described a new method for patient scheduling in highly constrained

situations. With the use of genetic algorithm and additional machine learning we have

provided an effective method that finds qualitative solutions. The introduction of actors,

sources and activities allows all kinds of scheduling problems to be described and solved.

With the modification of genetic algorithm the premature convergence to a suboptimal

solutions is avoided and successful rearrangement of existing schedules is achieved. An

adequate internal representation of individuals together with chosen genetic operators

guarantees all solutions to be feasible, and in this manner the possibility of very important

user interaction with the system. The ease of controlling the direction of evolution by

adjusting importance of the parameters that affect the quality of final solution makes the

method even more appropriate to be incorporated into an interactive scheduling tool.

Because of its effectiveness and low computing resources consumption it could be used also

for very complex problems.

References

[1] Thomas Baeck. Evolutionary Algorithms in Theory and Practice. Oxford University Press, Inc., 1996.

[2] E. K. Burke, D. G. Elliman, R. F. Weare. A Genetic Algorithm for University Timetabling. AISB Workshop on Evolutionary Computing, Leeds, 1994.

[3] E. K. Burke, D. G. Elliman, R. F. Weare. Automated Scheduling of University Exams. Proceedings of IEEE Colloqium on Resource Scheduling for Large Scale Planning Systems, Digest No. 1993/144.

[4] E. K. Burke, D. G. Elliman, R. F. Weare. A Hybrid Genetic Algorithm for Highly Constrained Timetabling Problems. Proceedings of the 6th International Conference on Genetic Algorithms (ICGA’95, Pittsburgh, USA), pp. 605-610, Morgan Kaufmann, San Francisco, CA, USA.

[5] Kenneth De Jong. Learning with Genetic Algorithms: An Overview. In Bill P. Buckles, Frederick E. Petry (eds): Genetic Algorithms. IEEE Computer Society Press, Los Alamitos, CA, USA, 1994.

[6] Stephanie Forrest. Genetic Algorithms. ACM Computing Surveys, pp. 77-80, Vol. 28, No. 1, March 1996.

[7] David E. Goldberg. Genetic and Evolutionary Algorithms Come of Age. Communications of the ACM, pp. 113-119, Vol. 37, No. 3, March 1994.

[8] David E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison Wesley, Reading MA, 1989.

[9] John H. Holland. Adaptation in natural and artificial systems. MIT Press, Cambridge MA, 1975.