PLANEC - A EUROPEAN PROJECT FOR DEVELOPING AN INTELLIGENT DECISION SUPPORT SYSTEM FOR THE CARE OF THE ELDERLY Background of the PLANEC-project What is the PLANEC-project? PLANEC-application as planned in the project plan The concept of care in the PLANEC -research Concepts of monitoring, planning and evaluation in the Planec -research The production of welfare -approach Research to be undertaken in the PLANEC -project The approach of iterative application development Object-oriented analysis (OOA) Methodological triangulation and research dependencies in the PLANEC-research REFERENCES PLANEC - a European project for developing an intelligent decision support system for the care of the elderly Marja Vaarama, Ph.D. Stakes, Social Research Unit P.O. Box 220 00531 Helsinki, Finland tel. +358 0 39672213 fax. +358 0 39672007 email:marjav@stakes3.fi Background of the PLANEC-project Care markets for the elderly have undergone several reforms specially during last decade, and more is to be expected due to the population ageing. In almost every European country the proportion of elderly people will increase in the years and decades to come. By the year 2000, in the European Union older people will comprise more than one-fifth of the population, and by 2020 they will represent more than one quarter. More important, the average age within the elderly population will increase significantly, resulting to increasing number of frail elderly people in need of external help and care. Simultaneously the changing family patterns tend to decrease resources for informal care giving. (see e.g. Pacolet & Wilderom 1991, Walker & al. 1993, 13). Since 1980'ies, the discussion of how to stop the increase of public expenditures has strengthened in Europe, resulting to several reforms in the care systems. Usually the reforms seek to alter the balance between institutional and community care, moving the balance of provision away from nursing and residential care homes in favour of domiciliary-based support. Greater provider pluralism is explicitly encouraged within the developing 'mixed economy' of care. (see e.g. Jamieson 1991, Nijkamp & al. 1991, Knapp 1995, Munday 1995). These developments indicate important consequences for the size of public care sector in every European country. The issue of how to provide care in a cost-efficient way for increasing numbers of frail older people is becoming more and more important question. Scarcity of means makes unavoidable to choice between different alternatives. When addressing the choices in welfare and health services distribution, more attention should be paid on the allocative efficiency of the care system. (Knapp 1995). Current and future developments of markets for health and social care in European countries generate new information needs. In these circumstances policy-makers, planners, purchasers and providers of care need more and better information specially on performance of the care system. Developments in macro-planning and delegation of care as well as coordination responsibilities increase needs for tools intended for management of care, including effective use of information of the care system. There is undoubtedly a need for new, broader and more refined data systems and tools for planning and evaluation of care, and for bringing inter-sectoral coordination between different care actors. There is also a growing need for comparative research in the field. What is the PLANEC-project? PLANEC is a cross-sectoral European research project, seeking to answer to the above information needs by designing, developing and demonstrating an intelligent information system for socio-economic monitoring, evaluation and planning of elder care. The intended functionality of PLANEC -application will enable its use as a decision support system for policy development as well as an analytical tool for scientific research. End-users of PLANEC are to be found amongst national, regional and local authorities, care providers, financing agencies (like care insurers) and researchers in the socio-economics of care. It is worth to note, that PLANEC will be a tool for system level care management, not for a client-level care tayloring. The need of this type of application has been investigated preliminarily in a COST A5 working group. (see Vaarama & Kaakinen 1994). The anticipated results of the PLANEC-project are (1) a verificated and demonstrated prototype application with relevant market analysis, (2) an analysis on needs, potential and future exploration of IT in the field of monitoring, evaluation and planning of elder care, and (3) key indicators (PLANEC-indicators) for European comparative research of care. The project is coordinated by Stakes (FIN). Other partners are the Institute for Applied Gerontology Free University of Amsterdam and the Netherlands Institute of Gerontology (NL), the Institute of Human Ageing at the University of Liverpool (UK), the Hospital of Mostoles (SP), WINLEAP (FIN) and SPSS-Benelux (NL). An effort to strengthen the European dimension in the research has been made by inviting Italy (the regional board of Reggio Emilia Romagna/Social services department), Germany (University of Bamberg/ Urbanistik und Socialplanung) and France (Ministere des Affaires Social et de l'Integration) to collaborate in the PLANEC-research on user needs and in validation of the prototype application. The project will be carried out during the years 1996-98, and it is mainly financed by the Telematics Application Programme of the IV Framework Programme of the European Commission. In Finland, a project similar to the PLANEC has been carried out by STAKES, resulting in the year 1990 to the first version of national system-level care planning application called EVERGREEN (Hämäläinen & Vaarama 1992, Vaarama 1993; 1995b). In the year 1996 the development of the second version with extended facilities for monitoring and evaluation of care will be completed (Vaarama & Kitinoja & Heino 1996). This EVERGREEN 2000 -application is indented to socio-economic monitoring, evaluation and modelling of welfare and health care for the elderly. The application has been developed in close collaboration with Finnish municipalities to ensure it's adaptability in the Finnish care policy system. Even if strongly adapted to the national care policy model, the EVEREGREEN has been a subject for a worldwide interest. In fact, the idea of the PLANEC-project has it's background in the interest shown in the first EVERGREEN-application by several European countries Thus, the EVERGREEN may be considered as a precursor of PLANEC. PLANEC-application as planned in the project plan PLANEC-application will have its focus on the whole continuum of care services for the elderly: public, private as well as voluntary domiciliary and community care forms, intermediate care forms, and different care forms in the institutional settings, as well as resources and costs of the care system. The application will enable socio-economic monitoring and evaluation of the existing care systems, as well as modelling of alternative strategies for care policy and their consequences (e.g. in the numbers of clients, the resources required for the services regarding personnel, funds and premises, as well as both total and per head expenditure). The key components of PLANEC will be (Figure 1): 1. a relational database containing data on demographics, socio-economics, demand and use of care services, service structure, resources and finance. Since these data are to be collected from various data sources dispersed among several organizations, facilities for electronic data interchange linked to the World Wide Web are included 2. modules for socio-economic monitoring and evaluation of the existing care systems with a variety of performance indexes addressing topics such as efficiency, effectiveness and economy of the services system 3. basic modules for design of alternative care models (e.g. a trend- extrapolation model and models for optimization of the balance between the demand, supply and resources of elderly care) Figure 1. A preliminary structure for the PLANEC -application PLANEC will run on a PC in a Windows-environment, thus enabling the use of the application in distributed computer systems. The core will be developed using standard Windows application development tools to ensure its usability throughout Europe. The Windows environment has many characteristics that support the concept of modularity and reusability of generic technology in the course of software development, like DDE (dynamic data exchange) and OLE (object linking and embedding) and DLL (dynamic link libraries). Generic technology will be used in order to support statistical analysis and calculations, and to achieve intuitive and easy-to-use functionality with sufficient user flexibility. Basic statistical functionality of PLANEC-application will be extended by calling functions from the world-wide used SPSS statistical package, which will strengthen the applications' usability for socio-economic research. The demonstrator will be developed as a workstation version with access to telecommunication services. These services will be tested in demonstrator stage by implementing a World Wide Web -server that can be accessed via Internet. The demonstrator will include all the functional and operational capacities necessary to verify and validate the application. PLANEC will be developed following a user driven approach, ensuring that key requirements such as user friendliness, flexibility and adaptability will be met. In every project country, a user group representative to their system of elder care will be established, and all the PLANEC-research will be performed in close collaboration with these groups. The concept of care in the PLANEC -research Before going further, the concept of care in the PLANEC -research has to be clarified. Theoretically the term "care" seems to be somewhat weak and confusing, (see e.g. Alber 1993). In general, there is a distinction to make between care and cure. "Cure" refers to acute treatment, while "care" refers usually to action aiming to support elderly's lowered ability of function in their daily living. This support and care can be given at home or in the institutional settings by formal of informal care givers. (see alsoVaarama 1995a). In the European welfare states, there exists different models of care, where public and private, formal and informal care actors have varying roles in the care markets. The core services in the elder care are various extramural and intramural services provided by welfare and health authorities, families, volunteers, non-profit and for-profit organizations. Thus, the elder care is a multi-actoral function, involving many ideologies, goals and intentions of both formal and informal actors (see e.g. Gross 1991). By formal care we mean care services provided either by public authorities or private service providers. Also paid care at home may be classified to be somewhat "formal". Informal care forms include various activities performed by families, neighbours and other volunteers. The financing of care is usually a combination of public funds and client's co-payments, but the structures of this combination can vary a lot. It may be expected, that in the greying European countries the public sector will have an increasing role in maintaining the adequate care system to meet the growing needs of ageing population. (e.g. Pacolet & Wilderom 1991, Gaunt 1992). Not only the mix of public and private provision and financing of care vary, but also the models of organization, collaboration and coordination of health and welfare services in elder care differ among European countries. E.g. in Finland, there is an increasing trend to get these two sectors collaborating closely together to improve performance of the care system. Research findings in the country also suggests, that from the economic point of view specially in the elderly care it is essential to look at these two sectors simultaneously when evaluating the performance of public care. (e.g. Häkkinen & Luoma 1994, Vaarama 1995a). However, in most European countries, health and welfare services for the elderly seems to function quite sectorozised, and often there are no formal coordination structures between these sectors. Also the status of welfare and health sectors may be different in different countries, and professional boundaries between the health and welfare sectors may be high. In the PLANEC-project, it is essential to be aware, that this complexity brings about many challenges and practical restrictions to an effort to develop a trans-European software application aiming to cover both welfare and health care services. Therefore, an essential issue is to carefully define the territory of the PLANEC-system. According to the project plan, the PLANEC-application should cover the whole continuum of care. Here the term "care" is interpreted to include those welfare and health care services, which are given occasionally or in long term to the elderly to support their living at home or to arrange their care in long term in the institutional settings. (cf. Vaarama 1995a). Therefore, acute health services consumed by the elderly will not be included in any detailed manner in the application. However, to be able to monitor the total consumption of services and total costs of the elderly care system, as well as the care's share of the total, some indicators of acute care consumption are also needed. For this reason also acute clinical calls and short term inpatient care should be included in the system at least in terms of numbers of totals both in primary and in spezialised health care. Because the organization of care is varying a lot among European countries, this demand may be difficult to meet in every country. To solve this problem it is important to aim to an application flexible enough to be used independently both in welfare and in health sectors, but also in a joint entity or in coordinates of them. Concepts of monitoring, planning and evaluation in the Planec -research Because the aim of the PLANEC-research is to develop a tool for planning, socio-economic monitoring and evaluation of care with an opportunity to seek balance of care by matching needs to resources, the theoretical starting points for the PLANEC-research are theories of policy planning and economic evaluation. There exists a great variety of literature concerning evaluation and planning of welfare. Monitoring is a somewhat weaker concept, and often it is seen as a part of evaluation process. To simplify, the process of evaluation includes (1) planning, where the evaluation criteria (goals) are given, (2) monitoring, which may be described as follow-up of the ongoing developments and (3) evaluation, where the extent to which the goals have been achieved is measured. (e.g. Ball & Bernard 1987, Sixsmith 1993, Phillips & al. 1994, Vaarama 1995a). In general, 'evaluation' is concerned with judging merit against some yardstick. It involves the collection, analysis and interpretation of data bearing on the achievement of an organisation's goals and programme objectives. In the evaluation, particularly in the health care sector, attention has been paid to service inputs, throughputs and outputs. These are quantifiable key indicators of performance and provide evidence of the extent to which specified targets are being met. Monitoring, involving the systematic and continuous surveillance of a series of events, is more concerned with what is going on. Also in monitoring, key indicators are essential. E.g. services profile covering the whole supply of services and economy of them serves as an example of a useful measure to services monitoring. (see Vaarama 1995a). In many cases, a combination of monitoring and evaluation will provide a much fuller and more sensitive analysis of a developing situation. (see e.g. Phillips & al. 1994). With regard to planning of welfare, there are a variety of approaches to implement. (e.g. Lindblom 1959, Gilbert & Specht 1977, Mooney 1978, Hill 1993). From the PLANEC-perspective, it is useful to understand the concept of corporate planning. For several decades, local authorities and other organisations have been urged to adopt a more rational and corporate approach to planning and evaluating their services. The kind of corporate planning which has been advocated has the following features (Phillips & al 1994): 1. the determination of goals and objectives - in relation to various client groups and their needs/problems; 2. the evaluation of the alternative means costs, benefits, and distribution thereof of achieving the objectives 3. the selection of the most promising means 4. the setting of specific targets 5. implementation 6. evaluation and review. In the PLANEC-research is essential to understand the linkages between planning, monitoring and evaluation. Taken together, they are all parts of a comprehensive process of policy evaluation. (see e.g. Sixsmith 1993). In the framework of the PLANEC-research we can summarize, that (1) monitoring means continuous follow-up of the developments in the care sector to get early signs of serious changes compared to what is seen desirable (2) evaluation of elder care means performance evaluation, where the evaluation criteria (yardstick), the evidence (data on achievements) and key indicators of performance (measures) are needed. While encapsulating these dimensions to one theoretical framework, and offering advanced measures for evaluation, the production of welfare -model (see Davies and Knapp, 1981;1988, Knapp, 1984;1995) offers an useful approach to implement in the PLANEC-research. The production of welfare -approach The production of welfare approach is a summary of the complex linkages between policy goals and practices, services and achievements and the (material) resource and (immaterial) non-resource inputs that make them possible. (Knapp 1984;1995). Because the PLANEC-application is intended to be a macro level care management tool, the immaterial dimension of production of welfare approach cannot be fully implemented in the theoretical model of the application development. However, the approach gives a framework for distinguishing the key elements in the welfare and health services production, and a framework on which to hang an evaluation of the performance of the care system, which is the intended core task of the PLANEC-application. Figure 2 presents an example of how the production of welfare approach can be used in system level performance evaluation of public financed care services . (Vaarama1995, 82, see also Drummond & al. 1987). Figure 2. A model of performance evaluation for the public financed care system First, the model describes the basic elements of production of welfare in public (financed) care system. Second, it describes the linkages between policy goals, welfare production and performance evaluation. Third, the model clarifies the concept of efficiency and levels of efficiency measuring. In the figure, efficiency is defined as a function of effectiveness, productivity and economy. Depending on the measures used, effectiveness can be defined either in terms of changes in client's well-beeing ("needs satisfaction") or in terms of goal achievement. In the system level research, the concept of effectiveness is usually considered as the degree of success the care services system has in achieving the (planned) policy goals given to it. (cf. Goodman & Pennings, 1980). Even if Vaarama uses the concept of performance as a synonym to the concept of efficiency, she suggests not to use the concept of efficiency evaluation if intermediate outputs (services) and not the final outputs (changes in well-being of a client) are used as output measures. Based on Knapp's (1984) argumentation, she suggests in this case the use of the concept of performance evaluation, because intermediate outputs tends to measure the performance of the care system rather than the effects of the given services to the client's well-beeing. Research to be undertaken in the PLANEC -project From the above discussion, the following research objectives for the PLANEC -research can be concluded. The research aims to: 1. describe and analyse the terminology of elder care in various project countries; to create a common conceptual framework from these; and to develop socio-economic indicators for elder care (PLANEC-indicators); 2. describe and analyse the tasks and interrelationships between various care actors in provision and financing of elder care, as well as in processes of monitoring, evaluation, planning and coordinates of these; to describe and analyse the information use and needs of various care actors related to monitoring, evaluation and strategic planning (MEP) of elder care at macro (national, regional, local, distributed local) level; and to translate these into system requirements for application development; 3. to design, develop, verificate and validate the prototype application in an iterative process with users; 4. to describe and analyse both retrospectively and prospectively the information technology potential in the field of elder care; 5. to perform market analyses to assess potential markets for PLANEC and to support the marketing of the application. There are two types of research in the PLANEC -project: 1) research directly supporting the practical application development and 2) research supporting the usability of the application (figure 3). Figure 3. The PLANEC-research The research directly supporting the application development emphasises different dimensions of user needs, their translation to system requirements and to a model of the system to be designed, creation of a common conceptual framework and the PLANEC-indicators, as well as design, development and validation of the prototype application. The research intended to support the usability of the application focuses on three topics. First, information technology potential related to monitoring, evaluation and planning of care will be analysed. Second, an exploration on future of elder care will be performed. Third, to create market awareness, through the development process market analyses will be performed. The approach of iterative application development The goal of the PLANEC-project is to develop a marketable software application. Therefore, the development of the PLANEC-application is based on the methodology of iterative process for design, testing and evaluation of software applications, involving the users, researchers and software engineers. The essence of this approach is progressing of the development in a spiral of requirements, analysis, design and implementation. Requirements will be changed as the project evolves, because expectations change by seeing what is achievable.(Figure 4.). Figure 4. Model of iterative process of development in the PLANEC-project In this process end users will evaluate the engineering work and make suggestions for modifications. Based on this input, the next phase of planning and risk analysis occurs. The culmination of risk analysis results a decision whether or not to implement the suggested modifications. Object-oriented analysis (OOA) For the complex systems like the development of the PLANEC-application, it is crucial to get a wide and right understanding of the problem domain to be able to analyse the system's requirements in that light. For this, there are various methodologies like functional decomposition, data flows, and information modeling, which has been discussed and practiced in the systems development profession for over a decade. Nowadays the object-oriented analysis (OOA) and design (OOD) has been applied more and more in the application development processes. From these, the system analysis based functional decomposition and object-oriented analysis are the major alternatives for methods to be used in the development of the PLANEC -system. The methodology of functional decomposition has been applied in the late 1980'ies e.g. in the Finnish EVERGREEN -research. However, according to Coad & Yourdon (1991,19) the OOA is a more comprehensive, all-encompassing method, incorporating the best ideas of the three above mentioned methods. The basic idea of the object-oriented analysis is, that the process of extracting the "needs" of a system have to deal with what the system must do to satisfy the client, not how the system will be implemented. Here the systems analysis usually begins with a requesting document (from the client, from the marketing divisions etc.), and a series of discussions. In any case, the audience includes clients, problem domain experts, developers, and possibly other interested parties (e.g., auditors, contracting officers, etc.), who all may need to understand and agree with the proposed set of requirements. The requirements document should communicate a complete, consistent, and feasible statement of what is needed in the system. Requirements include functional operations and quantified operational characteristics such as ease of use, reliability, availability, maintainability, and performance. Requirements also include interfaces that the software must deal with, environments the software must accommodate, and any other applicable design constraints. To summarize, a requirements document has two purposes: (1) it formalizes the needs of the client, and (2) it establishes a list of mandates. The strength of the OOA is, that it maps the problem domain and system responsibility directly into a model. Using the methodology of OOA , the PLANEC-research aims to manage the complexity of elderly care action. Because the PLANEC-application should be able to function in highly varying care environments, understanding of these different worlds of care is crucial. The PLANEC-researchers will analyse the data generated by national studies, and translate their results into a object-model, where functional, analytical and modelling requirements of the PLANEC-system are defined. The process then continues as a process of object-oriented design (OOD) and programming, led by software engineers and performed in a dialogue with researchers and users, resulting to the prototype application to be again tested by the users. This methodology is called as iterative design, development and evaluation method. (see University of Maryland, 1991, Vaarama 1995b). Because the core task of the PLANEC-application is socio-economic evaluation and planning of the care system for the elderly, the OOA in the PLANEC-research will focus on the basic elements of welfare production (policy goals, inputs, production process, outputs, costs) and the care markets (customers/demand, service providers/supply, service financiers/inputs), regulation of the care markets and relationships between these. Using the OOA, the PLANEC-research aims to (1) identify and differentiate the subjects (client, provider, financier) of care markets; (2) their attributes (need, intermediate outputs, costs); (3) the tasks related to the MEP of care, and (4) the relationships between them. The goal of analysis is to fully specify the problem and the application domain in iterative process of development, driven by user research.. The structure of the analysis is described in figure 5. Figure 5. The structure of object-oriented analysis (Lunn 1995) Methodological triangulation and research dependencies in the PLANEC-research In the framework of the method of iterative design, development and evaluation of software applications, in the PLANEC-research a variety of methodological approaches will be applied. The research will mainly be performed by desktop researchers in a close dialogue with the national user-group members. In the inventory study of user needs the methodologies are mainly descriptive ones: content analysis of elderly policy documents and legislation, user interviews and self-completing forms, analysis of national surveys and discussions in the user-sessions. To standardize the country analyses and to ensure collection of a good information base for creation of a preliminary object-model for monitoring, evaluation and planning (MEP) of care, in this part of study the researchers will use a common investigative framework. To standardize the information to be collected from the users, an user-questionnary will be used. Research on IT-potential will be based on methodology of SWOT-analysis. An explorative study on other future changes of the care environment will be carried out by means of a scenario delphi. The inventory study on user needs will give an input to functional specifications of the PLANEC -system. Here the object-oriented analysis (OOA) will be applied to specify the system requirements as an object model of MEP of elder care. This model again gives an input to system design. The OOA will be followed by development of the PLANEC-indicators, based on the theoretical model of production of welfare -approach to meet the requirements of performance evaluation of a system. In analysing of the object model, existing OOA-tools will be used to enable an efficient and reliable analysis of system requirements. Choosing e.g. SELECT OMT Professional would be sensible, because it is technically a subset of SELECT Enterprise, the object-orientated designing tool used in designing of the PLANEC -system. Use of SELECT OMT Professional in OOA would benefit the system design phase in an effective way particularly because it enables transferring of descriptions made in OOA directly to SELECT Enterprise. SELECT OMT Professional offers support for Object Modeling Technique (OMT) and Jacobson's Use Case methods. It incorporates the OMT and Use Case modeling techniques with a multi-tier client/server architecture and an iterative, incremental development process. Diagrams can be transferred to other programs like word-processor through clipboard or they can be exported to Windows metafile -format. In the system design phase, the object-orientated designing tool SELECT Enterprise will be used. According to the project plan, the PLANEC-application will be based on a relational database, and on use of generic technology. System design and application development will be performed in an controlled iterative process with users, which means that the results of system design and development will be continuously evaluated by the users, and their feed-back will be taken into account in further development. At the end, the prototype application will be verificated and validated. First the validation process involves the national user groups of PLANEC-project, but at the end the prototype application will be demonstrated in a larger group of users than has been involved in the application development. Here the methodology is based on the above mentioned model of iterative evaluation of a software application. The user feedback will be documented, their experienced researched by a standardized questionnaire, and all the results will be reported. In addition, market research will be performed to investigate pontential markets for PLANEC -application. Beside using the policy documents concerning the care of the elderly, the data used in the PLANEC -research will mainly be generated in the iterative design, development and evaluation process with the potential end-users of the PLANEC-application in the project countries. Complementary data and validation feed-back will be collected from France, Germany and Italy by collaborating with potential end-users in these countries. The data will be collected by questionnaires and structured user- interviews, in the user feed-back sessions, from statistics and from previous research. REFERENCES Alber, J. (1993). Health and Social Services. Teoksessa Walker, A. & Alber, J. Guillemard, A-M. Older People in Europe: Social Economic Policies. The 1993 Report of the European Observatory. Commission of the European Communities, 100 - 131. Ball, M. and Bernard, M. (1987). "Monitoring as a way to improve our practice". In M. Bernard (Ed.). Developing Services fo the Elderly Mentally Ill. Beth Johnson Foundation, Stoke. Coad, P. & Yourdon, E. (1991). Object Oriented Analysis. Second edition. New Jersey. Davies, B. & Knapp, M. (1981). Old People's Homes and the Production of Welfare. Routledge & Kegan Paul. London. Davies, B. & Knapp, M. (1988). Introduction. The Production of Welfare Approach: Some New PSSRU Argument and Results. Teoksessa Davies, B. & Knapp, M. The Production of Welfare Approach: Evidence and argument from the PSSRU. The British Journal of Social Work. Vol. 18. Supplement. University Printing House. Oxford, 1 - 13. Drummond, M. F. & Stoddart, G. L. & Torrance, G. W. (1987). Methods for the Economic Evaluation of Health Care Programmes. Oxford Medical Publications. University Press. Oxford. Gaunt, D. (ed.). (1992). Shifting Welfare Mixes in the Social Service Sector - Consequences for Citizenship Rights of the Elderly, of Persons with Disabilities and their Carers. International Expert Meeting Stockholm, Sweden 3 - 5 April 1992. Conference Report. European Centre for Social Welfare Policy and Research. Vienna. Gilbert, N. & Specht, H. (eds.). (1977). Planning for Social Welfare. New Jersey. Goodman, P. & Pennings, J. (1980). Critical issues. Teoksessa Lawler III, E. & Nadler, D. & Cammann, C. (eds.) Organizational Assesment: Perspectives on Measurement of Organizational Behaviour and the Quality of Work Life. John Wiley & Sons. New York. Gross, A. M. (1991). Israel. Teoksessa Evers, A. & Svetlik, I. New Welfare Mixes in Care for the Elderly. Eurosocial Reports Volume 40(2) European Centre for Social Welfare Policy and Research. Vienna, 83 - 96. Hill, M. (ed.). (1993). The Policy Process. A Reader. Harvester Wheatsheaf. Hertfordshire Hämäläinen, H. & Vaarama, M. (1992). EVERGREEN- a microcomputer simulation in the planning of services for the elderly. In Bouma, H. & Graafmans, J.A.M. (1992). Gerontechnology. IOS Press. Amsterdam, 431-433. Häkkinen, U. & Luoma, K. (1994). Determinants of Expenditure Variation in Health Care and Care of the Elderly Among Finnish Municipalities. STAKES. Themes 12. Helsinki. Jamieson, A. (ed.). (1991). Home Care for Older People in Europe: A comparison of policies and practices. Oxford University Press, Oxford. Knapp, M. (1984). The Economics of Social Care. Studies in Social Policy. MacMillan Education. Hong Kong. Reprinted 1986. Knapp, M. (1995). The Economic Evaluation of Mental Health Care. Hartnolls Limited, Great Britain. Lindblom, C. L. (1959). The science of "muddling through". Administration Review 19 (2), 79 - 88. Lunn, K. (1995). Object Modelling. Training Course. Shell Research Ltd, Liverpool 1995. Mooney, G. H. (1978). Planning for Balance of Care of the Elderly. Scottish Journal of Political Economy. Vol. 25 (2), 149 - 164. Munday, B. & Ely, P. (1995). Social Care in Europe. Prentice Hall. Harvester Wheatsheaf, Hertfordshire. Nijkamp, P. & Pacolet, J. & Spinnewyn, H. & Wilderom, C. & Winters, S. (eds.) (1991). Proceeding of a seminar on National Diversity and European Trends in Services for the Elderly. Katholieke Universiteit Leuven and Vrije Universiteit Amsterdam. Commissie van de europese gemeenschappen. Leuven. Pacolet, J. and Wilderom, C. (eds.). (1991). The Economics of Care of the Elderly. Avebury. Aldershot. Phillips, C., Palfrey, C. and Thomas, P. (1994). Evaluating Health and Social Care. The MacMillan Press Ltd, London 1994. Sixsmith, A. (1993). The Importance and Methodology of Evaluation. Elderly Care a World Perspective. Charmont Hall, London 1993. University of Maryland. (1991). The long form of a generic user-evaluation questionnaire for interactive systems. Human-Computer Interaction Laboratory, University of Maryland. Vaarama, M. (1993). EVERGREEN - a software for planning welfare and health services for the elderly. A paper presented in the Wordl Conference of Technology in Human Services, HUSITA3. Maastricht, June 15-18, 1993. Vaarama, M. (1995a). Vanhusten hoivapalvelujen tuloksellisuus hyvinvoinnin tuotanto -näkökulmasta. Performance in the Care of the Elderly. A production of welfare - analysis of the public care system in Finnish municipalities. Stakes, tutkimuksia 55. Gummerus, Jyväskylä 1995. Vaarama, M. (1995b). Information Technology Use in Search for Cost-effective Care Policy for the Elderly - a Finnish Example. A paper presented in the III European Congress of Gerontology. Amsterdam, August 30 - September 2, 1995. Vaarama, M. & Kaakinen, J. (eds.). (1994). The Need, Requirements and Contents of the European Database on the Elderly Policy /Services. Proceedings of the workshop held in Helsinki 2 - 4 March 1994. COST Social Sciences, Action A5 Ageing and Technology. European Commission. Vaarama, M. & Kitinoja, A-P. & Heino, L. & Kandelberg, K. (1996). Evergreen 2000. User Manual for version 1.0. Stakes. Helsinki 1996. Walker, A. & Alber, J. Guillemard, A-M. (1993). Older People in Europe: Social and Economic Policies. The 1993 Report of the European Community Observatory. Commission of the European Communities. 17