Lessons learned from expertise

Aspects to Handle in Evaluations and user-involvement. Developing services for private users to co-develop in their own private homes during their spare-time, 24/7, calls for new circumstances that need to be considered when planning and determining the focus for the intervention. The conditions, which we have identified and will discuss below, are based on our experiences from working in projects facing these limitations (Ståhlbröst 2006).

Here we presente some of the aspects which are important to handle:

• What the approach and purpose for the evaluation is.
• What the main question that needs to be answered is.
• Should the evaluation be of summative, formative, or interactive character
• What kind of data is expected from the evaluation, qualitative or quantitative
• Define what methods to use in the evaluation; interviews, logs, surveys, observations in relation to the purpose
• Study the context to determine what, in the context, that can have influence on the evaluation results
• Create questions, observation schemas or other preconditions for the evaluation- develop questions on the basis of the identified needs and requirements
• Define the number of users and selection criterions such as age, gender, occupation, and so forth
• Define what the characteristics of the innovation are. Identify important “has to” in the evaluation, such as; when the test has to be done, the duration of the test, the character of the interaction, what a natural behaviour around the innovation is, degree of participation and so forth
• Consider how to stimulate and encourage the test-pilots to use the service/products during the determined time in as natural manner as possible. In addition, aim to create a natural usage that gives the test participants a good notion about the evaluand.