Web Tool for Health Insurance Design by Small Groups:
Usability Study
Laurie Kantner, Tec-Ed, Inc., USA
Marion Danis, National Institutes of Health, USA
Susan Dorr Goold, Mike Nowak, Lesa Monroe-Gatrell,
University of Michigan, USA
The authors describe iterative usability evaluation of a
web-based collaborative health insurance benefits planning
application, which was developed by the U.S. National
Institutes of Health and the University of Michigan.

Applying Contextual Design to ERP System Implementation
Inka Vilpola, Kaisa Våånånen-Vainio-Mattila, Taru Salmimaa,
Institute of Human-Centered Technology, Finland
Enterprise resource planning systems (ERP) affect the daily
work of millions of users. Applying Contextual Design
introduces the missing user-centered approach in ERP
implementations and improves the system success.

Making Oracle Behave
Sofie Vanophem, Kris Vanstappen, The Human Interface
Group, Belgium
The authors describe how usability consultants worked
with developers using Oracle Designer to build an ERP tool
for temporary agencies, creating a custom user interface
for 7,000 end-users familiar with Windows, not Oracle
applications.

---

Web Tool for Health Insurance Design by Small Groups: Usability Study - Laurie Kantner, Marion Danis, Susan Dorr Goold, Mike Nowak, Lesa Monroe-Gatrell

Application visualization to help people select health coverage. There are over 103 potential points/health coverage people can decide on.

Methodology Issues Question: How to simulate real use?
- In Lab or out of lab testing?
- In Person (lab or home)
- Remote Testing (moderator on phone)

Study

- 3 groups of 9 people
- Diversity within groups
- Observation of all Round 1 sessions, half of Round 2 and 3 sessions; No Round 4 sessions because they were like Round 1
- Think a loud and read a loud protocol

Concerns about Attrition

- Expensive to recruit >9 per group
- Must observe all sessions
- Incentives by paying people after specific rounds so they had to hang in there to get all the money (to battle attrition)
- "Failure Remediation" - give participant progressive hints when they fail

Observation Issues

During round 2 and 3 the debated about who to observe. Sometimes they did it in the lab and sometimes not. But not bringing everyone in to user test, they kept people from dropping out of the study.

Findings

Discussion area of website was still not as interactive as having someone in the same room with you. There has to be some real world use of this product to figure out how to implement this.

Questions

1. You used a dart board method. Did you find any comments about that?

Some people wanted an arrow that said start here, but that was the only thing.

2. [Indiana University] Did you have smaller user group sizes because of resources or because In the Wild/Reality Testing requires relationships to develop with participants that limit the number of users as the Reality Testing workshop found?

It was mostly resources, but we did have good relationships with the participants.

---

Applying Contextual Design to ERP System Implementation - Inka Vilpola, Kaisa Våånånen-Vainio-Mattila, Taru Salmimaa

ERP System Implementation does not mean design.

Findings

Apply CD in ERP systems implementation is a step towards user-centrednessof COTS product implementations.

Questions

1. What was the effectiveness of CD as opposed to not doing it?

They liked the affinity diagramming. It was good to see what the system can and cannot do. It helps for potential enhancements of the system for work processes.

2. [Economical Insurance] How did the business analysists in the company deal with you coming in and analyzing their practices?

They were relieves (we don't have to do this ourselves). They were medium sized companies. They were afraid of doing the project themselves - huge amount of time and money. They were happy to have us with them.

[How much knowledge did you have of the product and subject matter?]

This is not a product specific method. But many principles of these systems require an expert that knows the products.

3. This is not different than CD. It is not just about particular tasks. Can you do a quick explanation of sequence models and how you changed them?

Since we observed people in different places of a process - we have different sequence models. He or she has multiple tasks. At this stage, we have models nearly every sequence with various sequences. Now we see that they cannot be concatenated. What we do is take the biggest problems in each sequences and evaluate them with the workers to see how they do this with the ERP system.

---

Making Oracle Behave - Sofie Vanophem, Kris Vanstappen

Building and ERP tool for Oracle.

Unusual Problem: had to counteract standard Oracle interface design guidelines

Goal: Create a custom front end for dozen of headquarter employees

Challenges: Distracted (Constantly on the phone), Interrupted continuously participants have to quickly navigate through files.

Lessons Learned
- Need some "partners in crime" from your user group who is willing to talk, listen, and discuss with you
- Check and double check new builds of the application to get all the bugs out
- LIve with some of the peculiaraities that cannot be rootd out

Questions

1. [Oracle] The program you showed is about 10 years old and it is not being shipped with our current stuff. So the gross [Audience - 20 years old...I'm the VP of HCI] implications you made about quality are borderline unprofessional. I do not see any reason why you are ridiculing it without hitting eBay or Yahoo.

We have shown this case before and they saw it as an opportunity that their tools are flexible enough to build all types of systems around it.

[Audience - [Oracle person] we do not have design teams in Holland. Only in US. What you showed is a default program. We put Java skins over it so it is nicer. I don't know why you are showing this.]

We visited SAP, Peoplesoft, Oracle, and such. We asked the if it was possible to do this and all the information I have is what the people told me.

I'm a usabiilty designer.

[We use jBuilder. This is a case tool over 20 years old. It has been discountinued for 5 years. Why are you showing this!?]

Within the boundaries of what we are told. We have to build something the users can use.

[Moderator asks to take it off line]

2. [SAP] How did your firm get this tool? Did they buy it from Oracle?

Yes, they had a completely different application before Oracle came in.

[Well I just wanted to say that SAP has tools that could do with a tad redesign. So it is nice to see someone reviewing tools. It is good work.]

3. [Manchester] Rescue you from the Oracle guys, not withstanding whatever version this is. The problem has not gone away. We are looking at ARPs from Peoplesoft and we are having the same problems to customize the look and feel. There is always this problem - large scale configuration, despite skins - I want to know what did this do to the bottom line of the project? Who ended up with the bill? You or Oracle?

The Partners in Crime were not from Oracle. They were CEOs who know if there were several thousands of people using this application, they would have to invest in training and user support. So we had prototyping up front to build this kind of customs screens and what it would cost. The CEO agreed with it because what would be the bill if they did not do it?

Management layers are good to have as partners in crime. We had to results from user testing to prove it. We did not get it for free.

[There are interesting points how you brought design guidelines into configuration.]

4. [Oracle] I agree with you. But you have mentioned Oracle. CHI isn't about this. We don't see any sessions on SAP, Google. As soon as you mention companies here - it gives us a good or bad name. We should not put company names on the screen. It gives partisan discussions. You should patch out the company logo. That is not what CHI is about!

5. [SAP] We can all stand to have a thicker skin. I would like to see something about SAP up here and defend what we are doing to improve it. We should not personally attack the speaker.

6. CHI does connect with industry and you cannot really discuss real experience without naming names. People will still know if you blank out names because it looks like so and so. Look at it as productive learning. Let's keep the engagement going. Academia gets hit for not being with companies...and here companies are hitting researchers.

7. [Moderator] You should look at a paper that talks about, "How to tell you if your baby is ugly."

---