Reflections on usability testing at the SAP BusinessObjects User Conference

November 04 1 Comment Category: User Experience, Xcelsius

This post has been moved to my new blog, Uncharted. Please continue the discussion there.

It’s been a couple of weeks since our round of usability tests at the SAP BusinessObjects User Conference (rolls right off the tongue, doesn’t it?) and I wanted to share a couple of thoughts about the experience. This was the first time we did usability tests since I joined the team back in March and it was also my first time meeting our community of users in person.

The first thing that struck me was the willingness of people who use Xcelsius to come and participate in the testing sessions. There were 13 sessions total, with room for one tester and one observer, and we had most of them filled before the conference even started. Our recruiters said they had people practically busting down the doors, trying to get into the sessions. This kind of support is exactly what we need in order to improve our product. For those who couldn’t make it, this is what it looked like:

Xcelsius usability testing at SAP BusinessObjects User Conference

The second thing that really impressed me was the quality of feedback we received. I have to admit, I was a little nervous going into these sessions because I know a lot of our users get frustrated with some of Xcelsius’ shortcomings. However, everyone that participated was very supportive and provided great constructive feedback. We learned a lot from watching people use the product – some of which we expected, and some that took us by surprise. Spending 13 hours looking at Xcelsius through the eyes of our customers was a very enlightening experience.

Finally, the thing I enjoyed most was meeting everyone in the Xcelsius user community in person. I follow many discussions online (including on LinkedIn, SDN, and Twitter) and it was very exciting to be able to put a face to the names of people I have come to respect through these discussions. I tried to meet as many people as I could, through lunches, networking events, the Xcelsius Community event, and the usability testing sessions. Through all of this I learned a lot about what people do with Xcelsius and their top concerns. There were two questions that I heard more than any others, though, and I’ll leave you with the answers here:

  1. Q: When is the next version of Xcelsius coming out?

    A: Unfortunately, I can’t comment on that. All I can say is, “soon.”

  2. Q: Are you related to Matt Lloyd?

    A: You know how some people are related by marriage? Well, Matt and I are related by product. Beyond that, there is no relation. It’s just an awesome last name.

Also, if you want to see photos from the Xcelsius Community event, you can find them on Flickr.

This post has been moved to my new blog, Uncharted. Please continue the discussion there.


  1. Usability testing at conferences « Interfaces - December 8, 2009

    [...] Some more trawling around on the web, and I found another useful article on the subject of carrying out testing at a conference. [...]