online feedback

Project update: What’s new in May?

by admin on May 22, 2011

It’s been a few weeks since our second expert workshop, so here comes a brief update on what has happened since:

  • As you might have noticed, the announcement for the one-day conference on 28 June is out. We have a stellar line-up of speakers, including Malcom Ashmore, Andrew Balmer, Alex Wilkie, Ian Stronach and Stefan Schwarzkopf. While they all come from academic backgrounds, they promise to give some interesting (and, I have been told, entertaining) feedback on the project and issues of online evaluation more generally. If you would like to participate, please register soon. The event is free, but places are limited.
  • We also have been around a bit to talk about the project. Two occasions have been particularly interesting. On 19 April 2011, I participated in a panel discussion at the Internet Freedom Conference in Strasbourg organised by the Council of Europe. ‘Multistakeholderism’ is a popular idea in this context, and the group was particularly interested in the potential of web-based reviews and ratings for fostering participation and engagement in policy-making. The video of Panel 6 is still online.
  • Harvard-MIT-Yale Cyberscholar Working GroupOn a very different occasion, I presented the project at the Harvard-MIT-Yale Cyberscholar Working Group at the MIT Media Lab. This was a great opportunity to get feedback from a very diverse crowd of people, including media designers, HCI researchers, lawyers and social scientists. There was also a second presentation by Nick Bramble, which very nicely highlighted the important legal issue of third-party liability for content posted on review and rating websites.
  • Of course, we have also been working on the prototype. It has been far from easy, given the shoestring budget and tight timeframe we are on. However, while things are moving slowly, they are moving and we hope to have something to tinker with soon. If you think you can contribute anything to the process from design skills to a developer brain, it’s not too late.
  • Finally, a lot of people got in touch and offered their support or simply showed interest in the project. In this context, have a look at other initiatives, such as Eric Goldman’s and Jason Schultz’s new project Doctored Reviews that aims to help people deal with restrictions on online patient reviews.

More updates soon. Again, don’t forget to register for the conference.


Here are some quick impressions from yesterday’s second expert workshop. Again, we met in London at the offices of Which? — this time to dicuss and imagine the prototype we are supposed to built over the next couple of weeks.

Building on insights from the first expert workshop, we focused on a number of rapid design exercises. We split up into two groups and equipped ourselves with pens, paper, flipcharts and sticky notes. Under James’ guidance, we started by devising personas, i.e. concrete individuals who might use How’s my feedback?, and then sketched their journeys to and through the website. This also led us to consider some of the functionalities of the prototype and rethink its scope and purpose.

A cup of pens. Jonathan and Stefan tracing user journeys. Dixon making a point.

As expected, the design process was much messier, but also more interesting than any textbook could have taught. A few examples:

  • Working with constraints: Although it was tempting to assume a world of unlimited resources, we constantly had to remind ourselves of the shoestring budget we are on. For example, why not negotiate cooperations with major operators of feedback platforms? Why not devise an algorithm that would crawl, collect and crunch transactional data to arrive at useful feedback scores for specific platforms? Or why not have an independent team of researchers explore and evaluate the schemes according to a set of universal principles? While all these ideas seemed great, we would hardly be able to pursue any of them with the time and resources at our disposal. So we were forced to think creatively about less expensive alternatives.
  • Increasing ambiguity: Another interesting phenomenon we encountered can be described in terms of Mackenzie’s certainty trough, a schematic representation of how certainty about an established technology might be distributed. Put simply, what looks like a clear case from a reasonable distance is getting more and more uncertain and ambiguous the deeper you become involved. This happened — among other things — when we dived into the website and its possible uses and features. When thinking about ‘How’s my feedback?’ in terms of quite specific situations, we sometimes feared to loose focus and needed to remind ourselves of the reasons we set out to do this.
  • Configuring users: Designing a website requires a lot of imagination. Specifically, imagining individual users turned out to be more difficult than expected. We often slipped back into talking about what ‘users’ or ‘consumers’ generally want, how smart or dumb they are and what problems they have. Built into these claims were a number of rather strong assumptions that did well in strengthening our respective arguments, but less so in helping us think from the perspective of ‘actual’ users. So thank you very much to the imaginary “Caroline (57)”, “John (35)” and “Fantom (43)” for keeping us on track.
  • Steering clear of ‘ideal types’: Another challenge was to steer clear of ideal conceptions of existing schemes and use them as the only reference point for ‘How’s my feedback?’. For example, we often found ourselves inadvertently referring to a ‘Tripadvisor-like’ system, even though the challenge is certainly much broader. This also brought up the critical question of the target of the scheme: what kind of object is ‘feedback’ and how can it usefully be assessed?

Despite (or perhaps because of) these difficulties, we managed to explore a range of cases, uses and functionalities. Among other things, we discovered new aspects like “the potential to cause trouble” or the possibility of becoming a clearinghouse that might help inquiry and research more than decision-making. It is still a long way to go, but discussing the issues with a terrific group of experts is a more than worthwhile (and fun!) part of the journey.

Many thanks to everyone who contributed, especially Lorraine Aziz, Melanie Dowding, Peter Harris, Chris Emmins, Kirsten Guthrie, Helen Hancox, Dixon Jones, Noortje Marres, Stefan Schwarzkopf, Paul Smith, Marcus Taylor, Esther Vicente, Jonathan Wolf.