Using Feedback Army for Efficiency and Insight

I learned about Feedback Army when it launched a few years ago, and enjoyed it a whole lot then with Holono.  It’s a service to send off questions you may have on your website and get customer feedback.  Feedback Army works by sending tasks to Mechanical Turk, so essentially is a pretty wrapper, but I’ve found it to be a useful one.  The cost is not that much more than that on Mechanical Turk, and because Mechanical Turk has such a bad UI, it can actually make some sense. Image Previously in user testing of Raise people liked the website but were very confused by our message.  We’re in the middle of changing our landing page, so decided to use Feedback Army to get 25 responses on what people thought of our site as is.  We used the service to ask each person the following questions: 1. What does this site do? 2. What aspect of the site confused you? 3. What would improve this site? 4. How does it work? 5. If you were to come up with a slogan for the website, what would it be? The responses were much more positive than we were expecting, with most grasping the idea of the website.  Of course, much of this may well have been because the reviewers didn’t want to spend much time on it and “it’s good” is an shorter answer than “it’s confusing…”   There was still a bit of confusion, some bug finds, and some decent suggestions.  A few select quotes below:

  • “The only thing that would need improving would be to make any text that is in a light gray color more darker for those of us who can’t see.”
  • “It was not clear what the site does until I scrolled down a little and read the description.”
  • “On the /features/ page, the line “Each month we will send out checks with that month’s revenue to each cause in the amounts mentioned on the side.”, it’s somewhat unclear what “…amounts mentioned on the side” means. My immediate reaction was to look at the sidebar of the page I was currently on. It should just say “…amounts shown.” or “…amounts stated on the summary page.” or something along those lines.”
  • “On the /donations/ page. The numerical values representing contributions increase in real-time. Even though the numbers increase in varying amounts, their synchronous second-by-second updates implies a possibility that the figures to not actually represent actual numbers, but are just a product of predefined scripts. Updates triggered by actual page loads (or at random intervals, simulating triggering) will have a better impact on perception of accuracy.”
  • “I think it’s best that this site not have a slogan. Slogans are starting to become cliche if they haven’t already. The title itself serves as a great self-slogan, but could be better if depicted with an arrow pointing straight up after the “e”, implying “direct and constant growth” (Rather than the current logo with an curved arrow up and over the “R”, which can imply “Plateauing”, or “Attempted effort”).”
  • “ If I was supposed to pay for something or not I couldn’t really tell.”

—————————————- Most of the responses this time were incredibly short, though a few were quite good.  Next time we’ll probably go straight to Mechanical Turk and put emphasis on writing a longer description. One good part was that people suggested slogans, and some may be good.  Here are some of the better ones: - “Raise Awareness, Raise Hope” - “Browse the web, Raise money for the world” (Surprisingly similar to our previously considered possibility, “Browse the web, change the world” - “Raise your Awareness!” - “Browse and Bestow” - “Saving while Surfing” - “Relief Replace Rejoice” - “Helping Others is Only One Click Away” - “Browse. Donate. Smile”

We’ll test a few of these on Mechanical Turk.  Results will be posted as they come in. ## Lessons Learned

  1. Feedback Army is convenient, though the responses can be very short.
  2. Most users do seem to understand what Raise does from our current website.
  3. The specific way we donate (per time spent from initial user registration) is still unclear and unintuitive.
  4. Users are still concerned over the risk of downloading browser extensions, and if the money is actually going to their nonprofits.

You can see all of our feedback from Feedback Army here.