[ Content | View menu ]

Infomaki facts and figures

Written on March 5, 2009 by Michael Lascarides

Captive observation balloons. Digital ID: 1619619. New York Public Library
Our home-grown usability survey launched a couple of weeks ago and it’s spent most of the intervening time quietly gathering a ton of feedback from our users. Since there’s been a lot of interest around the app, I’d like to share some of the findings so far. First, some top-line stats:

Usability questions posted: 78
Total unique repsondents: 2,410
Total questions answered: 26,067
Average questions answered per person: 10.8
Average responses per question: 334
Average responses per day: 1,184
Obscenities submitted: 2 (Incredibly, the two nasty words showed up within the first 10 minutes of launch–thank you, New York! Even more incredibly, we haven’t had another since.)

The response has been terrific, with roughly a 2% click-through rate from the main site, and 80% of those visitors answering at least one question. In fact, feedback has been so plentiful that we have twice pulled the report because we had enough responses and needed to write new questions.

We’ve added a number of generic demographic questions to the mix (how old are you, where do you live, etc.) and the hope is that in future versions, we will be able to segment responses to one question based on the answers to another question. For example, we can test familiarity with certain terms in one question and segment out those responses by age (for any respondents who answered both questions).

Behind the scenes, there are definitely some improvements that need to be made. It’s becoming clear that a frequent pattern of use is to test the same question (”Where would you click to…?”) over screen-shots of several variant designs. Right now, one must enter the same question repeatedly to get these comparisons. Then once the screenshots are uploaded, it would be helpful to view the results for all variant designs for one question on a single page.

Infomaki screenshot grid view
One improvement that has been made is a statistical grid overlay that includes absolute click-counts and percentage of total clicks on each segment of a 40×40 pixel grid. This became a necessity for legibility once the number of responses grew into the hundreds, and the clouds of individual click-marks condensed into unintelligible blurs. (We also added a button to hide all click marks and stats grids to just see what’s underneath!)

So enough about the app itself–what have we learned? Well, one big lesson is that our global navigation designs with fewer choices vastly outperform those with more. Our first draft of the new global navigation had eight elements, but user testing quickly found some gaps and ambiguities. For example, we tend to think of the NYPL Digital Gallery as a kind of special project, so we were testing navigation items named “NYPL Projects”, “Online Resources” or “Online Collections” as a destination page for DG and similar initiatives. But testing showed that users don’t think it’s that special; they think it’s just another way to find things. When presented with one of those choices, a significant number of users still clicked “Books and Materials” as the place to find the image collections. Removing the “special projects” link (whatever its name) led to a user consensus on “Books and Materials” as the go-to destination.

We’ve also been able to ask “little questions”, something that’s not practical in a more formal survey. For example, we’re playing with the idea of a “Community” section as a destination for social networking initiatives (blogs, our iTunes page, our Facebook group, user-generated content and more), but we’re struggling with what to name it. We liked “Community” as a label, but tests showed no one was clicking on it so we quickly added a handful of follow-up questions to get to the bottom of what “community” means to our users, like whether people use the word “neighborhood” or “community” to talk about where they live (it’s 9-to-1 in favor of “neighborhood” in New York, by the way). The question “What would you MOST expect to see on a page labelled ‘Community’?”, coupled with a few set choices followed by an open text field, garnered the most free-form responses of any question so far. We haven’t cracked the Community code yet, but we’re getting closer. The ability to quickly get follow-up questions (even seemingly trivial ones) in front of users is proving to be incredibly insightful.

60 Comments

Comments closed

  1. Comment by Jonathan Rochkind:

    This is good stuff. Once you’ve gone a few more rounds with it, I’d encourage you to submit an article proposal to the Code4Lib Journal on it. I’d be happy to see an article on your testing technology, but also heavy on how you are using it — how you fit it into your design process, how you come up with questions, how you use the results, what you’ve learned from it.

    March 5, 2009 @ 2:49 pm
  2. Comment by Michael Lascarides:

    Thanks, Jonathan! It’s a crazy busy week here, but I’m hoping to have a draft to you by your March 20 deadline.

    March 10, 2009 @ 12:17 pm
  3. Comment by Amelia:

    Hi Michael,

    I’m a doctoral candidate whose dissertation is about social media, museums, and the concept of community. The last bit of your post here was very interesting to me since it gets at how people conceptualize this word. Your last post about Infomaki mentioned you were willing to share data with grad students. Might it be possible for me to get a view of the data related to your community questions?

    Thanks,
    Amelia

    March 30, 2009 @ 10:14 am
  4. Comment by Susan Teague-Rector:

    this is really cool!! – is any of the code open source :D

    April 7, 2009 @ 11:25 am
  5. Comment by Michael Lascarides:

    @Susan: A full open-source release is planned within the next few weeks. Look for an upcoming announcement on this blog!

    April 10, 2009 @ 9:15 am
  6. Comment by Jon Lesser:

    This looks very cool. I’m looking forward to the code release!

    April 21, 2009 @ 1:25 am
  7. Comment by wholesale new era hats:

    That is an awfully astounding column you’ve posted.Thanks a lot for that a fantastically amazing post!
    wholesale new era hats

    January 8, 2011 @ 5:30 am
  8. Comment by Merrin:

    Hi,

    Is NYPL continuing with Infomaki? I’ve just come across it and it looks really useful – but the code etc is from 2009 – which made me wonder if the project had been suspended?

    Cheers,

    Merrin

    April 3, 2012 @ 10:00 pm
  9. Trackback from apartment hunter:

    … [Trackback] …

    [...] Informations on that Topic: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    June 1, 2012 @ 11:13 pm
  10. Trackback from ex terug:

    … [Trackback] …

    [...] Read More: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    June 18, 2012 @ 9:45 am
  11. Trackback from הובלות:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    June 28, 2012 @ 5:12 pm
  12. Trackback from www.disegnitatuaggio.altervista.org/tribali.html tatuaggio tribale:

    … [Trackback] …

    [...] Informations on that Topic: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    July 3, 2012 @ 6:47 pm
  13. Trackback from how to make a website:

    … [Trackback] …

    [...] Informations on that Topic: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    July 10, 2012 @ 4:02 pm
  14. Trackback from pret en ligne:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    July 14, 2012 @ 10:36 am
  15. Trackback from More hints:

    … [Trackback] …

    [...] There you will find 68225 more Infos: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    July 18, 2012 @ 9:44 pm
  16. Trackback from source:

    … [Trackback] …

    [...] Informations on that Topic: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    July 18, 2012 @ 11:30 pm
  17. Trackback from cheapautocarinsurancequotes.org:

    … [Trackback] …

    [...] There you will find 68608 more Infos: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    July 20, 2012 @ 11:34 am
  18. Trackback from http://ratingsofinsurancecompanies.com:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    August 1, 2012 @ 3:58 am
  19. Trackback from how do i get life insurance:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    August 9, 2012 @ 9:05 am
  20. Trackback from payday loans 1000:

    … [Trackback] …

    [...] Informations on that Topic: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    August 10, 2012 @ 8:45 pm
  21. Comment by cheap longchamp bags:

    Las Mavericks over the summer, appear in a instructing personnel income, prior front desk staff Terry – Stokes when the founders within the levels about variety, in addition to consequently stand above the various prospects in becoming next Portlandcoach.

    Terry And Stokes positioned, which means all the Mavericks require as soon as possible to discover a ideal secretary person, for making ” up ” for the emptiness following he / she left. Were gossips that this Mavericks contain longchamp bags neared ex- Pacers advisor : Micheal O’Brien, want to permit him to subscribe to the group to be a box foreseeable future admin news reports.

    In keeping with ESPN the actual popular columnist Mark And Stein discovered upon Forums recently, O’Brien seems to have officially gotten to in addition to calf muscles agreement, he’s going to become the major secretary in the banc.

    About Twittollower, Stein said the fact: and also calves “- Sean O’Brien comes with achieved a contract, definitely become the primary admin longchamp bags sale of the alc ? longchamp bag ve, your partner’s associate coach Sean — Carlisle

    August 21, 2012 @ 5:12 am
  22. Trackback from click resources:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    August 28, 2012 @ 12:38 am
  23. Trackback from young driver car insurance:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    August 29, 2012 @ 9:44 pm
  24. Trackback from paskolos studentams:

    … [Trackback] …

    [...] Read More: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 9, 2012 @ 8:46 am
  25. Trackback from scott tucker houston tx:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 10, 2012 @ 1:46 pm
  26. Trackback from honn rei:

    Favorite link…

    [...] ordinarily posts some fascinating stuff like this. If you are new to this blog [...]…

    September 12, 2012 @ 7:23 am
  27. Trackback from Porr:

    … [Trackback] …

    [...] Find More Informations here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 13, 2012 @ 12:51 pm
  28. Trackback from Burr Ridge IL:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 14, 2012 @ 6:55 am
  29. Trackback from free pampers:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 16, 2012 @ 11:16 pm
  30. Trackback from car insurance companies in pensacola florida:

    … [Trackback] …

    [...] Informations on that Topic: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 17, 2012 @ 9:35 pm
  31. Trackback from благовония:

    … [Trackback] …

    [...] Read More: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 19, 2012 @ 10:01 pm
  32. Trackback from check this site:

    … [Trackback] …

    [...] There you will find 60441 more Infos: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 20, 2012 @ 1:31 pm
  33. Trackback from e cigarette:

    … [Trackback] …

    [...] There you will find 97972 more Infos: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 25, 2012 @ 10:34 am
  34. Trackback from vender ouro:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    September 26, 2012 @ 1:56 am
  35. Trackback from tire mining:

    Websites to visit…

    [...] needs pretty long time for achieving good results. One must have real fortitude [...]…

    September 30, 2012 @ 1:46 am
  36. Trackback from The diet solution Review:

    The diet solution Review…

    Infomaki facts and figures | NYPL Labs…

    October 1, 2012 @ 12:32 pm
  37. Trackback from quick cash loans:

    … [Trackback] …

    [...] Read More here: labs.nypl.org/2009/03/05/infomaki-facts-and-figures/ [...] …

    October 2, 2012 @ 11:28 pm
  38. Comment by Longchamp Pliage:

    http://www.lcpliage.com/ Longchamp Pliage

    October 7, 2012 @ 10:59 pm
  39. Trackback from Cheap Steelers Jerseys:

    Cheap Steelers Jerseys…

    Infomaki facts and figures | NYPL Labs…

    October 9, 2012 @ 6:13 am
  40. Trackback from jeux de casino en ligne:

    Websites to visit…

    [...] involves fairly long time for getting great results. One must have real dedication [...]…

    October 9, 2012 @ 10:09 am
  41. Trackback from home business ideas for women:

    Trackback…

    [...] ordinarily posts one or two quite interesting stuff of this nature. When you are new to this site [...]…

    October 10, 2012 @ 2:39 am
  42. Trackback from 2013 low cost business ideas:

    Read was interesting, stay in touch……

    [...] we wish to pay tribute to a number of internet sites on the net, regardless of whether they aren’t linked to us, by linking to them. Under are a few webpages really worth checking out [...]…

    October 12, 2012 @ 12:27 am
  43. Trackback from 趨㽿:

    趨㽿…

    Infomaki facts and figures | NYPL Labs…

    October 14, 2012 @ 2:50 am
  44. Trackback from rainbow riches slot machine:

    Sources……

    [...] that’s the end of this article. Here you’ll see several sites that we presume you’ll appreciate, simply click the links over [...]…

    October 16, 2012 @ 10:24 pm
  45. Trackback from home business ideas for women:

    Trackback…

    [...] always a huge fanatic of backlinking to blog writers that I appreciate but are rarely getting lots of link love from [...]…

    October 17, 2012 @ 3:41 am
  46. Trackback from low cost business ideas:

    Awesome website…

    [...] normally posts certain quite interesting stuff of this nature. When you are new to this page [...]…

    October 18, 2012 @ 4:14 am
  47. Comment by longchamp bags:

    This way, the particular Louis Vuitton buckskin items started out generations coming from technology to be able to technology descendant constantly abide by the particular search for top quality, superiority frame of mind. Nowadays, from your buckskin goods (including bags, suitcases, briefcases, and so forth. ) lengthy to be able to trend clothing, footwear, timepieces, fine jewelry, creating products and also accessories, Louis Vuitton continuing to boost regarding the modern significance, nonetheless it provides explain to you the particular.

    October 18, 2012 @ 5:03 am
  48. Comment by payday loans UK:

    At payday gwrtdrfs Loans you buoy inspire a speedy, payday loans UK >:]] easy small signal payday loan of up to 1500. Thither’s no daily endeavor and no sound songs. The apply is totally on-line and buoy be realized in 60 bat of an eye. A payday Loanword is a colossal method payday loan >:]] to screen small expression notes surge issuances, portion you refrain from costly belief carte or swear burden. You buoy pament unforeseen nibs or level conscientious delicacy yourself or soul added. pay day loans >:-[ Assume’t away whatever also clock distressing almost those sudden chicamin difficulties and create now your payday with payday Loans. Penetrate Hither to boast away how even your payday Accommodation inclination charge. Handle Instantly and get the chips cashed to your bill in 15 minutes.

    October 18, 2012 @ 9:59 am
  49. Trackback from home business ideas:

    Favorite webpage…

    [...] In addition to a massive butterfly structure, with ribbons near the charming photograph of the relaxing born [...]…

    October 20, 2012 @ 3:18 am
  50. Trackback from low cost business ideas:

    Great website…

    [...] we ran into a very good site that you might love. Look it over if you’d like [...]…

    October 22, 2012 @ 11:24 pm
  51. Trackback from url:

    url…

    Infomaki facts and figures | NYPL Labs…

    October 23, 2012 @ 4:47 pm
  52. Trackback from business opportunities:

    Great sites…

    [...] Let me reveal a few similar facts [...]…

    October 24, 2012 @ 12:48 am
  53. Comment by personal loans:
    October 24, 2012 @ 9:48 pm
  54. Trackback from low cost business ideas:

    Favorite link…

    [...] plenty of time to study or visit the particular content material or sites we now have linked to below the [...]…

    October 25, 2012 @ 12:46 am
  55. Trackback from home business ideas:

    Recommended websites…

    [...] I noticed someone having a debate about this on Tumblr and so it tied up to [...]…

    November 6, 2012 @ 10:48 pm
  56. Trackback from lexington law:

    Recent Blogroll Additions……

    [...] we like to honor a number of sites on the net, even if they are not linked to us, simply by linking to them. Below are some sites really worth looking over [...]…

    November 7, 2012 @ 9:11 pm
  57. Trackback from elevation group scam:

    Websites to check…

    [...] Internet sites of interest we have a link to [...]…

    November 8, 2012 @ 1:29 am
  58. Comment by where to buy viagra:
    November 9, 2012 @ 10:26 pm
  59. Trackback from business opportunities:

    Nice Focus……

    [...] listed here you’ll find the link to a few sites that many of us presume you need to have a look at [...]…

    November 10, 2012 @ 8:26 am
  60. Trackback from Amazon Gutschein:

    Superb website…

    [...] Are you without any doubt certain that is definitely the right facts? I observed something else. I’ve basically had another concept that pop into my mind [...]…

    November 18, 2012 @ 12:24 am