An RPG Company (kinda) Performs Market Research!!!

This "living ruleset" thing looks like the stirrings of a tempest in a tea pot to me.  Quite frankly, it's the least most RPG publishers should do to take the pulse of their audience.  I see no difference between WotC's surveys and what Raggi does (though I suspect Raggi's attempts are more effective).  Neither option is terribly scientific, and both heavily favor those who regularly use the platforms on which the data-collection occurs and enjoy blathering about their own opinions.  (In short, people like me!)

Though now I'm curious if they do anything to collect data via D&D Encounters.  Sure, they collect "results" but do they collect what races and classes are played?  What about spells prepped and cast?  Abilities used?  How long certain fights take?  Solutions attempted in the face of challenges?

On the other hand, how useful would this data be?  There's long been talk about D&D being shaped by organized play in directions that are not terribly friendly to private games. 

At the end of the day, I'm happy to see folks performing any sort of market research on RPGs.   After decades of stupid and incorrect "conventional wisdom" (Box sets killed TSR!  Adventures are loss-leaders, necessary but a drain on publishers!) it's nice to see folks actually taking the time to find out what gamers actually think and want and use and buy. 

Right now, within easy reach of me, are the core dead-tree resources I use regularly in my weekly games: Moldvay's Basic, Cook's Expert, Vornheim, and 2e's Al-Qadim book (primarily for the equipment lists) and Monstrous Manual.  It's not a collection I think any market strategy team would ever devise.  It is, however, a collection of book-types that have served me very well over the years: basic rulebooks, a monster book, and a book of gear and services PCs should be able to purchase whenever they've returned to civilization.  Vornheim mostly gets used for generating NPCs and for its wonderful searching-a-library rules (among other odd bits in it).  The 1e DMG isn't at hand, but I pull it out when doing prep work.

This collection hasn't changed much since 1990.  Back then, the Moldvay/Cook books were replaced by the 2e PHB.  The Al-Qadim book was replaced by the Arms & Equipment Guide and Arora's Whole Realms Catalogue.  The Monstrous Manual was heavily supplemented with 1e's MM and MM2, largely for the demons, devils, daemons, and modrons, all of whom made regularly appearances in my college game.

I bring this up to speak of the limits of the sort of market research I see WotC performing.  They're looking backward: what did we do right and what did we do wrong?  A stronger focus on utility would probably serve them better, but they need to take a broad view of utility.  I replaced two books narrowly focused on my need (the Arms & Equipment Guide and Arora's) and replaced them with a book that, ostensibly, has little or nothing to do with that need (Al-Qadim).  Utility has nothing to do with what's on the cover and everything to do with what's inside and what can and does get used at the table.

How do you capture that data?  Maybe by asking DMs to take snapshots of their gaming table at the end of the game so you can see what books and resources are there, having been used.