Reviewing the reviews

As the senior editor in Macworld ’s Reviews department, I get questions. Oh boy, do I get questions. So let me just clear up some things, dear readers, that may be on your mind as you read Macworld reviews. A peek into the sausage factory, if you will.

How does Macworld choose products to review?

Every Macworld editor is involved in choosing products for review. Each editor has his or her own beat, or product category, to follow. Jon Seff follows the audio and video beats, for example, while I follow utility software. (To find out who covers a particular product category, see the Macworld Editorial Contacts list.)

Sure, we editors field press releases and phone calls from companies. But we also look—and hear about—products just about everywhere else, too. We talk to our Mac-loving friends, hunt down tools for our own use, get suggestions from our writers, and search VersionTracker and the Web in general. Companies don’t necessarily need big PR machines to get covered in Macworld.

We periodically meet as a group to decide which products we’ve heard about will be interesting to our readers. A lot of products don’t get reviewed, as much as we’d like to review them all.

Why didn’t you include Product X in your latest comparison review?

There are few reasons for not including a product.

  • The product hasn’t been revised lately.
  • The latest revision may be very minor.
  • It doesn’t compete in the same market as the other products.
  • We didn’t know about the product, for whatever reason. (If this happens, we get very upset. In fact, I appreciate the reader letters and e-mails that point out a product we don’t know about.)

Do you review a product (or give a good rating) because a company advertises with Macworld ?

This could be the question (and accusation) that irks us all the most—including our ad sales folks. Editors have no idea who’s advertising in any given issue until we get our printed copies, and the brave souls who sell ads for Macworld don’t even know what we’re reviewing until the review’s almost finished—a pretty rare situation, these days. Call us naive—we feel strongly that “separation of church and state” is the way to do independent product reviews properly.

How do you actually review products?

We request evaluation copies of software and evaluation units of hardware from the companies whose products we want to review. Usually they send them to us, because that’s the industry standard way of doing it. When we ask and ask, but receive no answer, we try to buy the product.

How can I get my product reviewed in Macworld ?

First, contact me at reviews@macworld.com, and ask for our “Working with Macworld’s Reviews Department” document for guidance. Then, contact the editor who covers your product category, and let him or her know the product exists. After that, we’ll determine whether we’re going to review the product. It also really helps to send us the product, so we can check it out for ourselves.

Who writes Macworld reviews?

Mostly freelance writers, and we have quite a stable of them. But I like to try to con some of our staff editors to write them too, whenever I can.

How do you assign mouse ratings to products?

We have a very fancy spreadsheet that weighs these criteria:

  • Required Features (Are they present?)
  • Innovation
  • Design Implementation
  • Performance
  • Reliability
  • Ease of Use
  • Value

And then the spreadsheet spits out a mouse rating.

Once we have a baseline mouse rating, the reviews editors may decide to adjust the rating up or down, in collaboration with the writer.

There you have it, dear readers. A guide to the most-often asked questions about Macworld reviews. Now it’s your turn. Go ahead and ask some questions on our forums. I’ll answer as many as I can. (A note to angry readers: I realize that product reviews can really get up the ire, but please, keep your comments and questions productive.)

  
Shop Tech Products at Amazon