If you’re like me, one of the first things you turn to when reading a newspaper, magazine or Web site are the product reviews. Whether it’s for an immediate purchase, to stay informed or just a casual interest, many people are interested in the opinions of others.

Consumer magazines have recognized this and beefed up their review sections, while most major shopping Web sites prominently feature product reviews from their visitors. However, our industry is very different. There are no sources for independent product reviews that I know of, and the material that is out there is generally written by the manufacturers to make their product appear in a favorable light.

Product demonstrations are a notoriously poor way to evaluate technology for purchase. If they go well, you wonder if it was the skill of the presenter that was responsible. If they go poorly, you wonder the same thing.

The only answer I have found is that sooner or later, you have to perform product evaluations yourself.

Analyzing Performance
When evaluating a product, most of us are looking for four things. I have listed them in my order of priority. Your priorities may be different, so I’ve added my reasoning as well.

Performance is first on my list for one simple reason: If it doesn’t do what you need it to do, nothing else matters.

Don’t confuse this with product specifications on published data sheets. The performance you are looking for is the answer to the question, “Will it suit my needs?”

Remember: You’re not judging a beauty contest looking for a single winner. Your goal is to find a product that performs a specific set of tasks. It is even better if you find more than one product that does what you need.

To evaluate performance, you would ideally put a product you are testing into your production environment, but that’s not always possible. If you have to test products “off-line,” try to simulate real-world conditions as closely as you can.

With a video product such as a DVR, use a live camera feed or DVD footage from a camera. If you’re looking at monitors, make sure room lighting in your test area is comparable to the environment where they would be used.

IP cameras need to be tested on networks with some traffic. If your test network is an open highway, simulate some traffic by streaming video or music files across it.

Sometimes, it can be difficult to resist the temptation to compare products and pick the best. This came up recently when testing embedded DVRs for a client. I was looking at two different models with very similar specifications. Despite the similarities, a video test pattern showed one product was clearly superior to the other.

I found myself ignoring the lower quality unit since I was sure that it would be rejected. The fact that either unit would have met the client’s needs was temporarily lost on me.

Ultimately, I went back and tested each unit — using moving video images rather than test patterns — to make sure I was being objective. After evaluating the remaining criteria, we came back to video quality. The poorer performance wound up being the tiebreaker.

Make Sure to Rank Reliability
If your first question was whether or not the product would do what you needed it to do (performance), your second question should be, “How well will this product hold up?” This is why I emphasize moving on to reliability.

If there’s a product that is better than “good enough,” but looks likely to fall apart before it’s time, it won’t be your first choice. Pricing may be dropping as technology advances, but we don’t have disposable products just yet.

While this may seem like a category that requires a crystal ball, it can often be surprisingly easy to predict.

First, take a look at the product — pick it up, shake it a little, check out the buttons and controls. Does it feel like something substantial? Do the buttons work, or do they occasionally stick? Are the rear panel connections secure or do they look like they’ll fall apart? Your first impression is the sum total of your life’s experiences, so give some credibility to your “gut reaction.”

Your next step is to plug the product in and operate it. If it will be rack-mounted, try to test it in a rack. Otherwise, restrict the airflow around it and check for heat.

Since thermal issues seem to be the No. 1 cause of problems with many products — particularly digital ones — feel the heat flowing out of the vents on the product and check out the case temperature. Is it too hot to touch?

Products that run hot almost always fail before their cooler running counterparts, so this should be a major warning bell if you’re reaching for the burn cream.

If you have permission — or can be very careful — open the product up and look inside the case. Don’t do this unless you know what you’re doing and take the necessary precautions. As the warning sticker may say, there can be lethal voltages inside or you could damage the product and have to pay for it even if you don’t like it.

Once inside, look at the circuit boards. Are they mounted securely in place with connections that feel solid and aren’t easily pulled apart? Disconnect the power and check out the internal temperatures of various components for excessive hot spots.

Understanding Usability
Another important factor is how well thought out the product is and how easy it is to use. While some downplay the significance of usability, a thought-out user interface and user-friendly features are often signs of a company’s understanding of its customers.

While easy-to-use may not mean that one product is better than another, it usually means you’ll get more out of it — unless you’re the type who enjoys referring to the manual.

If you are technically inclined, you may be the wrong person to evaluate this. Try to find a nontechnical person who understands the application and sit them down in front of the product without a manual. They should be able to operate at least the basic features intuitively, relying on the front panel legends, help screens, a “quick guide” setup sheet and experience with consumer products such as televisions and microwave ovens.

Even a technical person should be able to figure out better than half of the functions without looking at the manual.

Take a quick read through the manual as well. Sometimes, a well-written manual can make even advanced features seem easy, and most products are too complex to rely entirely on intuition.

Looking for Features
Last on my list is features, which is usually the tiebreaker for me. A feature to me means added, “bonus” functionality that differentiates a product. On a monitor, for example, it might be calibration presets, a greater selection of inputs or picture-in-picture functionality.

I consider ease of installation a feature. A product can be a great performer, reliable and easy to use but a bear to install. Your priorities may be different, but if you’re an integrator and plan on installing a lot of them, this will naturally be a key consideration.

Don’t forget the features that have nothing to do with the product itself but should nonetheless be factored into your evaluation. These include warranty, local support and, especially, the way you are treated when you call for tech support or have any questions.

The packaging should be a factor as well — if one product is more carefully packaged and comes with all the needed accessories, it should score higher than the one that forces you to go to Radio Shack to look for a connector or breakout box.

Last Step Is Documentation
When you’re done with your
evaluation, be sure and write up your review. Even if you’re the only one who will ever read it, it is important to document your conclusions and how you reached them.

Keep your notes as well, but be careful who you share your opinions with. Everyone loves reading reviews, but I’m not sure that means everyone loves the reviewer!

If you enjoyed this article and want to receive more valuable industry content like this, click here to sign up for our FREE digital newsletters!

Related Content