The most common complaints seem to be about configuring the Express. And I admit, it's tricky. I wrestled with mine for a little while when I first got it, too.
The first thing to remember is that you need to be connected to the AirPort Express somehow in order to configure it. Okay, that sounds obvious. But sometimes it's easy to get bogged down in the details and lose track of something as basic as selecting the AirPort Express (out of the box it'll have a name starting with "Apple Network") from the AirPort menu in your menu bar before running the AirPort Setup Assistant.
And then there are the people who are buying the AirPort Express to use it as an Ethernet bridge -- in other words, people who aren't planning on using it with a wireless-capable product of their own. For this, two pieces of advice.
First, if you're planning on using the AirPort Express as a bridge to an older Mac without wireless capabilities, that doesn't mean you need to use that Mac to configure your Express. Instead, install the AirPort Express software on a laptop or other Mac with AirPort access, and use that Mac to configure the Express.
Second, if you don't have any computer with wireless access handy, that's okay. Attach your Mac to the AirPort Express via an Ethernet cable, directly. Then you should be able to connect to it and set it up.
Now some more general advice. First, even if you think you're a really, really geeky Mac user, don't start with the AirPort Admin Utility. Apple has really done a remarkable job with the AirPort Express Assistant, and you should give it a try. Yes, it's a wizard-style interface -- what's the challenge in that? -- but when you're in a situation where you're configuring two wireless devices at once (for example, setting up a wireless relay from one AirPort Extreme or Express to another one), using the AirPort Admin Utility can get really hairy. In contrast, the Assistant is a breeze.
And most importantly, when in doubt, press the tiny reset button on the AirPort Express with a paper clip or other tiny object, and hold that button down for five seconds. This will reset the Express -- a great lifeline if you find it impossible to connect to it by any fashion, wired or wireless.
Finally, if you're still considering whether or not to buy an AirPort Express, I recommend you read up on the product and consider if you really need one. It's undeniably cool, but unless you're planning on using more than one of its unique features, you might be better off with a much cheaper product from some other company. Then again, if you value ease-of-use, the AirPort Express Assistant blows away the Web-based (and Windows-centric) setup routines of most other wireless products.