Sometimes, skill challenges suck. I'm not one of those people who thinks that they need to be stricken from the game forever, but I have certainly run skill challenges that were not at the top of their form. In fact, I ran one on Monday, and it was mostly terrible. The challenge was getting across the Tibean Wasteland. For the most part, the development of the challenge wasn't as important as the act itself, which should have been my first clue that the challenge wasn't even needed in this case, and I would have been better off making it a series of role-playing moments, a combat encounter, and some investigation. Honestly, I think an investigation skill challenge would have been more fun in its place, and now I'm a touch sad that I didn't run one. I'd offer to do one this coming Monday, but I have different plans for this upcoming session. Maybe when we reach the Spine of the World, or the city of Sternum...
I think that's probably the one thing that differentiates skill challenges that are awesome and skill challenges that suck: what they're for. I understand that some players hate them in general; the math is screwy, or they believe that the challenge would be better left to good role-playing. Personally, I disagree, but I'll admit that when a skill challenge isn't going to clearly effect things, it's pretty stupid to have a skill challenge at all. Whether they passed the skill challenge or not, my players would have crossed the Tibean Waste. Making it a skill challenge didn't add any tension or drama to the act, and if it had, I think I would have failed as a DM. That's not a thing that should invoke tension, it's a simple act and it should just get done. Travel, meet some people while you're travelling, fight some bitches what get up in your grill, then be done with it.
When you're crafting a skill challenge, it has to mean something. I think that's part of what made the Night at the Opera skill challenge such a success; the players knew that if they did not make their show a success, they would have to come up with a different (probably much more difficult) way to get into the Mayor's house and get the information they need to open the Dungeons & Dragons Inc. Lodge. That is a damned fine skill challenge. Development for that makes a big difference. If you succeed, you get to go to a fancy party, and when the guests are all unconscious or fucking, you can do what you need to do. If you fail, you have to bust in the door, deal with the guards, find a way to subdue the mayor and his guests, and get done what needs doing. The first way is much more fun, much less trouble, and is better for role-playing and story-telling purposes. The second way involves a lot of combat, a bunch of unwanted attention, some trouble-shooting and an uphill battle to get to your goals.
And I think that is what a skill challenge should really be about. When you're planning these things, keep your goal in mind. And your goal should be fun. What would be the fun way to get this goal accomplished? What would be the less-fun way to do the same thing? Now, make the fun thing the reward for accomplishing the skill challenge. Make the unfun thing the penalty for failure. Set this up from the beginning, let your players know what is at stake before they start trying to figure out how they're going to roll this, and make them think about the actions they'll be taking to use those skills effectively. That will add tension to the challenge, make it worth doing, and make the whole experience a lot more fun to play through.