I came across an article on PopSci.com that provides some argument to show how golf can be considered a sport:

It’s an age-old question debated in pro shops and pubs across America: is golf a sport? Neil Wolkodoff, director of the Rose Center for Health and Sports Sciences in Denver, thinks it is, and he has some data to back up his claim. Wokodoff took eight better-than-average golfers and tracked their heart rate, oxygen consumption, carbon dioxide production, and how far they were walking through a few rounds.

“The study shows there’s significant energy expenditure in golf, more than bowling and some other sports it’s been compared to,” Wolkodoff said to the AP. “There are a lot of sports that don’t have this level of energy expenditure.”

Subjects walking and carrying their clubs burned 721 calories per round, while the lazy folks in the carts burned just 411. Surprisingly, there was no difference in carrying clubs versus using a push cart, so save your back and rent the cart. The data also suggested that players went past their anaerobic thresholds after walking through two uphill holes (feeling the burn).

So golf burns more calories than an hour of billiards (216), fishing (302), or even a relaxed canoe trip (345), but we’re not sold that energy expenditure alone defines what a sport is. Everyone agrees that cheerleaders have bodies to prove they’re burning calories, but good luck getting consensus on whether it’s a sport. Curling only burns 345 calories during an hour of competition, but in Canada, it’s not just a sport, it’s the national pastime. Is Tiger Woods proof that golf is a sport, or is John Daly confirmation to the contrary? That probably depends on whether you’ve got a set of clubs in the garage.

I guess looking at the burn-rate of calories is one way to look at it, although one of the commenters did notice that the author did his comparisons to other sports incorrectly. The author claims that one round of golf burns 721 calories, or 180 calories per hour (based on a four hour round), which makes it the sport that burns the least calories in one hour.