While this area has lost its former evocative seediness, this is still New York City--so watch your wallet, especially in crowds. Otherwise, Times Square is quite safe most of the time, though it can feel a bit quiet at 2 AM.
This area is the home of some of the most prestigious private clubs such as the Harvard Club, Yacht Club and fabulous hotels on 44th street between 5th and 6th avenue. The area is not just for tourists is a place where locals hangout for lunch or after work visiting some great restaurants.