You can take an Amtrak train to Kingston, RI, from either Boston or New York. The former is about an hour, the latter takes 3.5 hours. Greyhound and Megabus operate between New York and Providence, an hour away from Kingston. Once in Providence, the Rhode Island Public Transportation Authority offers transportation to Kingston.
Kinney Azalea Gardens is a sweet botanical garden in a historic setting. You can stop by Proclamation Ale Company for a quick beer, and there is also a gun range just to the east of the University of Rhode Island.
There are a couple of great spots to eat around Kingston, with most restaurants clustered around Fortin Road and Chapel Way. Here you'll find Mediterranean, Mexican, and Thai cuisine, burgers, and more.
If you have a car, you're only about 15-minute drive from the Narragansett Town Beach. Even if it's not swimming weather, you can stroll along the sand as well as enjoy the area's seafood restaurants and cafes on the water.