Refuel at Coffee & Gelato with an ice cream cone (my choice) or a late afternoon pick me up cappuccino (Brian's choice). This hidden gem isn't on Simon's Town picturesque Main St. but the back garden is a nice place to chill out before hopping back in the car.
If you want to skip the ice cream and just sit down for a cup of coffee, check out Blue Door Coffee on 3 Rectory Lane.