Yes, you have to go to Cape Point, when you visit Cape Town! I mean it is kind of cool to stand at this point, look at the ocean and think: The next land is Antarctica. You can stroll around Cape Point quite a bit, walk up to the lighthouse, take pictures at the Cape of Good Hope sign, and even hike down to Diaz beach.
I recommend you make a nice day trip out of it: Start your drive in Cape Town, make a stop in Muizenberg, a stop in Simon's Town to sea the penguins if you want, and drive back view Kommetjie, Noordhoek and Chapman's Peak, maybe ending for the sunset in Camps Bay. This way, it's a beautiful round trip.