To be able to see the northern lights it's important to get away from the light pollution of the city. A great way to do this is to hop on a boat from Reykjavik harbour and sail out to sea. That way you get unrestricted views 360 degrees around you. if the lights come out, you wont miss them.
The great benefit by going on a a boat with Special Tours is that they have a large seating area inside and even a cafe. So you can stay warm and cozy and when the crew says the lights are visible, you simply go outside to watch them.