Become a fan of h2g2
The Sun rises in the east and set sets in the west. At midday it is due1 south if you are in the northern hemisphere or due north if you are in the southern hemisphere. At 6am, it is due east, and at 6pm it is due west, even if it dark at these times of day. It moves between these points at a constant speed.
So at 9am (GMT) the Sun will be exactly southeast.
It is important to note that all times in this article refer to solar time - that is, midday is taken as being when the Sun is at its zenith. In some places this may differ by several hours from 'official' time, due to Daylight Saving Time and/or extended time zones to keep a large country on the same time.
By pointing the hour hand of an analogue watch at the Sun, south will be half-way between 12 o'clock and the hour hand. If the Sun is very high in the sky, it can be difficult to judge its direction; in this case, place a straight stick vertically in the ground and align the watch hand to this; due north will now be half-way between 12 o'clock and the hour hand.
With a little practice, it should be possible to take a fairly accurate reading knowing the time but without using a watch directly. It is a good idea to take a moment to think about the direction you have produced, to avoid silly errors.
The Sun is not directly over the equator. It moves around during the year because of the Earth's changing axial tilt. The tropics are defined as the areas on the Earth where the Sun is directly overhead at some point during the year; so if you are within or very close to the tropics, the determination of north or south as described above may not be reliable. You can correct for this if you know what time of year it is. The solstices (midsummer and midwinter) are when the Sun is furthest from the equator at midday; on the equinoxes (mid-spring and mid-autumn) it should be over the equator.
The Arctic and Antarctic Circles are defined as the areas where the Sun never sets on Midsummer's Day and never rises on Midwinter's Day. Within these circles, although the Sun will rise and fall in the sky, it may not do so enough to be obvious. Sadly, there is no way around this; it is not possible to navigate using the Sun when the Sun is not visible or not visibly moving. In particular, since the Sun is visible at night, using the method described above will locate the North Pole, not the South. (It may be possible to come to a rough estimate taking into account precise measurements, your latitude and the date, but this is a complex procedure requiring measurement equipment, and a compass or GPS is far preferable.)
Dawn and sunset times vary according to the time of year. Most importantly, you must use local solar time2, not Daylight Saving Time. However, the Sun is always due east at 6am and due west at 6pm.
It could be cloudy! Again, there is no way round this; if you cannot tell where the Sun is, you cannot navigate using it.
It is possible to navigate using the stars, but this is a much more complex skill. The constellation Orion is usually near the equator, but this varies greatly according to the time of year. Polaris (alpha Ursae Minoris, the North Star) and Crux (the Southern Cross) are good year-round indicators if you can locate them.
Navigating accurately using the Moon is impossible for all practical purposes. Even a rough direction requires knowing the times of moonrise and moonset. Add to this the fact that on most nights there will be a period where neither the Sun nor the Moon is visible, and the advantages are very small. If necessary, the same rules listed for the Sun may be applied if you know the local 'lunar time' (time based on when the moon is at its highest) rather than 'solar time'.
Other Handy Indicators of Direction
Moss grows in the shade, so it will favour the north side of trees in the northern hemisphere. However, in generally overcast places, such as woods, this may not be detectable. Longer branches on trees may also grow towards the equator in strong sunlight. Communication satellites must be in geostationary orbit, so satellite dishes will face a point directly over the equator, as will solar panels. This can be a useful indicator of general direction if you are well outside the tropics.
While this will never be an exact method for travelling accurately across featureless wastes, it does provide a quick and (with practice) simple method of checking that you are heading roughly the right way. There is no shame in using another rough method to back it up - for example, if you know you are in a city with a north/south/east/west street grid, it becomes very simple to check you are heading in the right direction.