Geoffs tip with \textless and \textgreater will work well for you. However, you could type these symbols directly in your editor and they would be correctly printed if you use the recommended font encoding, Cork resp. T1:

\usepackage[T1]{fontenc}

Have a look at the T1 encoding table, search for the symbols . Afterwards, open the OT1 encoding table, which is the default. At the two corresponding places you will find the upside down exclamation resp. quotation mark. That should explain it.

For further reasons, why you should use T1 encoding, have a look at this question: Why should I use \usepackage[T1]{fontenc}?

You should use a font supporting T1. For instance use the very good Latin Modern font, derived from the standard fonts:

\usepackage{lmodern}

Or install the cm-super package which provides the standard Computer Modern fonts with T1 support.