PHP (or possibly Apache itself) inserts a default charset= field in the Content-Type: header. This can cause problems with the syndication feeds, if you are not using the exact same character set on your blog. In my case the character set I use on this site is iso-8859-15 while the default that shows up is iso-8859-1.
I modified wp-rss2.php to return the character set specified in the WordPress options, and then went through other files to apply the same change. You can download the full patch that should apply cleanly to WordPress 1.2.