Any sharp edge in an image has an infinite number of frequencies once Fourier transformed.

This is why images get blurry when resized up! For some reason better algorithms never seem to become popular.

Heh. I hear you. But but on the other hand, a sawtooth is perfectly everyday with an infinite series of coeffs: http://en.wikipedia.org/wiki/Triangle_wave .

Every function with a Fourier series will have an infinite number of non-zero Fourier coefficients unless it's merely a finite sum of sinusoids. (Like, obviously.) So yeah, those are "everyday" Fourier series.