Hi,

Even when not playing any audio, pulseaudio uses CPU and memory
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND       
 2257 marius    20   0  941m 7524 3188 S  2.9  0.4  14:22.57 chrome        
 2956 marius    20   0 2026m 537m 9612 S  2.9 26.8  27:40.42 java          
 1500 marius    20   0  437m 3100 1768 S  2.0  0.2  14:53.45 pulseaudio 

There are numerous reports about this
http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=pulseaudio+cpu

Do you have any suggestions for fixing or replacing it with something else?

Thanks