I disagree that this is off-topic; it's about the differences of the systems in a cloud environment, versus a non-cloud environment.  Seems like that makes it pretty apropos for a cloud list ;)
 
any advanced unix-like clone supports and recommends that -except linux :)

not sure where you get that idea; some of the distributions make life easier (on themselves) by not breaking them out during the default installations.  A default install from a CD is only meant for a person just starting out, however - and any of the distros designed to be used for "real" tasks have easy automatic tools for breaking apart partitions.  It's really easy in a kickstart file, for instance.  There is absolutely nothing of merit anywhere I've ever seen (and I've been a Linux sysadmin-type since late 94) that recommends using a single volume for everything.  It's a more accessible OS than any of the other UNIX variants, which means it has more easy howto-guides for beginners; some of those might recommend a single volume, but just for making things easy to get started (which isn't the same as a person doing real work with the system).
 
agreed, but to skip one layer of security because it is not the saint grail it not a smart move. I

It's not a layer of security.  It is false security - that's the point.  False security is worse than no security; at least when there is no security, people /behave/ as though there is no security.  But if people think things are secure, they do things that are then less safe.  It's a social engineering problem - one that impacts how an instance is used.  The right thing to do is to treat the instance-store (the S3 "volume" provided to the instance) as an insecure place, without adding any false sense of security to it.  Adding a layer of "security" that isn't one - hurts, not helps.
 
well this is not the case. S3 is used to store the linux image and during the instance creation

It is the case.  S3 is just a webserver, serving out your files when you do a get, changing them when you do a put, etc.  "partitions" is meaningless, as it is not a discrete filesystem.  Your single "filesystem" is spread out on hundreds of servers; partitions as boundaries are meaningless on non-discrete storage.
 
the system downloads the image and creates a local copy of it and the FS is created on the local hard drives. S3 is not suitable to store you root filesystem and operates a running system from there for multiple reasons(one is latency)

From my understanding, these are blade servers; there is no "local" hard drive - everything is either from a SAN (EBS) or from S3.  And yes, sometimes s3-backed systems experience I/O latency.

Brian