On Jan 23, 2008 6:02 PM, Warren Togami <wtogami(a)redhat.com> wrote:
Today InstantMirror is pretty useful for home and small office
mirrors,
but its limitations make it unsustainable without manual intervention of
the sysadmin.
I've been beginning to think that perhaps InstantMirror is heading down
the wrong path and we seriously need to rethink it. There are simply
too many limitations of the current "stateless" operation of
InstantMirror where it runs only on-demand as mod_python script:
- Synchronization/locking of multiple connections downloading the same
file is awkward and broken.
- There is no good way to clean up aborted tmp files.
- There is no good way to know what are old files that need pruning.
- There is no good way of keeping track of the "Big Picture" of its own
cache, "least recently used" knowing what files were unpopular locally
and should be pruned.
https://fedorahosted.org/InstantMirror/wiki/InstantMirrorDaemon
We need a daemon to handle all this. Perhaps the daemon could allow
socket connections from a mod_python script for accesses. Or perhaps it
might be better for the daemon itself to handle serving connections.
Stepping back, what we really need is:
A reverse proxy caching server with all the logic of squid or varnish,
except it stores its cache with file and directory names intact.
How do we get there?
1) Write a new daemon from scratch?
2) Write a new backend storage engine for squid or varnish? (Store
files in target directory structure, store metadata elsewhere.)
3) ???
Any thoughts?
Am about to investigate on-demand style mirroring as opposed to
rsync/ftp myself, and came across InstantMirror. What I have not yet
been able to determine is what is the benefit of InstantMirror over
just running squid and having yum always use said proxy?? (yum really
should be able to independently use an http proxy)
--
Fedora 7 : sipping some of that moonshine
(
www.pembo13.com )