hi ! while trying to download F9 what I face is quite frustrating ..in fact this can be faced by a lot of user in India . while wbut mirror is down (temporarily ) and other cannot handle all the load and bit torrent is quite slow . clicking on i386 direct download always gives error 412 : too many connected users . any way of avoiding this .. I am currently downloading from a mirror in USA and its quite slow.
subhodip biswas wrote:
hi ! while trying to download F9 what I face is quite frustrating ..in fact this can be faced by a lot of user in India . while wbut mirror is down (temporarily ) and other cannot handle all the load and bit torrent is quite slow . clicking on i386 direct download always gives error 412 : too many connected users . any way of avoiding this .. I am currently downloading from a mirror in USA and its quite slow.
Hi,
Going from what I know (someone might want to update me here): We currently only have 3 mirrors in India (http://mirrors.fedoraproject.org/), one is on a 45mbps link, while the other two are on 5mbps and 2mbps links.
From what I see, there isn't really much we can do apart from waiting for more Indian mirrors to come aboard, ISPs/Universities are the best bet and is actually where quite a few of our mirrors are located.
Another alternative is try an Asian or Europe mirror, China, Japan, Italy, France etc, many of these mirrors are on 100Mbps+ links and shouldn't be returning 421 errors. (I have no clue on the Indian internet topology but generally geographically closer locations have better links)
Noting the 421 errors, it's fairly hard to prevent them, the script (as Matt mentioned on IRC) could be adapted to query the server if there are slots open but thats not always possible.
- N.J.
Amendment: I noticed the two slower mirrors only allow ftp, try http://ftp.iitm.ac.in/fedora/ which is the server on the 45Mbps link and responds to me in NZ.
On Thu, May 15, 2008 at 10:04:51PM +1200, Nigel Jones wrote:
subhodip biswas wrote:
hi ! while trying to download F9 what I face is quite frustrating ..in fact this can be faced by a lot of user in India . while wbut mirror is down (temporarily ) and other cannot handle all the load and bit torrent is quite slow . clicking on i386 direct download always gives error 412 : too many connected users . any way of avoiding this .. I am currently downloading from a mirror in USA and its quite slow.
Hi,
Going from what I know (someone might want to update me here): We currently only have 3 mirrors in India (http://mirrors.fedoraproject.org/), one is on a 45mbps link, while the other two are on 5mbps and 2mbps links.
One thing MM would benefit from would be a weighted sampler.
Right now, it collects various lists of Hosts to return (same netblock, same country, same continent, global). It shuffles (python random.shufle()) each of the lists, then concatenates them, and sends that result to the end user. However, this does not take into account that one mirror has a 45Mbps link, and another a 2Mbps; they would each get chosen "randomly".
What I need is a replacement for random.shuffle() that takes a list of tuples: (something, weight). And returns a list of somethings that was generated with a statistical sampling based on weight.
Now, this is _probably_ a simple thing to do; my college number theory professor would mock me for having not "just done it" myself, but hey - I took that class _twice_ before passing it - I don't trust my skills in this area to get it right.
Volunteers?
From what I see, there isn't really much we can do apart from waiting for more Indian mirrors to come aboard, ISPs/Universities are the best bet and is actually where quite a few of our mirrors are located.
Another alternative is try an Asian or Europe mirror, China, Japan, Italy, France etc, many of these mirrors are on 100Mbps+ links and shouldn't be returning 421 errors. (I have no clue on the Indian internet topology but generally geographically closer locations have better links)
And please please, help find more mirrors in India.
Noting the 421 errors, it's fairly hard to prevent them, the script (as Matt mentioned on IRC) could be adapted to query the server if there are slots open but thats not always possible.
yea, I like _this_ less and less the more I think about it.
- N.J.
Amendment: I noticed the two slower mirrors only allow ftp, try http://ftp.iitm.ac.in/fedora/ which is the server on the 45Mbps link and responds to me in NZ.
which just came online yesterday.
One thing MM would benefit from would be a weighted sampler.
[...]
What I need is a replacement for random.shuffle() that takes a list of tuples: (something, weight). And returns a list of somethings that was generated with a statistical sampling based on weight.
Now, this is _probably_ a simple thing to do; my college number theory professor would mock me for having not "just done it" myself, but hey
- I took that class _twice_ before passing it - I don't trust my
skills in this area to get it right.
Here is a crude approach:
Say A, B and C are mirrors having bandwidth of x,y and z Mbps respectively. So lets have an array which has: A, A, A... (x times), B, B, B... (y times), C, C, C... (z times), and then generate a random number within [0, x+y+z] and use that as an index to fetch a mirror from the array.
Happy hacking, Debarshi
Debarshi Ray wrote:
Say A, B and C are mirrors having bandwidth of x,y and z Mbps respectively. So lets have an array which has: A, A, A... (x times), B, B, B... (y times), C, C, C... (z times), and then generate a random number within [0, x+y+z] and use that as an index to fetch a mirror from the array.
No, your array should be something like : ([A,x];[B,y];[C,z]) and when you pick your random number within [0,x+y+z], you parse the array decreasing your random number until it is less than the current server bandwidth (unless your array is kept static, it would be more costy to generate it (its length will be the sum of server bandwidth) than to parse a server list).
But as far as I am concerned, I would prefer giving back to the user a list of nearby servers links (which may be picked bandwidth-randomly) in which the user would choose a server by itself.
François
Do we really need an array? Lets see..
* Lets say we have three mirrors X (45 Mb), Y (5Mb), Z (3Mb)
* =>Total bandwidth = 53Mb
* Now lets say, at any given instance there are 100 requests.
* So proportionally, X should get (100/53)*45 = 85 requests. Y should get 9 requests and Z, 6 requests only.
Now
1. Generate a random no r. 2. If r<85 route to X, else if 85<r<95 route to Y else route to Z.
This will do the job.
On Thu, May 15, 2008 at 08:57:59PM +0530, susmit shannigrahi wrote:
Do we really need an array? Lets see..
Lets say we have three mirrors X (45 Mb), Y (5Mb), Z (3Mb)
=>Total bandwidth = 53Mb
Now lets say, at any given instance there are 100 requests.
we don't know this number (100) in the app...
Hence it needs to be a probability chosen out of the 53 (which we do know).
we don't know this number (100) in the app... Hence it needs to be a probability chosen out of the 53 (which we do know).
No, Wait.
I can show if this is true for 100, this will be true for _any_ number.
* As we are generating a random number within 100, the probability of each number being picked up is 1/100.
* So the probability of getting a number within 85 is always 85/100.
* Lets assume there are n requests to distribute.
* Whatever be the value of n, the random number's probability of being in range <85 is 85/100.
* So out of n requests, n*85/100 requests will always be routed to X irrespective of the value n.
* Lets say n=10..So X gets 10*85/100 = 8.5 requests.
* Lets say n=27 so X gets 27*85/100 = 22.95 requests.
* If n=1, then the probability that X will get that particular requests is 85%.
Correct?
susmit shannigrahi wrote:
we don't know this number (100) in the app... Hence it needs to be a probability chosen out of the 53 (which we do know).
No, Wait.
I can show if this is true for 100, this will be true for _any_ number.
- As we are generating a random number within 100, the probability of each
number being picked up is 1/100.
So the probability of getting a number within 85 is always 85/100.
Lets assume there are n requests to distribute.
Whatever be the value of n, the random number's probability of being
in range <85 is 85/100.
- So out of n requests, n*85/100 requests will always be routed to X
irrespective of the value n.
Lets say n=10..So X gets 10*85/100 = 8.5 requests.
Lets say n=27 so X gets 27*85/100 = 22.95 requests.
If n=1, then the probability that X will get that particular requests is 85%.
Correct?
Problem with this is that you never know n :)
Problem with this is that you never know n :)
Hmm..let me think again..
Problem with this is that you never know n :)
Why do we need to know n? Whenever a request comes in, 85% chance is that it will go to X. This is individually true for all requests. So totally 85% req will go to X.
can it be done like tihis ! it already detects that I am in india and routes me to nearest mirror .why let the user choose among the three mirror so that if he gets a 421 he can switch to another one .(or may be suggested another one ) .I am talking of get fedora part only ..yum is working fine .
just a thought .
susmit shannigrahi wrote:
Do we really need an array? Lets see..
- If r<85 route to X, else if 85<r<95 route to Y else route to Z.
This is de facto an array... Your's is hardcoded (and you need a code generator and you need to generate the code it for all GeoIP possible result). Mine is build up using GeoIP results.
François
On Thu, May 15, 2008 at 9:31 PM, Francois Petillon fantec@proxad.net wrote:
susmit shannigrahi wrote:
Do we really need an array? Lets see..
- If r<85 route to X,
else if 85<r<95 route to Y else route to Z.
This is de facto an array...
Well, may be, but without any operation on stack, thereby reducing time complexity and also reducing the overhead of creating, maintaining or updating an array. Is not it? :)
Matt Domsch wrote:
On Thu, May 15, 2008 at 10:04:51PM +1200, Nigel Jones wrote:
subhodip biswas wrote:
hi ! while trying to download F9 what I face is quite frustrating ..in fact this can be faced by a lot of user in India . while wbut mirror is down (temporarily ) and other cannot handle all the load and bit torrent is quite slow . clicking on i386 direct download always gives error 412 : too many connected users . any way of avoiding this .. I am currently downloading from a mirror in USA and its quite slow.
Hi,
Going from what I know (someone might want to update me here): We currently only have 3 mirrors in India (http://mirrors.fedoraproject.org/), one is on a 45mbps link, while the other two are on 5mbps and 2mbps links.
One thing MM would benefit from would be a weighted sampler.
Right now, it collects various lists of Hosts to return (same netblock, same country, same continent, global). It shuffles (python random.shufle()) each of the lists, then concatenates them, and sends that result to the end user. However, this does not take into account that one mirror has a 45Mbps link, and another a 2Mbps; they would each get chosen "randomly".
What I need is a replacement for random.shuffle() that takes a list of tuples: (something, weight). And returns a list of somethings that was generated with a statistical sampling based on weight.
Now, this is _probably_ a simple thing to do; my college number theory professor would mock me for having not "just done it" myself, but hey
- I took that class _twice_ before passing it - I don't trust my
skills in this area to get it right.
Volunteers?
Perhaps something like the attached test program. I am new to python so be gentle.
infrastructure@lists.fedoraproject.org