In the past I have always gotten on the terminal of a server and executed a script that did a bunch of wgets to get the CD isos for a release.
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
So I looked to crontab to run my script. But I could not see how to get crontab to run something once on the hour and then never again. Unless the last line in my script was to rm the script. Plus it will take more than an hour to download the isos. I suppose I could use 16 as the hour so it would only run it on the 16th hour of the day and that would give me a day to clear out my crontab.
Or is there a better tool? For example if I run the script with an & after it, then drop my connection will the script keep running?
On 10Nov2010 14:28, Robert Moskowitz rgm@htt-consult.com wrote: | In the past I have always gotten on the terminal of a server and | executed a script that did a bunch of wgets to get the CD isos for a | release. | | Well I want to grab the FC14 isos, and I am not home for a few days. I | can't do it in a SSH session to a server at home as I will drop my | session before all the images are downloaded. [...] | Or is there a better tool? For example if I run the script with an & | after it, then drop my connection will the script keep running?
Screen!
See "man screen". It lets you start a shell session (optionally with a distinctive session name) and detach from it. You can reconnect at any time.
I routinely do this for downloads, OS updates, long running hand-started daemons, etc.
In fact, I do it enough to use a wrapper script "scr":
http://www.cskk.ezoshosting.com/cs/css/bin/scr
which makes naming and use by name very easy:
scr FOO
makes new session named FOO. A plain:
scr
lists the sessions (like "screen -ls", but sorted and more readable).
BTW, you should also look at the "at" command for once off jobs.
But screen is more likely what you will find useful.
Cheers,
On 11/10/10, Robert Moskowitz rgm@htt-consult.com wrote:
In the past I have always gotten on the terminal of a server and executed a script that did a bunch of wgets to get the CD isos for a release.
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
So I looked to crontab to run my script. But I could not see how to get crontab to run something once on the hour and then never again. Unless the last line in my script was to rm the script. Plus it will take more than an hour to download the isos. I suppose I could use 16 as the hour so it would only run it on the 16th hour of the day and that would give me a day to clear out my crontab.
Or is there a better tool? For example if I run the script with an & after it, then drop my connection will the script keep running?
nohup or screen may be what you're looking for.
Andras
On 10 November 2010 12:28, Robert Moskowitz rgm@htt-consult.com wrote:
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
wget -b
n 11/10/2010 02:46 PM, Cameron Simpson wrote:
On 10Nov2010 14:28, Robert Moskowitzrgm@htt-consult.com wrote: | In the past I have always gotten on the terminal of a server and | executed a script that did a bunch of wgets to get the CD isos for a | release. | | Well I want to grab the FC14 isos, and I am not home for a few days. I | can't do it in a SSH session to a server at home as I will drop my | session before all the images are downloaded. [...] | Or is there a better tool? For example if I run the script with an& | after it, then drop my connection will the script keep running?
Screen!
See "man screen".
# man screen No manual entry for screen.
What do I have to install????
It lets you start a shell session (optionally with a distinctive session name) and detach from it. You can reconnect at any time.
I routinely do this for downloads, OS updates, long running hand-started daemons, etc.
In fact, I do it enough to use a wrapper script "scr":
http://www.cskk.ezoshosting.com/cs/css/bin/scr
which makes naming and use by name very easy:
scr FOO
makes new session named FOO. A plain:
scr
lists the sessions (like "screen -ls", but sorted and more readable).
BTW, you should also look at the "at" command for once off jobs.
But screen is more likely what you will find useful.
Cheers,
On 11/10/10, Robert Moskowitz rgm@htt-consult.com wrote:
# man screen No manual entry for screen.
What do I have to install????
screen
(# yum install screen)
Andras
On 11/10/2010 02:50 PM, suvayu ali wrote:
On 10 November 2010 12:28, Robert Moskowitzrgm@htt-consult.com wrote:
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
wget -b
I guess I have never really been clear about 'background'. This will continue to run even when the I drop the connection that spawned it? I seem to recall going to close a terminal window getting messages about background tasks and if I want to kill them as part of closing the window.
Plus I have not figured out how to wildcard the wget for the FC CD isos. When I tried this for FC13, the only thing (at least for a couple of mirrors I tried) the only file really in the .../fedora/releases/14/Fedora/i386/iso/ directory was the index.txt, so this would mean 5 parallel wget tasks.
On 10 November 2010 13:06, Robert Moskowitz rgm@htt-consult.com wrote:
On 11/10/2010 02:50 PM, suvayu ali wrote:
On 10 November 2010 12:28, Robert Moskowitzrgm@htt-consult.com wrote:
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
wget -b
I guess I have never really been clear about 'background'. This will continue to run even when the I drop the connection that spawned it? I seem to recall going to close a terminal window getting messages about background tasks and if I want to kill them as part of closing the window.
It is immune to hangups. You can check by running ps -uf.
Plus I have not figured out how to wildcard the wget for the FC CD isos. When I tried this for FC13, the only thing (at least for a couple of mirrors I tried) the only file really in the .../fedora/releases/14/Fedora/i386/iso/ directory was the index.txt, so this would mean 5 parallel wget tasks.
There is a subtlety here, wget supports wildcards for ftp urls but not http. Look up the man page, I think it is covered under how to specify urls. Its been a long time since I looked it up.
On 11/10/2010 03:32 PM, suvayu ali wrote:
On 10 November 2010 13:06, Robert Moskowitzrgm@htt-consult.com wrote:
On 11/10/2010 02:50 PM, suvayu ali wrote:
On 10 November 2010 12:28, Robert Moskowitzrgm@htt-consult.com wrote:
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
wget -b
I guess I have never really been clear about 'background'. This will continue to run even when the I drop the connection that spawned it? I seem to recall going to close a terminal window getting messages about background tasks and if I want to kill them as part of closing the window.
It is immune to hangups. You can check by running ps -uf.
Plus I have not figured out how to wildcard the wget for the FC CD isos. When I tried this for FC13, the only thing (at least for a couple of mirrors I tried) the only file really in the .../fedora/releases/14/Fedora/i386/iso/ directory was the index.txt, so this would mean 5 parallel wget tasks.
There is a subtlety here, wget supports wildcards for ftp urls but not http. Look up the man page, I think it is covered under how to specify urls. Its been a long time since I looked it up.
ARGH!!!! I had old scripts like this that I run all the time, but never look back at them. The following works:
wget -b -r -l1 --no-parent -nH -A.iso http://fedora.mirrors.tds.net/pub/fedora/releases/14/Fedora/i386/iso/
Only problem with this is that the wget log file is getting hugh with all the progress output. I will have to look into progress control next time....
On 11/10/2010 03:07 PM, Andras Simon wrote:
On 11/10/10, Robert Moskowitzrgm@htt-consult.com wrote:
# man screen No manual entry for screen.
What do I have to install????
screen
(# yum install screen)
Always helps to spell it right which is why this guess did not work the 1st time I tried it.
So reading up on screen, this is rather complex. I have gotten the wget -b working now.
On Thu, Nov 11, 2010 at 7:28 AM, Robert Moskowitz rgm@htt-consult.com wrote:
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
Maybe take a look at screen..
-c
On 11/10/2010 12:28 PM, Robert Moskowitz wrote:
Well I want to grab the FC14 isos, and I am not home for a few days. I can't do it in a SSH session to a server at home as I will drop my session before all the images are downloaded.
Use nohup to run the command. That will make it ignore your hanging up.
On 10Nov2010 16:19, Robert Moskowitz rgm@htt-consult.com wrote: | On 11/10/2010 03:07 PM, Andras Simon wrote: | > On 11/10/10, Robert Moskowitzrgm@htt-consult.com wrote: | >> # man screen | >> No manual entry for screen. | >> | >> What do I have to install???? | > screen | > (# yum install screen) | | Always helps to spell it right which is why this guess did not work the | 1st time I tried it. | | So reading up on screen, this is rather complex. I have gotten the wget | -b working now.
Screen's command line is rather complex. That's one reason I mostly use it via my "scr" wrapper. It makes starting a session and rejoining it very easy and makes naming them easy.
I mostly do only this:
scr # list screens and names scr FOO # create (or rejoin if it exists) screen FOO scr 3 # rejoin the third screen listed by "scr"
And you're off and running.
Then you just run your wgets inside, interactively as you might normally.
There are various other special purpose things I use screen for and then I use it directly, but scr does me 95% of the time, probably more.
Cheers,
On 10 November 2010 14:18, Robert Moskowitz rgm@htt-consult.com wrote:
Only problem with this is that the wget log file is getting hugh with all the progress output. I will have to look into progress control next time....
Try wget -b -q or wget -b -nv