Download files from a list using wget

I have a file which contains downloading links like this:

Google.com/image2

Google.com/image3

Google.com/image4

Google.com/image5

Google.com/image6

I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...

edited Jan 15 at 12:05

Zanna

50.7k13135241

asked Jan 15 at 8:10

no name

101

add a comment |

I have a file which contains downloading links like this:

Google.com/image2

Google.com/image3

Google.com/image4

Google.com/image5

Google.com/image6

I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...

edited Jan 15 at 12:05

Zanna

50.7k13135241

asked Jan 15 at 8:10

no name

101

add a comment |

I have a file which contains downloading links like this:

Google.com/image2

Google.com/image3

Google.com/image4

Google.com/image5

Google.com/image6

I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...

edited Jan 15 at 12:05

Zanna

50.7k13135241

asked Jan 15 at 8:10

no name

101

I have a file which contains downloading links like this:

Google.com/image2

Google.com/image3

Google.com/image4

Google.com/image5

Google.com/image6

I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...

command-line scripts

edited Jan 15 at 12:05

Zanna

50.7k13135241

asked Jan 15 at 8:10

no name

101

edited Jan 15 at 12:05

Zanna

50.7k13135241

asked Jan 15 at 8:10

no name

101

edited Jan 15 at 12:05

Zanna

50.7k13135241

edited Jan 15 at 12:05

Zanna

50.7k13135241

edited Jan 15 at 12:05

Zanna

50.7k13135241

asked Jan 15 at 8:10

no name

101

asked Jan 15 at 8:10

no name

101

asked Jan 15 at 8:10

no name

101

add a comment |

3 Answers
3

active

oldest

votes

Download all files, then move them using shell globs:

#!/bin/bash

wget -i /path/to/download_list

mv s* ./s/

mv b* ./b/

-i: Read URLs from a local or external file.

You might get a warning:

mv: cannot move 's' to a subdirectory of itself.

That's fine, you can ignore it, or use find instead:

#!/bin/bash

wget -i /path/to/download_list

find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;

find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;

And with a for loop you can run in on all alphabets, script name is script.sh:

#!/bin/bash

wget -i /path/to/download_list

mkdir -p {a..z}

for l in {a..z};

do

    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 8:13

Ravexina

32.2k1483113

i want with scripts

– no name
Jan 15 at 8:19

What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

– Ravexina
Jan 15 at 8:20

if file name starting c or x or k ? write all variants?

– no name
Jan 15 at 8:23

You didn't mention that in your question, then you have to use a loop, I'll update the answer.

– Ravexina
Jan 15 at 8:25

uea ,i want to use loop

– no name
Jan 15 at 8:30

|
show 1 more comment

An addition to @Ravexina's nice answer.

Solution without a loop:

wget -i /path/to/download_list

mkdir -p {a..z}

# mv the files with rename tool

rename 's/^((.).+)$/$2/$1/' *

# clean up empty directories

find . -maxdepth 1 -name '[a-z]' -type d -empty -delete

answered Jan 15 at 9:17

RoVo

7,2111741

add a comment |

A solution to download directly into the desired folder:

# expand file to list and iterate

for path in $(<"/path/to/download_list"); do

    # get file part of path

    name=$(basename "$path")

    # use first character of name as dir

    dir=${name:0:1}

    # create dir is not exist

    mkdir -p "$dir"

    # download path directly to dir

    wget "$path" -P "$dir"

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 11:06

Pelle

27316

add a comment |

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1109848%2fdownload-files-from-a-list-using-wget%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

3 Answers
3

active

oldest

votes

3 Answers
3

active

oldest

votes

Download all files, then move them using shell globs:

#!/bin/bash

wget -i /path/to/download_list

mv s* ./s/

mv b* ./b/

-i: Read URLs from a local or external file.

You might get a warning:

mv: cannot move 's' to a subdirectory of itself.

That's fine, you can ignore it, or use find instead:

#!/bin/bash

wget -i /path/to/download_list

find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;

find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;

And with a for loop you can run in on all alphabets, script name is script.sh:

#!/bin/bash

wget -i /path/to/download_list

mkdir -p {a..z}

for l in {a..z};

do

    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 8:13

Ravexina

32.2k1483113

i want with scripts

– no name
Jan 15 at 8:19

What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

– Ravexina
Jan 15 at 8:20

if file name starting c or x or k ? write all variants?

– no name
Jan 15 at 8:23

You didn't mention that in your question, then you have to use a loop, I'll update the answer.

– Ravexina
Jan 15 at 8:25

uea ,i want to use loop

– no name
Jan 15 at 8:30

|
show 1 more comment

Download all files, then move them using shell globs:

#!/bin/bash

wget -i /path/to/download_list

mv s* ./s/

mv b* ./b/

-i: Read URLs from a local or external file.

You might get a warning:

mv: cannot move 's' to a subdirectory of itself.

That's fine, you can ignore it, or use find instead:

#!/bin/bash

wget -i /path/to/download_list

find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;

find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;

And with a for loop you can run in on all alphabets, script name is script.sh:

#!/bin/bash

wget -i /path/to/download_list

mkdir -p {a..z}

for l in {a..z};

do

    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 8:13

Ravexina

32.2k1483113

i want with scripts

– no name
Jan 15 at 8:19

What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

– Ravexina
Jan 15 at 8:20

if file name starting c or x or k ? write all variants?

– no name
Jan 15 at 8:23

You didn't mention that in your question, then you have to use a loop, I'll update the answer.

– Ravexina
Jan 15 at 8:25

uea ,i want to use loop

– no name
Jan 15 at 8:30

|
show 1 more comment

Download all files, then move them using shell globs:

#!/bin/bash

wget -i /path/to/download_list

mv s* ./s/

mv b* ./b/

-i: Read URLs from a local or external file.

You might get a warning:

mv: cannot move 's' to a subdirectory of itself.

That's fine, you can ignore it, or use find instead:

#!/bin/bash

wget -i /path/to/download_list

find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;

find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;

And with a for loop you can run in on all alphabets, script name is script.sh:

#!/bin/bash

wget -i /path/to/download_list

mkdir -p {a..z}

for l in {a..z};

do

    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 8:13

Ravexina

32.2k1483113

Download all files, then move them using shell globs:

#!/bin/bash

wget -i /path/to/download_list

mv s* ./s/

mv b* ./b/

-i: Read URLs from a local or external file.

You might get a warning:

mv: cannot move 's' to a subdirectory of itself.

That's fine, you can ignore it, or use find instead:

#!/bin/bash

wget -i /path/to/download_list

find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;

find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;

And with a for loop you can run in on all alphabets, script name is script.sh:

#!/bin/bash

wget -i /path/to/download_list

mkdir -p {a..z}

for l in {a..z};

do

    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 8:13

Ravexina

32.2k1483113

edited Jan 15 at 23:52

dessert

22.7k56398

edited Jan 15 at 23:52

dessert

22.7k56398

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 8:13

Ravexina

32.2k1483113

answered Jan 15 at 8:13

Ravexina

32.2k1483113

answered Jan 15 at 8:13

Ravexina

32.2k1483113

i want with scripts

– no name
Jan 15 at 8:19

What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

– Ravexina
Jan 15 at 8:20

if file name starting c or x or k ? write all variants?

– no name
Jan 15 at 8:23

You didn't mention that in your question, then you have to use a loop, I'll update the answer.

– Ravexina
Jan 15 at 8:25

uea ,i want to use loop

– no name
Jan 15 at 8:30

|
show 1 more comment

i want with scripts

– no name
Jan 15 at 8:19

What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

– Ravexina
Jan 15 at 8:20

if file name starting c or x or k ? write all variants?

– no name
Jan 15 at 8:23

You didn't mention that in your question, then you have to use a loop, I'll update the answer.

– Ravexina
Jan 15 at 8:25

uea ,i want to use loop

– no name
Jan 15 at 8:30

i want with scripts

– no name
Jan 15 at 8:19

What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

– Ravexina
Jan 15 at 8:20

if file name starting c or x or k ? write all variants?

– no name
Jan 15 at 8:23

You didn't mention that in your question, then you have to use a loop, I'll update the answer.

– Ravexina
Jan 15 at 8:25

uea ,i want to use loop

– no name
Jan 15 at 8:30

|
show 1 more comment

An addition to @Ravexina's nice answer.

Solution without a loop:

wget -i /path/to/download_list

mkdir -p {a..z}

# mv the files with rename tool

rename 's/^((.).+)$/$2/$1/' *

# clean up empty directories

find . -maxdepth 1 -name '[a-z]' -type d -empty -delete

answered Jan 15 at 9:17

RoVo

7,2111741

add a comment |

An addition to @Ravexina's nice answer.

Solution without a loop:

wget -i /path/to/download_list

mkdir -p {a..z}

# mv the files with rename tool

rename 's/^((.).+)$/$2/$1/' *

# clean up empty directories

find . -maxdepth 1 -name '[a-z]' -type d -empty -delete

answered Jan 15 at 9:17

RoVo

7,2111741

add a comment |

An addition to @Ravexina's nice answer.

Solution without a loop:

wget -i /path/to/download_list

mkdir -p {a..z}

# mv the files with rename tool

rename 's/^((.).+)$/$2/$1/' *

# clean up empty directories

find . -maxdepth 1 -name '[a-z]' -type d -empty -delete

answered Jan 15 at 9:17

RoVo

7,2111741

An addition to @Ravexina's nice answer.

Solution without a loop:

wget -i /path/to/download_list

mkdir -p {a..z}

# mv the files with rename tool

rename 's/^((.).+)$/$2/$1/' *

# clean up empty directories

find . -maxdepth 1 -name '[a-z]' -type d -empty -delete

answered Jan 15 at 9:17

RoVo

7,2111741

answered Jan 15 at 9:17

RoVo

7,2111741

answered Jan 15 at 9:17

RoVo

7,2111741

answered Jan 15 at 9:17

RoVo

7,2111741

add a comment |

A solution to download directly into the desired folder:

# expand file to list and iterate

for path in $(<"/path/to/download_list"); do

    # get file part of path

    name=$(basename "$path")

    # use first character of name as dir

    dir=${name:0:1}

    # create dir is not exist

    mkdir -p "$dir"

    # download path directly to dir

    wget "$path" -P "$dir"

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 11:06

Pelle

27316

add a comment |

A solution to download directly into the desired folder:

# expand file to list and iterate

for path in $(<"/path/to/download_list"); do

    # get file part of path

    name=$(basename "$path")

    # use first character of name as dir

    dir=${name:0:1}

    # create dir is not exist

    mkdir -p "$dir"

    # download path directly to dir

    wget "$path" -P "$dir"

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 11:06

Pelle

27316

add a comment |

A solution to download directly into the desired folder:

# expand file to list and iterate

for path in $(<"/path/to/download_list"); do

    # get file part of path

    name=$(basename "$path")

    # use first character of name as dir

    dir=${name:0:1}

    # create dir is not exist

    mkdir -p "$dir"

    # download path directly to dir

    wget "$path" -P "$dir"

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 11:06

Pelle

27316

A solution to download directly into the desired folder:

# expand file to list and iterate

for path in $(<"/path/to/download_list"); do

    # get file part of path

    name=$(basename "$path")

    # use first character of name as dir

    dir=${name:0:1}

    # create dir is not exist

    mkdir -p "$dir"

    # download path directly to dir

    wget "$path" -P "$dir"

done

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 11:06

Pelle

27316

edited Jan 15 at 23:52

dessert

22.7k56398

edited Jan 15 at 23:52

dessert

22.7k56398

edited Jan 15 at 23:52

dessert

22.7k56398

answered Jan 15 at 11:06

Pelle

27316

answered Jan 15 at 11:06

Pelle

27316

answered Jan 15 at 11:06

Pelle

27316

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Ask Ubuntu!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Vtgyjfy