Download files from a list using wget












0















I have a file which contains downloading links like this:



Google.com/image2

Google.com/image3

Google.com/image4

Google.com/image5

Google.com/image6


I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...






command-line scripts







share|improve this question





























    0















    I have a file which contains downloading links like this:



    Google.com/image2
    
Google.com/image3
    
Google.com/image4
    
Google.com/image5
    
Google.com/image6


    I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...






    command-line scripts







    share|improve this question



























      0












      0








      0


      1






      I have a file which contains downloading links like this:



      Google.com/image2
      
Google.com/image3
      
Google.com/image4
      
Google.com/image5
      
Google.com/image6


      I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...






      command-line scripts







      share|improve this question
















      I have a file which contains downloading links like this:



      Google.com/image2
      
Google.com/image3
      
Google.com/image4
      
Google.com/image5
      
Google.com/image6


      I want to download all of them using a script. If the name starts with 's' download this file to the s directory, if it's b, then move it to b directory...






      command-line scripts




      command-line scripts






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jan 15 at 12:05









      Zanna

      50.7k13135241




      50.7k13135241










      asked Jan 15 at 8:10









      no nameno name

      101




      101






















          3 Answers
          3






          active

          oldest

          votes


















          10
















          Download all files, then move them using shell globs:



          #!/bin/bash
          
wget -i /path/to/download_list
          
mv s* ./s/
          
mv b* ./b/




          • -i: Read URLs from a local or external file.


          You might get a warning:



          mv: cannot move 's' to a subdirectory of itself.


          That's fine, you can ignore it, or use find instead:



          #!/bin/bash
          
wget -i /path/to/download_list
          
find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;
          
find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;



          And with a for loop you can run in on all alphabets, script name is script.sh:



          #!/bin/bash
          
wget -i /path/to/download_list
          
mkdir -p {a..z}
          
for l in {a..z};
          
do
          
    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;
          
done





          share|improve this answer


























          • i want with scripts

            – no name
            Jan 15 at 8:19











          • What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

            – Ravexina
            Jan 15 at 8:20













          • if file name starting c or x or k ? write all variants?

            – no name
            Jan 15 at 8:23











          • You didn't mention that in your question, then you have to use a loop, I'll update the answer.

            – Ravexina
            Jan 15 at 8:25











          • uea ,i want to use loop

            – no name
            Jan 15 at 8:30



















          5














          An addition to @Ravexina's nice answer.



          Solution without a loop:



          wget -i /path/to/download_list
          
mkdir -p {a..z}
          
# mv the files with rename tool
          
rename 's/^((.).+)$/$2/$1/' *
          
# clean up empty directories
          
find . -maxdepth 1 -name '[a-z]' -type d -empty -delete





          share|improve this answer































            0














            A solution to download directly into the desired folder:



            # expand file to list and iterate
            
for path in $(<"/path/to/download_list"); do
            
    # get file part of path
            
    name=$(basename "$path")
            
    # use first character of name as dir
            
    dir=${name:0:1}
            
    # create dir is not exist
            
    mkdir -p "$dir"
            
    # download path directly to dir
            
    wget "$path" -P "$dir"
            
done





            share|improve this answer

























              Your Answer








              StackExchange.ready(function() {
              var channelOptions = {
              tags: "".split(" "),
              id: "89"
              };
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function() {
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled) {
              StackExchange.using("snippets", function() {
              createEditor();
              });
              }
              else {
              createEditor();
              }
              });

              function createEditor() {
              StackExchange.prepareEditor({
              heartbeatType: 'answer',
              autoActivateHeartbeat: false,
              convertImagesToLinks: true,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: 10,
              bindNavPrevention: true,
              postfix: "",
              imageUploader: {
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              },
              onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              });


              }
              });














              draft saved

              draft discarded


















              StackExchange.ready(
              function () {
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1109848%2fdownload-files-from-a-list-using-wget%23new-answer', 'question_page');
              }
              );

              Post as a guest















              Required, but never shown

























              3 Answers
              3






              active

              oldest

              votes








              3 Answers
              3






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes









              10
















              Download all files, then move them using shell globs:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mv s* ./s/
              
mv b* ./b/




              • -i: Read URLs from a local or external file.


              You might get a warning:



              mv: cannot move 's' to a subdirectory of itself.


              That's fine, you can ignore it, or use find instead:



              #!/bin/bash
              
wget -i /path/to/download_list
              
find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;
              
find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;



              And with a for loop you can run in on all alphabets, script name is script.sh:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mkdir -p {a..z}
              
for l in {a..z};
              
do
              
    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;
              
done





              share|improve this answer


























              • i want with scripts

                – no name
                Jan 15 at 8:19











              • What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

                – Ravexina
                Jan 15 at 8:20













              • if file name starting c or x or k ? write all variants?

                – no name
                Jan 15 at 8:23











              • You didn't mention that in your question, then you have to use a loop, I'll update the answer.

                – Ravexina
                Jan 15 at 8:25











              • uea ,i want to use loop

                – no name
                Jan 15 at 8:30
















              10
















              Download all files, then move them using shell globs:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mv s* ./s/
              
mv b* ./b/




              • -i: Read URLs from a local or external file.


              You might get a warning:



              mv: cannot move 's' to a subdirectory of itself.


              That's fine, you can ignore it, or use find instead:



              #!/bin/bash
              
wget -i /path/to/download_list
              
find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;
              
find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;



              And with a for loop you can run in on all alphabets, script name is script.sh:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mkdir -p {a..z}
              
for l in {a..z};
              
do
              
    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;
              
done





              share|improve this answer


























              • i want with scripts

                – no name
                Jan 15 at 8:19











              • What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

                – Ravexina
                Jan 15 at 8:20













              • if file name starting c or x or k ? write all variants?

                – no name
                Jan 15 at 8:23











              • You didn't mention that in your question, then you have to use a loop, I'll update the answer.

                – Ravexina
                Jan 15 at 8:25











              • uea ,i want to use loop

                – no name
                Jan 15 at 8:30














              10












              10








              10









              Download all files, then move them using shell globs:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mv s* ./s/
              
mv b* ./b/




              • -i: Read URLs from a local or external file.


              You might get a warning:



              mv: cannot move 's' to a subdirectory of itself.


              That's fine, you can ignore it, or use find instead:



              #!/bin/bash
              
wget -i /path/to/download_list
              
find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;
              
find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;



              And with a for loop you can run in on all alphabets, script name is script.sh:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mkdir -p {a..z}
              
for l in {a..z};
              
do
              
    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;
              
done





              share|improve this answer

















              Download all files, then move them using shell globs:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mv s* ./s/
              
mv b* ./b/




              • -i: Read URLs from a local or external file.


              You might get a warning:



              mv: cannot move 's' to a subdirectory of itself.


              That's fine, you can ignore it, or use find instead:



              #!/bin/bash
              
wget -i /path/to/download_list
              
find -maxdepth 1 -iname "s*" -type f -exec mv "{}" ./s ;
              
find -maxdepth 1 -iname "b*" -type f -exec mv "{}" ./b ;



              And with a for loop you can run in on all alphabets, script name is script.sh:



              #!/bin/bash
              
wget -i /path/to/download_list
              
mkdir -p {a..z}
              
for l in {a..z};
              
do
              
    find -maxdepth 1 -type f -iname "${l}*" -not -iname script.sh -exec mv "{}" "./${l}" ;
              
done






              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Jan 15 at 23:52









              dessert

              22.7k56398




              22.7k56398










              answered Jan 15 at 8:13









              RavexinaRavexina

              32.2k1483113




              32.2k1483113













              • i want with scripts

                – no name
                Jan 15 at 8:19











              • What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

                – Ravexina
                Jan 15 at 8:20













              • if file name starting c or x or k ? write all variants?

                – no name
                Jan 15 at 8:23











              • You didn't mention that in your question, then you have to use a loop, I'll update the answer.

                – Ravexina
                Jan 15 at 8:25











              • uea ,i want to use loop

                – no name
                Jan 15 at 8:30



















              • i want with scripts

                – no name
                Jan 15 at 8:19











              • What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

                – Ravexina
                Jan 15 at 8:20













              • if file name starting c or x or k ? write all variants?

                – no name
                Jan 15 at 8:23











              • You didn't mention that in your question, then you have to use a loop, I'll update the answer.

                – Ravexina
                Jan 15 at 8:25











              • uea ,i want to use loop

                – no name
                Jan 15 at 8:30

















              i want with scripts

              – no name
              Jan 15 at 8:19





              i want with scripts

              – no name
              Jan 15 at 8:19













              What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

              – Ravexina
              Jan 15 at 8:20







              What you are seeing in my answer is a simple script, save it in a file, make it executable, then run it. You have to change it so it suits your needs.

              – Ravexina
              Jan 15 at 8:20















              if file name starting c or x or k ? write all variants?

              – no name
              Jan 15 at 8:23





              if file name starting c or x or k ? write all variants?

              – no name
              Jan 15 at 8:23













              You didn't mention that in your question, then you have to use a loop, I'll update the answer.

              – Ravexina
              Jan 15 at 8:25





              You didn't mention that in your question, then you have to use a loop, I'll update the answer.

              – Ravexina
              Jan 15 at 8:25













              uea ,i want to use loop

              – no name
              Jan 15 at 8:30





              uea ,i want to use loop

              – no name
              Jan 15 at 8:30













              5














              An addition to @Ravexina's nice answer.



              Solution without a loop:



              wget -i /path/to/download_list
              
mkdir -p {a..z}
              
# mv the files with rename tool
              
rename 's/^((.).+)$/$2/$1/' *
              
# clean up empty directories
              
find . -maxdepth 1 -name '[a-z]' -type d -empty -delete





              share|improve this answer




























                5














                An addition to @Ravexina's nice answer.



                Solution without a loop:



                wget -i /path/to/download_list
                
mkdir -p {a..z}
                
# mv the files with rename tool
                
rename 's/^((.).+)$/$2/$1/' *
                
# clean up empty directories
                
find . -maxdepth 1 -name '[a-z]' -type d -empty -delete





                share|improve this answer


























                  5












                  5








                  5







                  An addition to @Ravexina's nice answer.



                  Solution without a loop:



                  wget -i /path/to/download_list
                  
mkdir -p {a..z}
                  
# mv the files with rename tool
                  
rename 's/^((.).+)$/$2/$1/' *
                  
# clean up empty directories
                  
find . -maxdepth 1 -name '[a-z]' -type d -empty -delete





                  share|improve this answer













                  An addition to @Ravexina's nice answer.



                  Solution without a loop:



                  wget -i /path/to/download_list
                  
mkdir -p {a..z}
                  
# mv the files with rename tool
                  
rename 's/^((.).+)$/$2/$1/' *
                  
# clean up empty directories
                  
find . -maxdepth 1 -name '[a-z]' -type d -empty -delete






                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Jan 15 at 9:17









                  RoVoRoVo

                  7,2111741




                  7,2111741























                      0














                      A solution to download directly into the desired folder:



                      # expand file to list and iterate
                      
for path in $(<"/path/to/download_list"); do
                      
    # get file part of path
                      
    name=$(basename "$path")
                      
    # use first character of name as dir
                      
    dir=${name:0:1}
                      
    # create dir is not exist
                      
    mkdir -p "$dir"
                      
    # download path directly to dir
                      
    wget "$path" -P "$dir"
                      
done





                      share|improve this answer






























                        0














                        A solution to download directly into the desired folder:



                        # expand file to list and iterate
                        
for path in $(<"/path/to/download_list"); do
                        
    # get file part of path
                        
    name=$(basename "$path")
                        
    # use first character of name as dir
                        
    dir=${name:0:1}
                        
    # create dir is not exist
                        
    mkdir -p "$dir"
                        
    # download path directly to dir
                        
    wget "$path" -P "$dir"
                        
done





                        share|improve this answer




























                          0












                          0








                          0







                          A solution to download directly into the desired folder:



                          # expand file to list and iterate
                          
for path in $(<"/path/to/download_list"); do
                          
    # get file part of path
                          
    name=$(basename "$path")
                          
    # use first character of name as dir
                          
    dir=${name:0:1}
                          
    # create dir is not exist
                          
    mkdir -p "$dir"
                          
    # download path directly to dir
                          
    wget "$path" -P "$dir"
                          
done





                          share|improve this answer















                          A solution to download directly into the desired folder:



                          # expand file to list and iterate
                          
for path in $(<"/path/to/download_list"); do
                          
    # get file part of path
                          
    name=$(basename "$path")
                          
    # use first character of name as dir
                          
    dir=${name:0:1}
                          
    # create dir is not exist
                          
    mkdir -p "$dir"
                          
    # download path directly to dir
                          
    wget "$path" -P "$dir"
                          
done






                          share|improve this answer














                          share|improve this answer



                          share|improve this answer








                          edited Jan 15 at 23:52









                          dessert

                          22.7k56398




                          22.7k56398










                          answered Jan 15 at 11:06









                          PellePelle

                          27316




                          27316






























                              draft saved

                              draft discarded




















































                              Thanks for contributing an answer to Ask Ubuntu!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid …



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function () {
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1109848%2fdownload-files-from-a-list-using-wget%23new-answer', 'question_page');
                              }
                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown







                              -

                              Popular posts from this blog

                              The Binding of Isaac: Rebirth/Afterbirth

                              Image
                              Please consider supporting The Cutting Room Floor on Patreon. Thanks for all your support! The Binding of Isaac: Rebirth/Afterbirth From The Cutting Room Floor < The Binding of Isaac: Rebirth Jump to: navigation, search This is a sub-page of The Binding of Isaac: Rebirth . Afterbirth is the first expansion pack for Rebirth and features a variety of improvements. Contents 1 Unused Graphics 1.1 Old Man Keeper 1.2 Skinless Hush 1.3 Miscellaneous Unused Hush Sprites 1.4 Generosity achievement 1.5 Afterbirth+ Room Editor 1.6 Bearded Ultra Greed 1.7 Early Ultra Greed 1.8 Unused Brownie Expression 1.9 Frail's Spinning Head 1.10 Mom's Dead Hand Alt 2 Unused Audio 2.1 Unused Dead Womb Layer 2.2 Unused Hush Battle Outro 2.3 The Stain Grunt Unused Graphics Old Man Keeper Image file: gfxuimain me...
                              Read more

                              Mario Kart Wii

                              Image
                              Please consider supporting The Cutting Room Floor on Patreon. Thanks for all your support! Mario Kart Wii From The Cutting Room Floor Jump to: navigation, search Mario Kart Wii Developer : Nintendo Publisher : Nintendo Platform : Wii Released in JP : April 10, 2008 Released in US : April 27, 2008 Released in EU : April 11, 2008 Released in AU : April 24, 2008 Released in KR : April 30, 2009 This game has unused animations. This game has unused areas. This game has unused code. This game has unused objects. This game has unused modes / minigames. This game has unused graphics. This game has unused models. This game has unused items. This game has unused music. This game has regional differences. This game has a notes page This game has a prerelease article Mario Kart Wii is the sixth game in the series (not counting Arcade editions), which intro...
                              Read more

                              Dobbiaco

                              Image
                              Dobbiaco comune ( IT ) Dobbiaco ( DE ) Toblach Dobbiaco Vecchia Localizzazione Stato   Italia Regione Trentino-Alto Adige Provincia Bolzano Amministrazione Sindaco Guido Bocher (lista civica Indipendenti-Unabhängige) dal 10-5-2015 (2º mandato) Lingue ufficiali Tedesco, Italiano Territorio Coordinate 46°44′07.35″N 12°13′20.2″E  /  46.735375°N 12.222278°E 46.735375; 12.222278  ( Dobbiaco ) Coordinate: 46°44′07.35″N 12°13′20.2″E  /  46.735375°N 12.222278°E 46.735375; 12.222278  ( Dobbiaco ) Altitudine 1 213 m s.l.m. Superficie 125,42 km² Abitanti 3 356 [2] (31-7-2017) Densità 26,76 ab./km² Frazioni Carbonin ( Schluderbach ), Santa Maria ( Aufkirchen ), Valle San Silvestro ( Wahlen ) Comuni confinanti Auronzo di Cadore (BL), Braies, Cortina d'Ampezzo (BL), Innervillgraten (AT-7), San Candido, Sesto, Valle di Casies, Villabassa Altre informazioni Cod. postale 39034 Prefisso 0474 Fuso...
                              Read more