Hi there,I need wget to work through a text document full of sites, and export the site URL and the site's title into a different document. Also, it needs to run through a proxy. How would I do this, please?
here is about what might work. for i in $(cat input.txt) echo $i $( wget -qO- $i | grep title | sed 's/<[^>]*//g') >> output.txt also might make use of xargs. In my mind this is what the code is supposed to do: for each url in input, echo the url and the formated title, and append it to output.txt that is this portion: echo $i $(...) >> ouput.txt the formatting is done by using wget to write the html page to the console quietly ( the -qO- option), grep to filter for the phrase title, and the sed strips ( s for substitute, g for all or globally) the pattern of a tag ( a < followed by as many non-> chars)
