The WebZappr

Get a list of urls/domains from a text file

// November 5th, 2008 // 2 Comments » // linux

I was just in need of a little script that extracts all urls from a text file. Here is the result.

sed 's/http/\^http/g' FILENAME | tr -s "^" "\n" | grep http| sed 's/[\ |\\\|\"].*//g' | sed "s/['].*//g" | sort | uniq

And as my final goal was to extract a list of domain names from the file which I can later use in my php script here is the hardcore version which gives you a copy&paste array definition of all domains found in a file.

echo -n '$domains = array ( "' ; sed 's/http/\^http/g' FILENAME | tr -s "^" "\n" | grep http| sed 's/[\ |\\\|\"].*//g' | sed "s/['].*//g" | sort | uniq | awk 'BEGIN{FS="/"}{print $3}' | cut -d . -f 2- | grep -E '^[a-z]+\.[a-z]+$' | sort | uniq | tr "\n" "," | sed "s/,/\", \"/ig" | sed 's/, \"$//ig'; echo -n ' );'

Simple way to get a diff to a server without subversion

// October 26th, 2008 // No Comments » // linux

I recently ran into the problem that I had to update the changes I made on my subversion sandbox to a shared hosting account.

In general you would just check out the revision you need or run a svn diff.

Sadly this is not only possible and you need to figure out the changed files which you need to copy to the server.

This simple little command creates a tar archive with the changed files.

Note: you should be in the path of the checked out repository.

let revision_start=196; let revision_end=225; tar cfv ../r$revision_start-$revision_end.tar `svn diff -r $revision_start:$revision_end . | grep Index | awk '{print \$2}'`