Getting Started with SpiderFoot

At some point in my past, I may or may not have done some InfoSec. As part of that role, “foot printing” a target was a tedious task. Basically, you would try to find anything and everything you could about said target, without actually engaging. When you consider the number of datasources available (Google, gmane.org, DNS, etc, etc), you can begin to see where this becomes cumbersome.

Enter SpiderFoot

The folks at SpiderFoot (http://www.spiderfoot.net/) have done A LOT of work in to automate the data gathering and assimilation process. First, take a look at this preso to more fully understand the issue and what they’ve undertaken:

Installing SpiderFoot

The install process is relatively straight forward, but has a number of little libraries you’ll need to pull down and install. The following commands can be used on an Ubuntu 12.04 install to get you up and running.

# Update the box, install some tools
sudo apt-get update
sudo apt-get install git iftop iptraf vim curl wget lighttpd python-pip python-dev swig -y
sudo apt-get install -y build-essential linux-headers-`uname -r`

# Install the needed libraries
sudo pip install netaddr
sudo pip install M2Crypto
sudo pip install cherrypy
sudo pip install mako
sudo pip install dnspython

# Clone SpiderFoot
git clone https://github.com/smicallef/spiderfoot.git

# Run it
cd ./spiderfoot
python ./sf.py 0.0.0.0:5001

Starting Your First Scan

Once installed, you should be able to point your browser to the IP of the box on port 5001 (http://<my_ip>:5001) and you’ll be presented with the following:

Click “New Scan” provide it a scan name, domain name, and tick or un-tick options as is relevant (some of them are more ‘intrusive’ than others, like open TCP port):

Finally, click Run Scan, and well, get some coffee, it’ll take a while.

Disclaimer

Something something y’all are responsible for what you do with this information, only scan things you are authorized to scan, yadda yadda…