In the previous article of this series, we learned about how to use theHarvester tool. theHarvester is a good tool but, it is not sufficient when it comes to Information Gathering.
theHarvester is mainly used for passive reconnaissance, Passive Recon means gathering info about the target without directly querying it. Rather we used Social media profiles, and search engines to gather information about our target. If you haven't read the article have a look at Information Gathering with "theHarvester".
Now today, we will know about Recon-ng, which is more power full and is capable of doing both passive and active recon, but it is a little bit complicated, but don't worry we will get through it.
Important Note:
All the targets used in this post are from publicly available bug bounty platforms like HackerOne, Bugcrowd, and Intigriti. So they are all open for performing recon or running Recon-ng. Before proceeding make sure you have legal rights or written consent of doing recon or running Recon-ng on your target.
Getting Started:
Now, unlike the previous article, I am not going to show you how to install the tool from scratch. Rather I would recommend you use any Pentestion OS like Kali Linux, Parrot OS, BlackArch, etc...
If you don't want to use any of these then you can go to the wiki section of the tool, and choose your preferred installation method. If you ask me I would tell go with the Docker Version of the tool.
After you have done installing, you can simply enter the following command in the terminal to get started.
recon-ng
and you should be able to see something like this:
Now to exit the tool type
exit
And one more important tip before proceeding, If you ever feel to stop the tool immediately, for any reason, you can just press Ctrl+c.
Navigating the interface:
Now, if you paid enough attention in the previous step. you should be able to see something like this:
[*] No modules enabled/installed.
Now, since we are running this tool for the first time no modules are enabled on our default workspace. From Recon-ng v5 nothing is enabled by default. If you need any modules, you have to enable it via the marketplace, which we will come later.
Now in this tool, everything is separated by workspaces, since we are running it for the first time no workspaces are created. Now, we will create a workspace tropyl for our target tropyl.com.Before doing so let's see what we can do and what options are available in the tool by running help.
help
Now, you should see something like this:
Commands (type [help|?] <topic>):
---------------------------------
back Exits the current context
dashboard Displays a summary of activity
db Interfaces with the workspace's database
exit Exits the framework
help Displays this menu
index Creates a module index (dev only)
keys Manages third party resource credentials
marketplace Interfaces with the module marketplace
modules Interfaces with installed modules
options Manages the current context options
pdb Starts a Python Debugger session (dev only)
script Records and executes command scripts
shell Executes shell commands
show Shows various framework items
snapshots Manages workspace snapshots
spool Spools output to a file
workspaces Manages workspaces
we can clearly see that to create a workspace we have to run the workspaces command. But how?
let's see by running help:
help workspaces
It should give something like this:
Usage: workspaces <create|list|load|remove> [...]
Now, I think you got the point if you need to know anything or forget anything in the future. Just run help. Here, we can see have options to create, list, load, and remove a workspace. So, we will start by creating it.
workspaces create tropyl
After the creation of the workspace, we will be immediately taken to the workspace.
Now, you are in the workspace and what we can do with being in there. Let's see the options:
help options
and it should show the options
Usage: options <list|set|unset> [...]
here we have the option to list the options, set, and unset them. So, first list all the items in the options list.
options list
Name Current Value Required Description
---------- ------------- -------- -----------
NAMESERVER 8.8.8.8 yes default nameserver for the resolver mixin
PROXY no proxy server (address:port)
THREADS 10 yes number of threads (where applicable)
TIMEOUT 10 yes socket timeout (seconds)
USER-AGENT Recon-ng/v5 yes user-agent string
VERBOSITY 1 yes verbosity level (0 = minimal, 1 = verbose, 2 = debug)
Now, the above list is a global option, we can see we can use custom Nameservers, Proxy etc.. to query our target. But, these things are used in certain scenarios and very rarely.
Seeding the database:
Now, to make your life simpler Recon-ng has a built-in database. Where you can store data and delete it when required. Let's take a look into that.
help db
it shows the available options:
Usage: db <delete|insert|notes|query|schema> [...]
Now, we can clearly see we have options to delete, insert our data with the options to run the raw query, and see the DB Schema.
So, for our target, (tropyl.com) we will store the details of the target (in our case it is a website). To store details of our target, the tools come with a default table called domains we will use that.
[recon-ng][tropyl] > db insert domains
after running the above command we will we given options to enter further details, like this:
domain (TEXT): tropyl.com
notes (TEXT):
[*] 1 rows affected.
After you have entered the details of your target, let's see how we can retrieve the details when we require it.
show domains
and it should show all our domains on the DB like this:
Recon-ng has other tables like contacts, hosts, domains, etc... to see the full list you can use the help command like this:
show help
Using Marketplace:
Now, that we have some seed data, it's time to transform that data into something more useful using the marketplace. To do so we need some modules and this is the main part of the tool, the marketplace.
So, let's begin with what's available to us:
marketplace help
Usage: marketplace <info|install|refresh|remove|search> [...]
Clearly, we can see we have info, install, refresh, remove, and search. So, now we will first search the entire marketplace
marketplace search
Giving no input to search lists all the available tools in the marketplace. Which looks something like this:
Now, if you look carefully the table contains something called, Path, Version, Status, Updated, D, K
Path: It is the command you will use to load the module each time. Now, the path is the most valuable part of this marketplace. It tells many things, like consider this path recon/companies-contacts/bing_linkedin_cache this tells it takes information from the companies table and transforms into the contacts table that's why there is companies-contacts, this thing is super useful to know how a module behaves.
Version: Version of the module.
Status: Wheather or not this tool is installed in the system.
Updated: Now, this updated show the last time the tool was updated in the official repository.
D: This is a column for dependency, this is column is to tell you whether this module requires you to install any third-party dependency for the program to function.
K: Now, the K or Keys column is there to tell you whether a module requires any API key, username/password to function correctly.
Now, that you know some basics, let's began by installing a tool. In our case, we have some info on the domain table and we want to transform it into other data. To do so we will begin our like this:
marketplace search domains-
Now, here a list of the tools that fulfill our search criteria:
Let's install whois_pocs, it is a module that mines whois data from third-party whois database for domains and allows us to query that data and pull back various kinds of information about a domain name.
So, let's go ahead and install this. But one more thing this marketplace has a smart installation feature. Like you can provide any unique string in the path and it will install the tool or all the tools with that string for you, rather than entering the full name.
marketplace install whois_pocs
Now, it should install the module and show this:
[recon-ng][tropyl] > marketplace install whois_poc
[*] Module installed: recon/domains-contacts/whois_pocs
[*] Reloading modules...
This smart install becomes powerful when we need to install multiple modules like if you need to install all modules that transform domains you can just type marketplace install domains-. This makes the tool very flexible in installing, removing, and updating modules from the marketplace.
Loading and running modules:
Now, that we have a module to start let's see what we can do with this:
help modules
Usage: modules <load|reload|search> [...]
We can see that we can load modules, reload a module and we can search a module. Searching modules is like searching the marketplace, it is very similar. So, we will look at our installed modules first by running this command:
[recon-ng][tropyl] > modules search
and it should list all the available modules like this:
[recon-ng][tropyl] > modules search
Recon
-----
recon/domains-contacts/whois_pocs
So, go ahead and load this module:
modules load pocs
Now, here you can use the full path or can use the smart search feature, like what I did above.
Now, we are in the whois_pocs module, and let's see what we can do here with the help command.
[recon-ng][tropyl][whois_pocs] > help
So, now you will see a lot of similarity between the result of what we got when we ran the help command the first time. The only difference is there are few module-specific commands like goptions, info, and others are a subset of commands that are only applicable to the module.
Now, let's see the options for this module:
options list
[recon-ng][tropyl][whois_pocs] > options list
Name Current Value Required Description
------ ------------- -------- -----------
SOURCE default yes source of input (see 'info' for details)
Now, this is a big difference between global options and module-specific options. Now, every Modulein the framework does some sort of transformation so, this SOURCE option will always be available. Because every module has a default configuration when a developer creates a module they have to give us a default source of data and that typically comes from the database.
So, we need to get some information about what this default means in the options list and what sort of transformation it does. So, let's see:
[recon-ng][tropyl][whois_pocs] > info
Name: Whois POC Harvester
Author: Tim Tomes (@lanmaster53)
Version: 1.0
Description:
Uses the ARIN Whois RWS to harvest POC data from whois queries for the given domain. Updates the
'contacts' table with the results.
Options:
Name Current Value Required Description
------ ------------- -------- -----------
SOURCE default yes source of input (see 'info' for details)
Source Options:
default SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
<string> string representing a single input
<path> path to a file containing a list of inputs
query <sql> database query returning one column of inputs
Now, this gives a lot of information about this module including some additional data of that SOURCE option. So, here we can see the default is just a database query that queries the database and pulls company names specifically and then it transforms it to another data.
Now, let's look at the other options we have, <string> we can provide it as an individual string, we can use <path> to provide the source a path, which is a path to a list of inputs, or we can provide a custom query to source data from other parts of the database.
So, it is very flexible about what information it can take in. Now let's run this module and let's see what it can find for us.
[recon-ng][tropyl][whois_pocs] > run
So, we found something like this:
----------
TROPYL.COM
----------
[*] URL: http://whois.arin.net/rest/pocs;domain=tropyl.com
[*] No contacts found.
We got no result because the whois info is protected and there is near to no contact information available for tropyl as a domain. If you use any other company as a target you could have found something useful and informational. so let's give it a try with example.com.
And you should get a big list of info. right? I am not showing you the output to respect the privacy of the persons we got.
Since Recon-ng transforms everything into different datasets you can query some informational stuff like this:
show contacts
Initially, we entered a domain name and that data is transformed into contacts now we can do some useful thing with this data.
Expanding the dataset:
Now that we have some information in our contacts table, we will now transform this dataset into something more valuable.
So, we will go ahead and search for modules that can transform our contacts dataset into something more valuable.
[recon-ng][tropyl] > marketplace search contacts-
And you should now be able to see a list of modules available.
We are going to install fullcontacts, what this module does it takes emails from the contacts table as input and searches for publicly available profiles.
[recon-ng][tropyl] > marketplace install fullcontact
And this requires an API key. So, first, register an account in fullcontact.com and get a free developer API key, then come here, and let's add the API key to our keys database in recon-ng.
Note: If you don't have a business email use inboxkitten.com
Now, that you have an API key let's add it to recon-ng.
[recon-ng][tropyl] > keys help
Manages third party resource credentials
Usage: keys <add|list|remove> [...]
That means you need to add your API key in this format
keys add fullcontact_api <api_key_secret_here>
once added we can now load the module and run our transformation. Now, don't ask how to load a module you know it. right?
well, after running the module you should run
show profiles
This will list all the profiles that were available for those email addresses. But again you will find nothing there because example.com should not have any real profiles.
If you really want to get real profiles we should use any other domain name. Go ahead in HackerOne, pick a program, and do all the things for that domain then you can query some valuable data with it.
Don't think too much just go now and find one program and transform the data with recon-ng. I am going to use Uber here and search for publicly available profiles.
Once, done we can move to the next section. Where you will learn how to analyze the collected data and export the dataset.
Analyzing and Exporting the Dataset:
Now, I hope you have a full list of contacts and profiles or maybe other things to start analyzing the data and maybe you want it to export for any reason. Everything is possible in recon-ng.
Now, one the way of analyzing the data is it use the show command which we were using. And another way is to use the SQL query to retrieve data from the database.
But do you know one secret, there another hidden way of seeing and filtering the data we got.
So, exit from your recon-ng and clear the terminal.
Now, we will first see the help section of the recon-web, to know how actually should we use it.
recon-web -h
Once you saw it I think you got how to use the tool right? but, if you haven't then just paste the command:
recon-web --host localhost
And now it should be running in port 5000 by default.
If you navigate to the url you will see a lot of 0's in the left sidebar, like this:
This is because you are in the default workspace click of the default and then select your workspace and now you should see your metrics.
Now, you should notice in the header there is one row, called Tables, and we know we only have data in the domains, profiles, and contacts tables you may have something more or less depending upon what modules you used.
If you click any of them you will notice two more rows are revealed.
By using the fields row you can filter your data, and by using the export row you can export the data in your preferred format.
Conclusion:
In this article, we've only scratched the surface of what Recon‑ng can do. There are currently over 100 modules in the Recon‑ng marketplace, each providing access to new information and opening new doors to a potential foothold.
There's also the ability to script Recon‑ng sessions, either within the framework or via an external command-line interface. This allows for further automation and virtual hands‑off information gathering. So hopefully what you've seen here and the features mentioned above motivate you to dig in and learn more about this powerful tool.
But at a minimum, I hope you've gained enough of an understanding that you can begin putting this capability to work for you immediately, making Recon‑ng yet one more tool in your red team tool belt.
Thank you for reading this article, and I hope you join me to add to your arsenal of red team tools in the series and enhance it in the future. Till then have a nice day...