Recon — my way.
A detailed blog post on my reconnaissance processes for web applications security testing. I always wanted to write about this subject being asked by many friends, community members, etc. but I hooked wasting a lot of hours on a Meme Channel & The Big Bang Theory TV Series.
UPDATE: I created a GitHub repository with tools from this post and personal installation guide.
Recently, some new members of the InfoSec community asked me to share my recon process. Hence, I decided to begin writing this blog and tried to include such tools and services which helps me a lot while testing and will help the readers too, for sure.
- A tool I modified.
- Visual recon
- More Assets — More findings — More win.
- Data Storage Buckets.
- Github for Recon.
- Read Every JS
- Continuous Recon
- Extra points for recon.
Whenever I get an invitation for a new program or I want to test a target, I start my recon process by using the Knockpy.
Why I use knockpy Initially? It provides me with a quick overlook of the subdomains with a response code.
Once, I found a subdomain takeover bug within 2 mins.
I ran knockpy on an old program’s in-scope asset with almost 150 bugs resolved on HackerOne. Quickly saw 404 page pointed to AWS S3 bucket and bucket were available to create. Hence, with no delay, I created the new AWS S3 bucket and uploaded a text file with the encoded filename and reported the bug and guess what? I got the bounty that too within 15 mins.
Lesson: knockpy = quick win
2. A Tool I Modified.
I use a customized tool for subdomain reconnaissance.
I used a resolver tool by Malvinsh and customized it.
Malvinsh resolver tool runs as.
Malvinsh Tools Original output.
Malvinsh tool is doing two simple processes.
- Getting the IP address of the domain/subdomain from the wordlist using HOST.
- Performing the Nmap scan.
Using its logic I created two scripts.
Caution: Do not use these scripts on programs where automatic testing is forbidden and also on the targets on which you are not allowed to.
Subdomain.rb is a lightweight script to automate tools for subdomain finding and it’s damn flexible — more tools can be added easily.
Subfinder and sublist3r results sometime overlap. I tried to run both tools separately on a particular asset and got some distinct results — that’s why I kept both tools in this script.
The script is using the following tools to get the subdomains data.
Usage: ruby subdomain.rb domain.com
I, usually, create a file and add all subdomains from the above outputs. Then, I use the sort command to delete all duplicate/overlapped subdomains from the file.
**sort wordlist uniq**
I pass the final unique subdomains file to the recon.rb
Recon.rb is another lightweight script and it is also flexible hence, more tools can be added easily.Tools included in recon.rb
- Host : Resolve the subdomain.
Nmap : Perform Port scan on subdomain
Usage: ruby recon.rb wordlist
A tool with Above functionality in the organized fashion.
Interesting blog posts
3. Visual Recon
I use the previously generated wordlist from subdomian.rb for visual recon.
I generally use the following two tools.
Interesting blog posts
4. More Assets — More findings — More win.
I go for this section after reporting 2–3 issues in a particular program and wait for the response, if I find program interesting then I try to collect as much information as I can about the target using the following services.
Shodan Basic Commands. 443.https.tls.certificate.parsed.extensions.subject_alt_name.dns_names:domain.com “hackme.tld” + internal “hackme.tld” + sandbox “hackme.tld” + Staging “hackme.tld” + Development/Dev “hackme.tld” + production/prod hackme inc Credit: @nahamsec Interesting posts: https://0xpatrik.com/censys-guide/
Get the whois information of target using whois command or use an online tool.
If the company is not using Domain Privacy Service,
you will find the host-masters email address then use that email to find other domains registered on same email address using Reverse Whois Lookup. Targets Registered legal name also can be used.
Nmap Subdomain finding.
#Finding subdomains via Nmap. nmap --script dns-brute --script-args dns-brute.domain=domain.com,dns-brute.threads=6,dns-brute.hostlist=./sub1000.lst nmap --script dns-brute --script-args dns-brute.domain=domain.com,dns-brute.threads=6,dns-brute.hostlist=./sub10000.lst nmap --script dns-brute --script-args dns-brute.domain=domain.com,dns-brute.threads=6,dns-brute.hostlist=./sub100000.lst nmap --script dns-brute --script-args dns-brute.domain=domain.com,dns-brute.threads=6,dns-brute.hostlist=./sub1000000.lst #Wordlist : 1. sub1000.lst - https://drive.google.com/open?id=0B0h-Dh0Oss1zOGFqVVl1bTVpdWc 2. sub10000.lst - https://drive.google.com/open?id=0B0h-Dh0Oss1zTTdmQnZsQ0JCYzA 3. sub100000.lst - https://drive.google.com/open?id=0B0h-Dh0Oss1zdDBFT1dCc08ya0U 4. sub1000000.lst - https://drive.google.com/open?id=0B0h-Dh0Oss1zODNfSG1sbVJ1WE0 In case above links are not available : https://github.com/ehsahil/wordlists-for-nmap #Taken from http://blog.x1622.com/2016/11/subdomain-discovery-with-nmap-and.html
Facebook Crt transparency monitoring subscriptions.
Typical notification from Facebook when new asset on the same crt is available.
Interesting blog posts & tools taken from
5. Data Storage Buckets.
Common Places to find Data Storage buckets.
- CSP Headers
- Archive crawl
Tip: If a bucket is returning access denied. Try to search it on Google. There are good chances that the team has recently changed the permissions of the bucket and specific files have been indexed by Google (which are available with read permission).
If the application has file uploading functionality, then try to capture the file uploading request and see where the file is being uploaded. Sometimes you will find AWS or other data storage buckets, which cannot be detected by any other method.
If you find the bucket like upload-user-content-target-prod — try to change the prod to dev, staging, sandbox, etc.
AWS CLI — AWS CLI is useful for verifying or testing the permissions of the AWS S3 buckets, Creating Buckets and Read other buckets data. AWS Account needed to use CLI.
Bucket Finder — DigiNinja — **Great Tool for finding buckets using subdomains wordlist, can be integrated into recon.rb script but I don’t always use it.
nahamsec/lazys3 — LazyS3 is an another tool which I use almost frequently to find the staging, sandboxed, dev and production buckets.
gwen001/s3-buckets-finder — Another good tool for AWS S3 buckets.
Interesting blog post
6. Github For Recon.
Github is extremely helpful in finding Sensitive information regarding the targets. Access-keys, password, open endings, s3 buckets, backup files, etc. can be found on public GitHub repositories.
Interesting blog post
7. Read every JS.
- AWS or Other services Access keys
- AWS S3 buckets or other data storage buckets with read/write permissions.
- Open backup sql database endpoints
- Open endpoints of internal services.
Interesting blog post
Searching for the targets webpages in waybackmachine, the following things can be found.
- Old and abandoned JS files.
- Old API endpoints.
- Abandoned CDN’s Endpoints.
- Abandoned Subdomains.
- Dev & staging endpoint with juicy info in source code comments.
If you are getting 403 on a page, you can also search that 403 pages of targets in way back machine sometimes, you will find them open with helpful information.
9. Continuous Recon.
- The most essential thing in continuous recon is handling of the recon data for the future uses, for this, SecurityEscape created a tool known as Swiftness. I use Swiftness to hold all of my recon data for every target.
Swiftness- My personal websec checklist.
- I use the reminder to revisiting the targets and perform recon regularly. (Every month)
10. Extra points for recon.
Note: I have provided some data in gist because I regularly update them and those updates will be automatically available here.
Best of luck for all of your future infosec things.
If you have questions and anything about the post you want to ask me, Please contact me via twitter/fb. I’ll have my DM open.
Feedbacks and edits are welcome