Tweet
Short Developer Bio: Yogesh Ojha is a Research Software Engineer in TRG Research and Development, Cyprus where his research focuses on building solutions for Crime and Terror.As a Passionate Developer and a Hacker, Yogesh builds and maintains reNgine, an automated reconnaissance framework.He has delivered talks on IoT Security and Car Hacking at several conferences like BlackHat Europe, HITB Cyber Week Abu Dhabi, Open Source Summit, IoT Innovatech LATM, GreHack, NoConName, KazHackStan, FOSS Asia Summit, etc. When not building or breaking technologies, he spends his time with his dog Jasper.
Tool or Project Name reNgine: An automated reconnaissance engine(framework)
Short Abstract: reNgine is an automated reconnaissance engine(framework) that is capable of performing end-to-end reconnaissance with the help of highly configurable scan engines on web application targets. reNgine makes use of various open-source tools and makes a highly configurable pipeline of reconnaissance to gather the recon result.reNgine also makes it possible for users to choose the tools they desire while following the same reconnaissance pipeline, example - with reNgine you aren't limited to using sublist3r for subdomains discovery, rather reNgine allows you to combine multiple tools like sublist3r, subfinder, assetfinder, and easily integrate them into your reconnaissance pipeline. The reconnaissance results are then displayed in a beautiful and structured UI after performing the co-relation in the results produced by these various tools. The developers behind reNgine understand that recon result most often is overwhelming due to the humongous data, so that’s why reNgine also comes with advanced query lookup using natural language operators like and, or and not. Imagine, doing recon on facebook.com and filtering the results like http_status!404&page_title=admin|page_title=dashbo ard&content_length>0&tech=phporseverity=critical|severity=high&vulnerability_titl e=xss|vulerability_title=cve-1234-xxxxreNgine’s flexibility to easily incorporate any existing open-source tools and with advanced features like configurable scan engines, parallel scans, advanced query lookup on recon results, instant notification about the scan, scheduled scans, etc, separates reNgine from any other recon frameworks. reNgine can be used for both reconnaissance and actively monitoring the targets.URL to any additional information: Official Documentation: https://rengine.wiki reNgine v0.5 Major Update with Vulnerability Scan and Advanced recon Lookup Trailer and Demo: https://www.youtube.com/watch?v=DSOS_dkorBMreNgine release Trailer: https://www.youtube.com/watch?v=u8_Z2-3-o2MreNgine Development Timeline Video Trailer: https://www.facebook.com/10000176436...1638639238246/reNgine featured on Portswigger’s The daily Swig: https://portswigger.net/daily-swig/r...or-pen-testers reNgine community review: https://twitter.com/Jhaddix/status/1286547230078275585 https://twitter.com/ITSecurityguard/...58400926543879 https://twitter.com/ojhayogesh11/sta...21166811471872 https://twitter.com/search?q=https%3...rc=typed_query
Detailed Explanation of Tool: reNgine is an advanced reconnaissance framework for web application targets that uses various existing open-source tools to achieve this. The idea for reNgine came when I was bored during the lockdown and had nothing better to do. Back then I was working as a Security Analyst and my day job was to perform penetration testing on web applications. While I enjoyed my job, I hated performing recon on these targets because in almost all the cases the recon steps were pretty similar.
Except for certain cases, the recon steps I read, I performed, I saw others doing, were very similar. Same usage of tools, same usage of options/parameters/tuning. But I was bored with this recon methodology because, at times I needed the recon results to be saved in a structured way, come back the next day, and still do the analysis without wasting my yet another day on recon.
Also, since I had a day job, I used to do bug bounty during the night, and obviously, my office would fire me right away if I performed recon on bug bounty targets during my office hours, so also was looking for something that could help me schedule the scans on those targets, something like performing a scan every midnight, or lineup 100 scans on the pipeline and scan these targets one step at a time.
The recon results are very humongous on larger targets, and very difficult to search or find the specific results quickly. This was due to the reason that existing frameworks (open-source) had no ability to store the results on DB, almost all used text as output, and obviously, this wasn’t going to be helpful unless you write extremely complex greps.
So, I went on to create one for myself and named it reNgine, abbreviated for reconnaissance engine. Why Engine? It is because reNgine has the ability to customize the scan engines. These engines are Yaml based configurations, you can add, remove or customize them.
So what is reNgine and how it solved the problems that no other recon frameworks were providing?
One of the most impressive features of reNgine is that it makes use of something called Scan Engines, these engines are highly configurable and allow you to choose the tools you like, configurations you like, example so you are not limited to using subfinder for subdomain gathering, you can use multiple of them, as many as you want. How difficult is it to choose tools? Very simple, just add the tool name in YAML configuration and you’re good to go, reNgine will take care of the rest.This scan engine allows you to fine-tune the tools and perform scans in a much-advanced way. These scan engines have one to many relationships with the targets, meaning, you can define one scan engine, let’s say ‘Defcon Scan’ that does Subdomain Discovery at 100 threads, grab screenshots at 50 threads, and also performs vulnerability scan. Now, once this scan engine is defined, you can use it against n number of targets without the need to modify and fine-tune the parameters every once in a while.
Sample Scan Engine Configuration:
subdomain_discovery: uses_tool: [ subfinder, sublist3r, assetfinder, oneforall ] thread: 10 wordlist: default amass_config: config_short_name subfinder_config: config_short_name
port_scan: ports: [ top-100 ] exclude_ports: null thread: 10
visual_identification: port: xlarge thread: 2 http_timeout: 3000 screenshot_timeout: 30000 scan_timeout: 100
dir_file_search: extensions: [ php,asp,aspx,txt,conf,db,sql,json ] recursive: false recursive_level: 1 thread: 100 wordlist: default
fetch_url: uses_tool: [ gau, hakrawler ] intensity: aggressive
vulnerability_scan: concurrent: 10 template: all severity: all
excluded_subdomains: - test.rengine.wiki - hello.test.com
This configuration and finetuning can be used against n targets.
The result of this recon is then stored in DB for co-relation.
Technology Stack:
reNgine uses the following technology stack:Web Framework: DjangoDatabase: PostgresDistributed Message Broker: RedisAsync Tasks and Scheduling Scans: Celery and Celery-beat
Redis acts as a message broker between Django and Celery.Containerized everything by Docker
reNgine has a dashboard-like UI, which makes it easy to co-relate the recon results.Example: https://user-images.githubuserconten...087b2b48d3.pnghttps://user-images.githubuserconten...d626127d88.png The purpose of creating the dashboard-like UI was so that one can easily filter the recon results like, “Hey, I quickly want to filter a subdomain that has admin or dashboard in page title, and also has HTTP status as 200”. With the existing recon frameworks, this was quite impossible. reNgine’s dashboard makes it very easy to filter such use cases. Example: https://camo.githubusercontent.com/2...795f322e706e67
Key Features of reNgine:
Perform Recon:
- Subdomain Discovery
- Ports Discovery
- Endpoints Discovery
- Directory Bruteforce
- IP Discovery
- CNAME discovery
- Subdomain Takeover Scan
- Run multiple scans in parallel, running multiple scans is very simple, select n targets, choose the scan engine, and initiate the scan. reNgine and celery will take care of the rest.
- Run Clocked Scans (Run reconnaissance exactly at X Hours and Y minutes)
- Run Periodic Scans (Runs reconnaissance every X minutes/hours/days/week)
- Perform Vulnerability Scan using Nuclei and get notified when a vulnerability is discovered
- Send scan related notifications to Slack or Discord
- Perform Advanced Query lookup using natural language alike and, or, not operations
Example: https://user-images.githubuserconten...7e087c1a26.png
Upcoming Features:
Scan Comparision
Comparision of the scans performed on the target, to find out how many new vulnerabilities have been discovered since the last scan, how many new subdomains have been discovered since the last scan, etc. (Under Development)Interesting Subdomains Discovery
reNgine will discover the interesting subdomains based on the HTTP status, content length, and page title. For example, imagine the time saved by reNgine if reNgine tells you that, Hey admin.facebook.com is an interesting subdomain you might want to look up, now this is depended upon, HTTP status, content length, and many more factors (Under Development)
Source Code: https://github.com/yogeshojha/rengine
Target Audience:
The targeted audience is both Offence and Defence on Web application Security.
The audience on the offense can use reNgine to perform active reconnaissance and gather more information about their next penetration testing target. This information includes but not limited to subdomains, ip address associated with it, endpoints, visual reconnaissance screenshot gathering, ports scan, and vulnerability scan as well.
And, the audience on defense can learn how to use reNgine to perform periodic scans on their (Intra/Extra)net web services, run the periodic open-source-powered vulnerability scanner, and get notified instantly when a vulnerability is identified.The beauty of reNgine is that, with minimal penetration testing and security experience, one can run the entire reconnaissance and gather the result so that it is well suited for both offense and defense.
As the purpose of this demo lab would be to demonstrate the capabilities of reNgine, the demo would be outlined in such a way that it can be well received by the audience of both the offense and defense sides.
reNgine is something I have worked really hard, spent countless nights working on it. Within a very short period of time, reNgine became one of the popular reconnaissance tools. Presenting this to fellow hackers will certainly gather new ideas on making reNgine a more advanced reconnaissance tool, which is one of the major reasons why I wish to present this to Defcon.
On the other hand, presenting this to Defcon will foster the open-source and hacker culture as I will explain about the in and out of reNgine and hopefully bring in many developers to contribute to reNgine as well.
Also, I plan to announce a major update in reNgine during Defcon, which I believe will bring innovation and excitement among the attendees as well. And of course, Defcon is the right platform to make everyone aware of the updates, advancements, and new features of reNgine.