"Advanced" Shodan Queries
Shodan is one of the most powerful tools for research.
However, I often find myself in a situation that I have a query with too much results.
Shodan has some documentation with basic modifiers located at:
https://help.shodan.io/mastery/working-with-shodan-data-files
However, this is once again missing functionality.
Lets say I searched for some websites with specific code/text.
The next step would be to parse it, so the data will be consumable for further work.
The big problem with web data, specifically from Shodan, is that sometimes you don't know if the data was found on http or https.
In the link provided above, there is no property that identify the data source as I want, so this is the time to dive deeper into the not documented properties.
Lets take example.com as example, and look at the IP of the server at Shodan with details:
https://www.shodan.io/host/93.184.216.34/raw
You can see circled in red at figure 1, the output that I need. The property name is "_shodan.module".
Lets try to parse using that property:
As can be seen in figure 2, this property indeed exist, it might be not exactly what i needed but it way better than not knowing at all, specifically if you are working with a large data-set.
If you noticed, in figure 2 I used "ip_str" to parse the IP of the result, while in figure 2 such property didn't exist, for some reason Shodan decided to use different property names between the website and the CLI.
There are many other hidden properties in Shodan, some mandatory, some aren't, so you could only know about them when you take a look at your data.
If you need more properties to hunt for specific results, and the basic parsing isn't sufficient for you, I would suggest first download the bigger data-set and look manually at the downloaded JSON file for hidden properties that could help you narrow down to what you need.
However, I often find myself in a situation that I have a query with too much results.
Shodan has some documentation with basic modifiers located at:
https://help.shodan.io/mastery/working-with-shodan-data-files
However, this is once again missing functionality.
Lets say I searched for some websites with specific code/text.
The next step would be to parse it, so the data will be consumable for further work.
The big problem with web data, specifically from Shodan, is that sometimes you don't know if the data was found on http or https.
In the link provided above, there is no property that identify the data source as I want, so this is the time to dive deeper into the not documented properties.
Lets take example.com as example, and look at the IP of the server at Shodan with details:
https://www.shodan.io/host/93.184.216.34/raw
Figure 1: Shodan detailed "raw" result
You can see circled in red at figure 1, the output that I need. The property name is "_shodan.module".
Lets try to parse using that property:
Figure 2: Parsing results with almost desired output
If you noticed, in figure 2 I used "ip_str" to parse the IP of the result, while in figure 2 such property didn't exist, for some reason Shodan decided to use different property names between the website and the CLI.
There are many other hidden properties in Shodan, some mandatory, some aren't, so you could only know about them when you take a look at your data.
If you need more properties to hunt for specific results, and the basic parsing isn't sufficient for you, I would suggest first download the bigger data-set and look manually at the downloaded JSON file for hidden properties that could help you narrow down to what you need.
Comments
Post a Comment