Working with FastAPI, GO cli tool, Redis and Celery - Part 1

Recently, I have been experimenting with Python and understanding how I would integrate other tools and libraries within FastAPI. Its purely a learning curiosity.?

As FastAPI?is in Python, I though it will be easier to start by integrating 3rd party Python tools, such as Sublist3r. Sublist3r is a popular subdomain enumeration tool.

Use this repository link for this exercise:

https://github.com/qmqasim99/integrate_scanning_modules/tree/sublister_submodule

I can use git’s submodule command to easily link the Sublist3r source code into my project

git submodule add https://github.com/aboul3la/Sublist3r

First, I have a simple API endpoint:

@app.get("/scan/"

async def get_scan(url: str):

??result = sublister_wrapper(url)

??return result)        

This endpoint takes a query parameter and calls a wrapper function

Its the wrapper function that actually calls the Sublist3r code:

def sublister_wrapper(url: str)

??try:

????result = run_sublister(url)

????if result:

??????print('RESULTS', result)

????return result

??except:

????print('sublister failed for {}.'.format(url))




def run_sublister(url:str):

??subdomains = sublist3r.main(url, 40, url, ports= None, silent=False, verbose= True, enable_bruteforce= False, engines="google,bing,PassiveDNS")

??return subdomains:        

run_sublister is the main function that calls the Sublist3r library with a singe URL to search for. We can tell Sublist3r which search engines to use for enumeration. We have specified Google, Bing and PassiveDNS search engines.?

When the Sublist3r library is called, it takes its time to run subdomain enumeration and return the results.?

Whole code is run using Docker. Run docker-compose up —build to create an image and run the container.

After running the app, go to https://localhost:8000/docs and try out the API by entering example.com in the url box.

You should be able to see the progress of the enumeration process in the terminal:

Now, interestingly, Sublist3r has been unstable for me and sometimes does not return results. Nevertheless, I still wanted to integrate this tool into FastAPI as a learning experiment.

This task is complete. I can make the 3rd party library work from my project without actually installing it on my local machine.