MINER attentionmodel_group questions

[laxman2405]

Hi,

I am pretty much new to RESTler and wanted to explore on it. So, we have our own REST API and we gave it to RESTler and it was able to give out some vulernabilities in the APIs. I tried to replicate the same with MINER as well. When I ran the Fuzz command with MINER, this also gives same number of vulnerabilities.

So, my question in the attentionmodel_group is, there are 2 files attentio.py and generation.py. In these two files, there are multiple .npy, .pkl files used and loaded using numpy. Couple of questions on this.

1. Do these files generate automatically under the folder?
2. How do we create these files according to our API specification. Any docs that I can refer to will be helpful as it was not mentioned how to use those in the READme file.

Appreciate your help in advance!

[laxman2405]

Any update on how to do this, please?

[m0ck1ng]

Hi Laxman, sorry for the late reply.

The training module in MINER, like attention.py and generation.py, do require .pkl, .npy files. We would like to clarify that these files are automatically collected during fuzzing campaign by a collection module without any pre-positioned datasets. When fuzzing REST APIs, the test cases regarded as interesting (e.g. passing server's check, triggering errors) will be preserved and compose a dataset (c.f. traingset.pkl). The further operations then will extract .pkl, .npy from the dataset.

We suggest you read the Chapter3.2 in MINER's paper carefully, where the process of training and collection is detailed.

In addition, the file paths in MINER's codebase are the absolute paths and hardcoded in the source codes. I have no idea whether it causes any trouble.

[laxman2405]

I did run the MINER in our windows machine by changing the path from /home/MINER to our local path from windows. But still, the .pkl and .npy files are not generated. Any file that we can check to debug or find the reason behind this?

Things we did:

1. We had our own swagger json file and we generated the Compile folder from RESTler and placed the Compile folder in MINER and ran the Restler test command, the files didn't get generated. Is there any other step that we are missing here?

[laxman2405]

Any inputs on the query?
Does MINER run in windows ?

[Lawrenmeta]

Any inputs on the query? Does MINER run in windows ?

Lately I also try to explore on restler , i want to konw how could you run miner in windows , which part will be changed?

[laxman2405]

I have just changed the paths that are hard coded in the attention model python files and thought it would run on windows. The fuzz command is running, but my expectation is, after we run the fuzz command, it should generate bunch of .pkl, .npy and other files that are mentioned in those python files in the given path. But, even when run fuzz for 48hrs, it won't generate any files under that folder. So, my assumption is MINER will not run on windows. Correct me if my assumption is wrong.

[puppet-meteor]

I have just changed the paths that are hard coded in the attention model python files and thought it would run on windows. The fuzz command is running, but my expectation is, after we run the fuzz command, it should generate bunch of .pkl, .npy and other files that are mentioned in those python files in the given path. But, even when run fuzz for 48hrs, it won't generate any files under that folder. So, my assumption is MINER will not run on windows. Correct me if my assumption is wrong.

Hi, it's been more than a year since we last use MINER, so the details are already been forgot. I've forgetten whether the compile process works on windows, while the compile process must work on Ubuntu. Could you provide MINER runtime log files for your case?
BTW, why you run MINER on windows? As we fuzz a target cloud service through APIs, the tested cloud service is insensitive to the OS of the testing machine. You guys can run MINER on Ubuntu if windows machince is not necessary.

[laxman2405]

I have just changed the paths that are hard coded in the attention model python files and thought it would run on windows. The fuzz command is running, but my expectation is, after we run the fuzz command, it should generate bunch of .pkl, .npy and other files that are mentioned in those python files in the given path. But, even when run fuzz for 48hrs, it won't generate any files under that folder. So, my assumption is MINER will not run on windows. Correct me if my assumption is wrong.

Hi, it's been more than a year since we last use MINER, so the details are already been forgot. I've forgetten whether the compile process works on windows, while the compile process must work on Ubuntu. Could you provide MINER runtime log files for your case? BTW, why you run MINER on windows? As we fuzz a target cloud service through APIs, the tested cloud service is insensitive to the OS of the testing machine. You guys can run MINER on Ubuntu if windows machince is not necessary.

Yes, we did run miner on ubuntu 20.04 as well. But we modified the Compile folder and added our own Compile folder that got generated after compiling with our own swagger json file. We had our own application which was running the background in local in ubuntu and the we triggered restler fuzz command. Is it required that the application has to run on cloud and not in local?

Also, Miner is using gitlab apis in the compile folder. Can we run miner directly after we clone without any changes? Does it hit the gitlab apis and identify the bugs?

[puppet-meteor]

running apps on cloud is a more reasonable and practical way.
For your second question, I've forgetten the details. It should be the ready-to-use version.
Again, you can provide the runtime log file to see if error occurs during MINER running.

[laxman2405]

We did run the miner code as is after cloning and we get these in the logs.
network.testing.140070828668672.1.txt

[puppet-meteor]

DataDrivenChecker Works as shown in logs, so MINER's design works.
While this log is too short to see the training process.
BTW, the execution speed is pretty slow. It takes 2 mins for a sequence testing, which is different compared to our experiments.

[laxman2405]

Also, how does Miner uses attention.py and generation.py files. Do they get called automatically? Generation.py has a line to load word_to_ix.npy file as first step. Will that file be generated as part of fuzzing process?

Just by running restler fuzz command, does those python files gets called automatically or we have to manually run them ?

please help me with detailed steps as how to run miner, is it just running fuzz command? Then how does those puthon files gets called and when?

[laxman2405]

Can you please help me with those questions?

[laxman2405]

Is there any update on this issue?

[puppet-meteor]

Also, how does Miner uses attention.py and generation.py files. Do they get called automatically? Generation.py has a line to load word_to_ix.npy file as first step. Will that file be generated as part of fuzzing process?

Just by running restler fuzz command, does those python files gets called automatically or we have to manually run them ?

please help me with detailed steps as how to run miner, is it just running fuzz command? Then how does those puthon files gets called and when?

yep, attention.py and generation.py are called automatically. The related code is in MINER/restler_bin_atten/engine/engine/core/datacollect.py.
Other codes like 'MINER/restler_bin_atten/engine/engine/core/requests.py' and 'MINER/restler_bin_atten/engine/engine/core/sequences.py' will automatically call the functions in datacollect.py during fuzzing.
You can read MINER‘s paper and read codes to figure out the call logic.