Fix transform request handling from TDS #45
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Author
|
@allfro did you have any plans to merge this or are you working on some alternative fix? |
Owner
|
Hi @caolan Analyzing your merge request. I think there are a few problems though. In particular with your request to change config.py. That would break all local transforms. Also, I'm not sure how your first edit would work any different than the one-liner |
Author
|
Thanks @allfro, I appreciate the update. I'm also not sure why the one-line variant for the first edit didn't work, I think the message property didn't get properly set for some reason. |
Owner
|
No problem, Just take a look at Plume. It's a lot more performant than run-server. It's closer to being commercial grade. Requires twistd which I think is already installed using the setup script. |
The highlight function from c.m.utils is starting to get imported by lots of modules outside of the c.maltego module. So to prevent ugly circular imports it has been refactored to a more common place.
The previous fix didn't account for canari commands trying to call project_tree outside of a canari project directory. In this case, we want to look at installed packages. Thus the function now takes a package name as argument as well, and if we determine that we aren't inside a canari project directory, then we will try and search the list of installed packages. Another notably change, is that the function now handles multiple transform packages inside a single canari project. Multiple transform packages are essentially different python package (modules/directories) inside the src folder, just as distutils/setuptools defines them in setup.py. If the canari project contains multiple transform packages and a specific one hasn't been given as the argument, then it will ask the user to pick among the available packages. The returned dictionary is almost identical whether we find a transform package inside a canari project directory or installed on the system. Only the root information is missing if we are dealing with an installed package. In any case, 'pkg_name' has been added as a new value, such that we may know which transform package was 'chosen' either by the user or automatically if there was only one possibility.
Fixes: * Now has the correct help messages, and not the copy/paste from the install-package command. * Now has a -d/--dir argument instead of the copy/paste --working-dir from the install-package command. * -d/--dir only needs to be somewhere inside a canary project, and it will find the project directory by itself. If it is not supplied, then it will use the current working directory. * Will now infer the package name if it is not specified, and if a valid --dir or current working directory is used. Valid being a transform package directory such that project_tree() will return useful information. * If a package name is specified, and a canari project can't be located, then it will now look at the list of installed packages and find it there.
When trying to make a TransformDistribution from something other than a string (e.g., the actual module), it will fail as it tries to do string operations on the argument. Now it will raise an TypeError with a better description. We don't use isinstance(..., str) as that would not handle a unicode string, or any other (duck) type that would potentially fit.
We are dealing with ordinarry XML so threat it that way.
It was impossible to parse the following valid XML from Maltego, without
this fix, as it required the elements to be ordered according to how
they were defined in the c.m.m._Entity class.
---------------------------------------------------------------------------
import canari.maltego.message as msg
msg._Entity.parse('''
<Entity Type="Domain">
<AdditionalFields>
<Field Name="fqdn" DisplayName="Domain Name">paterva.com</Field>
</AdditionalFields>
<Weight>66</Weight>
<Value>paterva.com</Value>
</Entity>
''')
---------------------------------------------------------------------------
ParseError Traceback (most recent call last)
<ipython-input-2-abb8246f3cd9> in <module>()
7 <Value>paterva.com</Value>
8 </Entity>
----> 9 ''')
/usr/local/lib/python2.7/dist-packages/safedexml/__init__.py in
parse(cls, xml)
351 if field.required and field not in fields_found:
352 err = "required field not found: '%s'"
% (field.field_name,)
--> 353 raise ParseError(err)
354 field.parse_done(self)
355 # All done, return the instance so created
ParseError: required field not found: 'value'
…anari create-profile
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These are the changes I had to make to get the XML of incoming requests properly parsed and handled. Hopefully it at least points out the issues that need addressing when looking into the XML ordering bug you mentioned.