Suggestion of other known Yara repositories

[mlodic]

Hello,

first all of all thank you very much for this project, this is really something super useful! :)

I am the author and maintainer of a project known as IntelOwl where we embedded a list of known Yara rule repositories. In IntelOwl the users can analyze their files with these embedded rules, without having the hassle to extract and manage these rules by hand.

Excluding the Yara Community Repository that I get it is too noisy, these are the other repositories that I included in IntelOwl and that are not included in yara-forge:

• https://github.com/embee-research/Yara
• https://github.com/elceef/yara-rulz
• https://github.com/intezer/yara-rules
• https://github.com/stratosphereips/yara-rules
• https://github.com/sbousseaden/YaraHunts
• https://github.com/InQuest/yara-rules
• https://yaraify-api.abuse.ch/yarahub/yaraify-rules.zip

I would like to ask you whether you have already evaluated them or not. I hope it helps.

Keep up the great work! :)
Matteo

[Neo23x0]

Hi @mlodic,

I think I looked at most or all of these repos and didn't include them because they contained rules that have a high impact on performance like this one by @embee-research:

I think the base64 regex isn't even necessary to successfully detect the threat.

or look risky and I don't know if my yaraQA checks would catch it, like this one

I like his other rules but some are really scary.
I will try to integrate them on a separate branch and see if my yaraQA script can take care of the inefficient and risky rules.

Thanks

[embee-research]

hey @Neo23x0, there are definitely questionable rules in my main yara repo. The repo purpose was more for learning and as a demonstration for learners of what “could” be included in a yara.

In saying that, I believe there are some good and reasonably performant rules in there.

If I can move the “good” rules over into a separate repo, would you be open to including them into yara forge?

[Neo23x0]

Hi @embee-research,

Yes, I've seen the other good rules. I really like the work you do.
I already did the following:

1. included your repo in a PR that'll add more repos to YARA Forge : https://github.com/YARAHQ/yara-forge/pull/3/files
2. sent you a PR to fix some issues upstream : Fixed issues with rules upstream embee-research/Yara-detection-rules#3

The yaraQA quality checks will do the rest and move rules with issues out of the "core" set into the "full" set.

I'll also do some false positive checks with the rules of these three (or more) new repos today.

[Neo23x0]

@mlodic : I looked at the second repo by @elceef and saw this:

I believe that people who extensively use regex have no idea how much faster string matching would be. I'm going to rewrite that rule for you to see what I mean.

rule OLE2_AutoOpen_Reversed_Payload
{
	meta:
		description = "Detects suspiciously reversed payloads in OLE2 objects with auto-open macros"
		author = "marcin@ulikowski.pl"
		date = "2021-12-01"
		modified = "2023-12-21"

	strings:
		$auto_open1 = /(auto|document|workbook)_?close/ wide ascii nocase
		$auto_open2 = /(auto|document|workbook)_?open/ wide ascii nocase
		
		$s_http1 = "//:ptth" wide ascii
		$s_http2 = "//:sptth" wide ascii
		$s_programdata1 = "\\ataDmargorP\\\\:C" wide ascii nocase
		$s_programdata2 = "\\ataDmargorP\\:C" wide ascii nocase
		$s_windows1 = "\\swodniW\\\\:C" wide ascii nocase
		$s_windows2 = "\\swodniW\\:C" wide ascii nocase

	condition:
		uint32be(0) == 0xd0cf11e0
		and filesize < 1MB
		and 1 of ($auto_open*)
		and 1 of ($s*)
}

here as well :
It's better to have two regular expressions, each of them with a solid and long anchor, instead of two variations inside of the brackets.

[Neo23x0]

@intezer is freaking me out with that private rule stuff, which makes rule handling a pain in the ###

It's so .... unnecessary.
I won't include the rules unless someone reworks this and makes every rule stand on its own, which shouldn't be a problem.

[Neo23x0]

And then the unnecessary use of the ELF module, like in this example in @stratosphereips repo:

https://github.com/stratosphereips/yara-rules/blob/2a4033559beac61421bbf5192c320e415bde0287/malware/linux_ddos_interplanetarystorm_p2p.yar#L1

Importing "elf" has a huge impact on performance.
We told the guys from ReversingLabs about a similar issue earlier this year and they took some measurements:
reversinglabs/reversinglabs-yara-rules#6

[Neo23x0]

and the @InQuest rules contained things like this:

Maybe I'll include them at a later stage, when I'm sure that my yaraQA tool would catch these things and sort those rules out.

[mlodic]

thank you very much for your analysis, very appreciated :)

If you are ok with that, I'll integrate this project in IntelOwl so the people using that tool can easily leverage your work here.

[Neo23x0]

No problem at all