Webshield Detectors

Common Sense

In each iteration, the detectors fetch database access log data and validate the model. If a detector detects an unusual rise in traffic, the corresponding users should be blocked. Since multiple detectors are available, all of them can be used to analyze traffic in different ways.

Model – Aggressive Rise

The model defines the algorithm detectors use to identify aggressive users who are likely to be blocked.

The Aggressive Rise model works by comparing user access logs over different time periods—for example, in one-hour steps—to detect new groups of users generating the highest traffic. Each detector has a configuration variable *[DETECTOR_NAME]_INTERSECTION_PERCENT*, which specifies the overlap (in percent) between new and old groups.

If the intersection percent is greater than the configured value, we assume the groups represent the same users and the situation is normal. If the intersection percent is lower than the configured value, we assume this indicates unusual traffic and block the entire new group of users.

Additionally, the BLOCKING_WINDOW_DURATION_SEC parameter defines the time interval over which users are fetched.

---

Example

Assume the current time is 2025-01-01 02:00:00, and we have:

• BLOCKING_WINDOW_DURATION_SEC = 3600 (1 hour)
• DETECTOR_TFT_RPS_INTERSECTION_PERCENT = 10

In this case, the TFT_DETECTOR should fetch the top active users that exceed the detector’s threshold from the following two intervals:

• Group A: [2025-01-01 00:00:00 – 2025-01-01 01:00:00)
• Group B: [2025-01-01 01:00:00 – 2025-01-01 02:00:00)

The detector then calculates how many users from GroupB also exist in GroupA. If the percentage of overlapping users is less than 10%, the detector blocks all users from GroupB.

---

Currently, Aggressive Rise is the only model, and all detectors use it.

Floating Thresholds

The thresholds of detectors can be initialized with default values. WebShield is able to automatically adapt to the current situation. At each iteration, a detector updates its thresholds. The main idea is to calculate the standard deviation of the accumulated access log data.

For example, if we have 3 users with RPS values of 1, 2, and 3 respectively, the arithmetic mean is 2, and the standard deviation (1σ) is 0.82. The updated threshold is therefore 2 + 0.82 = 2.82. This means users with RPS greater than 2.82 fall into the risky group.

Detector IP_RPS

Aggregate users by IP address and calculate their RPS

NAME	VALUE	DESCRIPTION
DETECTOR_IP_RPS_DEFAULT_THRESHOLD	10	Installs the default RPS threshold
DETECTOR_IP_RPS_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_IP_RPS_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.

Detector IP_TIME

Aggregate users by IP address and calculate their cumulative response time.

NAME	VALUE	DESCRIPTION
DETECTOR_IP_TIME_DEFAULT_THRESHOLD	10	Installs the default accumulative time threshold
DETECTOR_IP_TIME_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_IP_TIME_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.

Detector IP_ERRORS

Aggregate users by IP address and calculate the number of responses finished with errors

NAME	VALUE	DESCRIPTION
DETECTOR_IP_ERRORS_DEFAULT_THRESHOLD	10	Installs the default responses error threshold
DETECTOR_IP_ERRORS_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_IP_ERRORS_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.
DETECTOR_IP_ERRORS_ALLOWED_STATUSES	[100, 101, ...]	Defines the list of response status codes ignored by WebShield

Detector TFT_RPS

Aggregate users by TFT-hash and calculate their RPS

NAME	VALUE	DESCRIPTION
DETECTOR_TFT_RPS_DEFAULT_THRESHOLD	10	Installs the default RPS threshold
DETECTOR_TFT_RPS_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFT_RPS_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.

Detector TFT_TIME

Aggregate users by TFT-hash and calculate their cumulative response time.

NAME	VALUE	DESCRIPTION
DETECTOR_TFT_TIME_DEFAULT_THRESHOLD	10	Installs the default accumulative time threshold
DETECTOR_TFT_TIME_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFT_TIME_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.

Detector TFT_ERRORS

Aggregate users by TFT-hash and calculate the number of responses finished with errors

NAME	VALUE	DESCRIPTION
DETECTOR_TFT_ERRORS_DEFAULT_THRESHOLD	10	Installs the default responses error threshold
DETECTOR_TFT_ERRORS_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFT_ERRORS_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.
DETECTOR_TFT_ERRORS_ALLOWED_STATUSES	[100, 101, ...]	Defines the list of response status codes ignored by WebShield

Detector TFH_RPS

Aggregate users by TFH-hash and calculate their RPS

NAME	VALUE	DESCRIPTION
DETECTOR_TFH_RPS_DEFAULT_THRESHOLD	10	Installs the default RPS threshold
DETECTOR_TFH_RPS_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFH_RPS_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.

Detector TFH_TIME

Aggregate users by TFH-hash and calculate their cumulative response time.

NAME	VALUE	DESCRIPTION
DETECTOR_TFH_TIME_DEFAULT_THRESHOLD	10	Installs the default accumulative time threshold
DETECTOR_TFH_TIME_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFH_TIME_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.

Detector TFH_ERRORS

Aggregate users by TFH-hash and calculate the number of responses finished with errors

NAME	VALUE	DESCRIPTION
DETECTOR_TFH_ERRORS_DEFAULT_THRESHOLD	10	Installs the default responses error threshold
DETECTOR_TFH_ERRORS_INTERSECTION_PERCENT	10	Defines, in percent, how many users from Group B also persist in Group A.
DETECTOR_TFH_ERRORS_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.
DETECTOR_TFH_ERRORS_ALLOWED_STATUSES	[100, 101, ...]	Defines the list of response status codes ignored by WebShield

Detector GeoIP

Aggregate users by city and calculate their total RPS. All users from cities with unusual traffic should be blocked. It is also possible to define a list of whitelisted cities that will be ignored by the filter.

NAME	VALUE	DESCRIPTION
DETECTOR_GEOIP_RPS_DEFAULT_THRESHOLD	10	Installs the default RPS threshold
DETECTOR_GEOIP_INTERSECTION_PERCENT	10	Defines, in percent, how many cities from Group B also persist in Group A.
DETECTOR_GEOIP_BLOCK_USERS_PER_ITERATION	100	Defines the number of users that can be blocked per check.
DETECTOR_GEOIP_PATH_TO_DB	/etc/tempesta/webshield/city.db	Defines the path to the MaxMind City GeoIP database.
DETECTOR_GEOIP_PATH_ALLOWED_CITIES_LIST	/etc/tempesta/webshield/allowed_cities.db	Defines the path to the MaxMind City GeoIP database.