Enable Unrestricted LLM Output with Parsing + Add One-Shot Anomaly Detection Support

[scherkao31]

This PR introduces several changes :

• Optional Unrestricted Token Output: Introduced a new boolean option to control whether the LLM is restricted to numeric tokens.
• parse_anomaly_response Function: Added a new function to process the more flexible LLM output. This ensures that we can still extract the necessary anomaly information even when the output is not strictly numerical.
• One-Shot Anomaly Detection: Integrated the ability to provide a single example of a normal sequence during the detection process, enabling one-shot.

[sarahmish]

Thanks @scherkao31! Great features to add! I have a few minor comments.

[sarahmish]

I'm wondering if we should have it more generalized, regardless of normal or not.

if X.ndim < 2:
    return _as_string(X)
else:
...

What do you think?

[sarahmish]

reformat docstrings to keep arg name and description on separate lines:

    """Extract anomalies from LLM response.

    Parse a list of lists of LLM responses to extract anomaly
    values and format them as strings.

    Args:
        X (List[List[str]]):
            List of lists of response texts from the LLM in the format
            "Answer: no anomalies" or "Answer: [val1, val2, ..., valN]".
    
    Returns:
        List[List[str]]: 
            List of lists of parsed responses where each element is either
            "val1,val2,...,valN" if anomalies are found, or empty string if 
            no anomalies are present.
    """

[sarahmish]

first line of docstrings should be shorter. I already have a description mentioned in my comment above

[sarahmish]

I would remove type checking here since it's a private function. I also recommend removing comments that are not necessary.

For private functions, we normally make them start with _, so that would be

def _parse_single_response(text):

[sarahmish]

you still have str typing in your implementation

[sarahmish]

Thanks @scherkao31 for making the changes! Some previous comments were not addressed so I kept them as unresolved.

In addition, I want to note:

• mistral_detector should be modified to set restrict_tokens to True.
• the PR is missing the hyperparameter settings for MSL
• do you plan include hyperparameters of Yahoo?

[sarahmish]

update d3-ai-orion to sintel-sigllm

[sarahmish]

update orion/data to sigllm/data

[sarahmish]

test_size argument doesn't make sense in load_normal. I would remove it. please make sure the code, docstrings, and function arguments are not using this variable.

[sarahmish]

I would not make this a function argument, but rather I would check if start or end are of integer or timestamp type. See comment below.

[sarahmish]

couple of things

• we don't know the exact name of the timestamp column, so you should use the variable.
• the user typically is not keen on specifying so many parameters. we can check if start or end are integers then we can use them to slice. Otherwise, assume that they are timestamps.
• check if the edge cases start=None but end has a value works and vice versa.

if start is not None or end is not None:
    if data.index.isin([start, end]):
        # If start and end are indices
        data = data.iloc[start:end]
    else:
        # If start and end are timestamps
        mask = (data[timestamp_column] >= start) & (data[timestamp_column] <= end)
        data = data[mask]

[sarahmish]

you still have str typing in your implementation

[sarahmish]

first line of docstrings should be shorter. I already have a description mentioned in my comment above

[sarahmish]

I still don't see a change to mistral_detector

[sarahmish]

typo permorm -> perform