Refactor Code for Osprey Data layer

[arun-esh]

Build up the Object Oriented Design to accommodate the easy processing for multiple languages, models, and other features which might be helpful in the future.

About data files:

There are two files:

• has messages of conversations
• has the information about the Predator

Get the idea by reading the files we discussed. Here is the brief idea what is required.

Object Oriented Design

There are mainly two entities we discussed:

• messages
• conversations (has messages object as an attribute)

Some methods required:

• loader (to load the data from path given as argument)
• translate (to translate the text to target language provided as an argument)

[arun-esh]

@hosseinfani Please assign this issue to me.

[arun-esh]

November 11, 2024 (Update):

Writing down my understanding about the current tasks and next steps. If there is any discrepancy in my understanding, kindly point it out.

Message() class and Conversation() class has been created. We have conversations in the dataset. Each row is a message related to particular conversation_id.

# Message class's attributes
        idx: str,
        author_id: str,
        time: str,
        n_chars: str,
        n_words: str,
        text: str,
        tagged_predator: str

Conversation() class needs to be updated as well. At this point, I have only added the csv_loader() method, but csv is not the standard input in this project, but xml_loader() method.

My next steps would be:

• fix the Conversation() class
• along with required loader methods.

[arun-esh]

Requirements:

• Each message must have link to its prev and next message
  We might not keep the same in the codebase. This is just like a practice. But we have the code if we need in future.

@staticmethod
    def xml_loader_test(filepath):
        convs = {}
        root = ET.parse(filepath).getroot()

        for conv_el in root:
            conv_id = conv_el.attrib.get('id', 'unknown')
            conv_obj = Conversation(conv_id)
            messages = []

            for msg_el in conv_el:
                text = msg_el.findtext('text')
                author = msg_el.findtext('author')
                time = msg_el.findtext('time')
                msg_obj = Message(author_id=author, time=time, text=text, conv=conv_obj)
                messages.append(msg_obj)

            for i, msg in enumerate(messages):
                if i > 0:
                    msg.prev = messages[i - 1]
                if i < len(messages) - 1:
                    msg.next = messages[i + 1]

            conv_obj.messages = messages
            convs[conv_id] = conv_obj
        
        return convs

[hosseinfani]

Hi @arun-esh
thank you. please commit/push it to the codebase. I'll do a little bit code refactor there.