Can you provide the pre-processing file of your dataset?

[JeetRoy97]

The data sources contain links. The sources are different websites containing different page structures. If possible, can you share the pre-processing file to extract such text data from those websites? Hoping for a positive response.

[odelliab]

Can you elaborate ? I am not sure what is requested here. From: Prasenjeet Roy ***@***.***> Sent: Wednesday, 13 April 2022 12:46 To: odelliab/HowSumm ***@***.***> Cc: Subscribed ***@***.***> Subject: [EXTERNAL] [odelliab/HowSumm] Can you provide the pre-processing file of your dataset? (Issue #1) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd — Reply to this email directly, view it on GitHub<#1>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AV46UGQVX2EXSYRTVQIHDILVE2JVXANCNFSM5TKCWMPQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[JeetRoy97]

The dataset contains title, summary, sources. So, for a QMDS task, the title is taken as query, similarly, summary as target summary, and sources as the text documents. If this is the correct interpretation, then, the sources are the weblinks. So, I am asking for scrap file so that I can gather the documents from the source links specified by you.

[odelliab]

Dear Roy, Thank you for your interest in our dataset. Your interpretation of the dataset is correct. Unfortunately, I cannot provide the scraping script as it is IBM’s proprietary. However, there exist many scraping libraries (see for example, Best Python Libraries to Perform Web Scraping | by Claire D. Costa | Analytics Vidhya | Medium<https://medium.com/analytics-vidhya/best-python-libraries-to-perform-web-scraping-9b147bdc44f4#:~:text=The%20first%20library%20and%20undoubtedly%20one%20of%20the,you%20need%20to%20do%20is%20create%20a%20spider.> ) you can use for this purpose. Odellia From: Prasenjeet Roy ***@***.***> Sent: Wednesday, 13 April 2022 16:02 To: odelliab/HowSumm ***@***.***> Cc: Odellia Boni ***@***.***>; Comment ***@***.***> Subject: [EXTERNAL] Re: [odelliab/HowSumm] Can you provide the pre-processing file of your dataset? (Issue #1) The dataset contains title, summary, sources. So, for a QMDS task, the title is taken as query, similarly, summary as target summary, and sources as the text documents. If this is the correct interpretation, then, the sources are the weblinks. ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd The dataset contains title, summary, sources. So, for a QMDS task, the title is taken as query, similarly, summary as target summary, and sources as the text documents. If this is the correct interpretation, then, the sources are the weblinks. So, I am asking for scrap file so that I can gather the documents from the source links specified by you. — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AV46UGTU4A5SV55XBSFPT3TVE3ATRANCNFSM5TKCWMPQ>. You are receiving this because you commented.Message ID: ***@***.***>

[JeetRoy97]

Hi,
Do u have a pre-processed dataset as json or csv file where the data is already extracted.

[odelliab]

Hi, Unfortunately, I cannot provide such data. As I mentioned, you can scrap the URLs quite easily. Odellia From: Prasenjeet Roy ***@***.***> Sent: Tuesday, 19 April 2022 9:43 To: odelliab/HowSumm ***@***.***> Cc: Odellia Boni ***@***.***>; Comment ***@***.***> Subject: [EXTERNAL] Re: [odelliab/HowSumm] Can you provide the pre-processing file of your dataset? (Issue #1) Hi, Do u have a pre-processed dataset as json or csv file where the data is already extracted. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: <odelliab/HowSumm/issues/1/1102152782 ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Hi, Do u have a pre-processed dataset as json or csv file where the data is already extracted. — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AV46UGRWLEBHA4GPM6KNHNTVFZIXNANCNFSM5TKCWMPQ>. You are receiving this because you commented.Message ID: ***@***.***>