Get rid of the redundant note info from the summary

[kathy-lee]

Hi there, thanks for making this great library.
I have a question about the summary of a Wikipedia page: Take https://en.wikipedia.org/wiki/South_Bruce_Peninsula as example, when I try to get its summary with this library('wikipage.summary'), it returns its summary with a prefix info as following:
"""
South Bruce Peninsula is not to be confused with the Municipality of South Bruce, Ontario

South Bruce Peninsula is a town at the base of the Bruce Peninsula of Ontario, Canada, in Bruce County between Lake Huron and Georgian Bay. It was formed on January 1, 1999, when the town of Wiarton, the village of Hepworth, and the townships of Albemarle and Amabel were amalgamated. The new municipality was created to provide necessary political representation, administrative support, and necessary municipal services on behalf of the residents.
Tourism, particularly cottage rental and providing services to visitors, is the major industry in the area. Many cottages are found along Sauble Beach (North and South).
"""
But actually the first paragraph is not part of the summary, it's additional note info, is there a way to get rid of it: "South Bruce Peninsula is not to be confused with the Municipality of South Bruce, Ontario" ?

Many thanks in advance!

[kathy-lee]

Hi @barrust , could you give some hint please? Thank you!

[barrust]

Really, there is nothing that can be done on the library's side as the summary is pulling the first section of the site. Summary is not part of the API but is really just the extraction extension. The only real solution for your case would be to do some NLP or text processing to determine if part or all is unnecessary.

Something simple might be to do something like this:

wikipedia = MediaWiki()
p = wikipedia.page("South Bruce Peninsula")

summary = []
for para in p.summary.split("\n"):
    if para and "not to be confused with" not in para:
        summary.append(para)

print("\n".join(summary))

Not perfect and doesn't support all languages (hence why it cannot be added directly to the library) but should do what you are looking for.

[kathy-lee]

Thank you for the reply, I get your point :)