Turn-off Warning messages on pybgpstream

[maxvantongeren]

Currently I'm trying to load in data using the following code:

# Filter out warning messages related to "Invalid prefix"
warnings.filterwarnings("ignore")

stream = pybgpstream.BGPStream(
            from_time="2020-07-07 00:00:00",
            until_time="2020-07-07 00:10:00 UTC",
            collectors=[],
            record_type="updates",
            filter="path {0}".format(9333)
        )

for elem in stream:
    print(elem)
    break

WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in withdrawn NLRI (parsebgp_bgp_update.c:1099)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in withdrawn NLRI (parsebgp_bgp_update.c:1099)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in announced NLRI (parsebgp_bgp_update.c:1125)
WARN: INVALID_MSG: Invalid prefix in MP_(UN)REACH_NLRI (parsebgp_bgp_update_mp_reach.c:109)

update|A|1594080063.000000|routeviews|route-views.sydney|None|None|4739|45.127.172.20|212.252.124.0/24|45.127.172.20|4739 7545 6453 34984 199333|4739:0|None|None

I only include a fraction of the warnings I get, which can't be ignored using the regular warnings package of python (I tried multiple approaches with this package). These error messages only appear when I try to load in data from 2020 and later. Data before 2019 seem to have valid prefixes according to the warning messages I get from 2020 on wards. I hope someone knows how I can turnoff these warnings internally in the pybgpstream package or how I can filter out these warnings in any other way. Thank you.

[hanskuhn]

I found a similar issue reported in 2021:

#211

A fix was commited to the main branch:

CAIDA/libparsebgp#84

I don't know what CAIDA's release cycle is so you might have to compile from source. Hopefully someone from CAIDA will be able to provide more information.

[kcclaffy]

@maxvantongeren can you give more info on what version(s) you are using? libbgpstream is using latest version of libparsebgp...

[maxvantongeren]

@kcclaffy sorry for my late response. I'm running pybgpstream on version: 2.0.4. My libbgpstream/bgpreader are on verson 2.3.0. These should be the latest versions, as I recently installed them.

[brendonj]

Looks like libbgpstream ignores all the warnings from libparsebgp, but only silences the Not-Implemented ones. I've made a small patch to also silence Invalid ones: #251. I've tested it with your sample code and it appears to work, though I can only assume it won't hide anything important or break anything down the line. If it looks good to others then I can merge this and make a new release with packages etc.

Until then you can try redirecting stderr to /dev/null to hide the messages, if you aren't using stderr for anything else.

[maxvantongeren]

@brendonj Thank you, I'll try that.