Discussion: New Format

[CameronMunroe]

Just an open discussion...

Initially, this project was built to identify likely sources of "hosting" traffic to help with blocking and detecting possibly SPAM.

Over the past few weeks and months, I've wanted to make this more of an open source identification of AS Networks in general.

I think we will leave this repo intact, and maybe create a new one with the new ideals in mind.

Any thoughts on this?

[finnbear]

For better precision & accuracy, perhaps it should have more granularity than this repository:

• If AS0 contains AS1, the format should support identifying AS0 as one type (applies to IP's in AS0 except those in AS1) and AS1 as another type (applies to all IP's in AS1)
  • Would need a whitelist to do this in the current format
• If AS0 contains many IP ranges, such as some hosting and some consumer, the format should support identifying them appropriately.

Related: #742

[CameronMunroe]

For better precision & accuracy, perhaps it should have more granularity than this repository:

* If AS0 contains AS1, the format should support identifying AS0 as one type (applies to IP's in AS0 except those in AS1) and AS1 as another type (applies to all IP's in AS1)
  
  * Would need a whitelist to do this in the current format

* If AS0 contains many IP ranges, such as some hosting and some consumer, the format should support identifying them appropriately.

Related: #742

Already do this regarding blackbox the service.

We based it on the most specific announcement. For example, 8.8.8.8 is announced by Level 3 and Google. Since Google has the most specific announcement we base it upon them.

[finnbear]

blackbox the service.

I'd like to know more about that. Is it proprietary? Is this the link? Is there a way to get a more granular downloadable IP or IP range list?

[CameronMunroe]

Yes that service.

In the backend we use https://ip2asn.ipinfo.app/api/v1/8.8.8.8 and select the 0 record. This selects the most specific announcement, which is the default used for routing.

For the actual blackbox service, I don't have any published or downloadable feeds.

[finnbear]

Thanks for clarifying. My naive method was to take the union of the IP's of all blocked ASN's and put them in a set (membership in that set means blocking). Sounds like I need to replicate your "most specific" calculation.

[abdullahdevrel]

I work for IPinfo. Would using our free IP to Country ASN database work for this purpose? I understand that for ASNs like Cogent, which provide connectivity to many companies but do not actually operate an ASN database, it would not be very effective in identifying them.

However, generally speaking, if you add another metadata layer which is the country level data per ASN, you can make the hosting range declaration more granular. Let me know what you think.

[CameronMunroe]

I work for IPinfo. Would using our free IP to Country ASN database work for this purpose? I understand that for ASNs like Cogent, which provide connectivity to many companies but do not actually operate an ASN database, it would not be very effective in identifying them.

However, generally speaking, if you add another metadata layer which is the country level data per ASN, you can make the hosting range declaration more granular. Let me know what you think.

I assume this is for personal use only, and not for paid products. Correct?

[abdullahdevrel]

@CameronMunroe No, the free database is licensed under CC-BY-SA 4.0, which is a commercially permissive license that only requires public attribution, that's all.

[finnbear]

identification of AS Networks

Depending on what is meant by "identification," these GitHub repositories might also be useful. Their license is more permissive.

• https://github.com/ipverse/asn-ip (asn -> handle + description + ipv4 + ipv6)
• https://github.com/ipverse/asn-info (asn -> handle + description)

(I use these for mapping this ASN list to an IP list)

if you add another metadata layer which is the country level data per ASN, you can make the hosting range declaration more granular.

@abdullahdevrel Are you referring to the fact that there are multiple entries in that database per ASN?

All entries that I could find have the same ASN name and domain name so, while more granular, any blocking decision would have to be based on country, which doesn't seem as useful as attributing IP ranges to a Cogent customer name.

Some entries for AS174 from the IP Info IP to Country and ASN database CSV

149.14.124.106,149.14.124.106,SA,Saudi Arabia,AS,Asia,AS174,Cogent Communications,cogentco.com
205.160.110.103,205.160.110.103,SA,Saudi Arabia,AS,Asia,AS174,Cogent Communications,cogentco.com
154.54.75.80,154.54.75.87,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
154.54.61.238,154.54.61.238,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
2001:978:3::1a0,2001:978:3::1a7,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
130.117.2.93,130.117.2.94,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
154.45.64.0,154.45.64.255,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
154.54.37.37,154.54.37.37,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
149.11.82.120,149.11.82.255,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
149.11.82.0,149.11.82.111,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
2001:550:0:1000::9a36:385a,2001:550:0:1000::9a36:385a,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
154.54.21.189,154.54.21.189,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
130.117.51.107,130.117.51.136,SE,Sweden,EU,Europe,AS174,Cogent Communications,cogentco.com
120.138.35.0,120.138.35.255,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
203.222.38.9,203.222.38.31,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2402:4480:30e::,2402:4480:30e:ffff:ffff:ffff:ffff:ffff,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2001:550:0:1000::9a36:57d1,2001:550:0:1000::9a36:57d1,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
154.54.192.0,154.54.192.255,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
203.222.63.85,203.222.63.85,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
203.222.160.24,203.222.160.25,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
154.24.30.44,154.24.30.47,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2001:550:0:1000::9a18:4b81,2001:550:0:1000::9a18:4b82,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2001:550:0:1000::9a1a:4016,2001:550:0:1000::9a1a:4018,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2600:0:1:1239:203:222:40:9,2600:0:1:1239:203:222:40:9,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
203.98.195.0,203.98.195.255,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
154.18.187.0,154.18.187.255,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
154.54.140.15,154.54.140.19,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2402:4480:2:2::,2402:4480:2:2::e:ffff,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
203.222.38.99,203.222.38.101,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
154.18.2.120,154.18.2.191,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
2402:4480:2:2::d6:0,2402:4480:2:2::e0:ffff,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com
203.222.160.223,203.222.160.223,SG,Singapore,AS,Asia,AS174,Cogent Communications,cogentco.com

However, your website does offer the customer names of Cogent IP ranges:

Some entries for ASN174 from IPinfo website

Netblock	Company	Num of IPs
[100.43.22.0/23](https://ipinfo.io/AS174/100.43.22.0/23)	APOG-HOWARD	512
[102.129.145.0/24](https://ipinfo.io/AS174/102.129.145.0/24)	Internet Utilities Africa (PTY) LTD	256
[102.129.152.0/24](https://ipinfo.io/AS174/102.129.152.0/24)	Internet Utilities Africa (PTY) LTD	256
[102.129.153.0/24](https://ipinfo.io/AS174/102.129.153.0/24)	Internet Utilities Africa (PTY) LTD	256
[102.129.232.0/24](https://ipinfo.io/AS174/102.129.232.0/24)	Digital Energy Technologies Limited	256
[102.129.234.0/24](https://ipinfo.io/AS174/102.129.234.0/24)	Digital Energy Technologies Limited	256
[102.129.235.0/24](https://ipinfo.io/AS174/102.129.235.0/24)	Digital Energy Technologies Limited	256
[102.129.252.0/24](https://ipinfo.io/AS174/102.129.252.0/24)	Digital Energy Technologies Limited	256
[102.165.16.0/24](https://ipinfo.io/AS174/102.165.16.0/24)	Digital-Energy-Technologies-LTD	256
[102.165.48.0/24](https://ipinfo.io/AS174/102.165.48.0/24)	Internet Utilities Africa (PTY) LTD	256

[abdullahdevrel]

@finnbear I did not want to pitch a paid service to an OSS project, but yes, we have a paid product called IP to Company API that provides exactly what you want.

The company data is publicly available through our website, so I would say to use the free ASN + Country database we have for the first layer of data addition. Then, through community contribution, you can use our site as a reference to get granular range information like here: #742

In terms of the quality of the free database we have, it is seriously unmatched by anything out there. Our geolocation database is based on active measurement and, on the other hand, the ASN database is produced through a combination of WHOIS and BGP peering data. It is also updated daily and provides full accuracy.

[finnbear]

@abdullahdevrel thanks!

I have a question: In your opinion, if an open source project like the one proposed here used your CC-BY-SA 4.0 licensed data, would you expect all users of that project be to acknowledge IPinfo? Or just acknowledge that project?

I ask because your website says "Attribution Required" and prominently lists adding a link to IP info on a project website or social media as a means of attribution.

However, creative commons lists GPLv3 as a compatible license to BY-SA.

AFAIK, only AGPLv3 (but not regular GPLv3) would prevent unattributed use in a web-server (that only makes decisions based on IP data, and doesn't redistribute it).