Skip to content

zakdances/housing-element-shapefiles

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
counties
counties
 
 
src
src
 
 
.gitignore
.gitignore
 
 
California_Zip_Codes.geojson
California_Zip_Codes.geojson
 
 
README.md
README.md
 
 
_README.md
_README.md
 
 
count.py
count.py
 
 
count_v2.py
count_v2.py
 
 
export_shapefile_from_geojson_file.py
export_shapefile_from_geojson_file.py
 
 
extract.py
extract.py
 
 
extract_aws.py
extract_aws.py
 
 
filter.py
filter.py
 
 
find_intersection_from_server.py
find_intersection_from_server.py
 
 
find_missing_outputs.py
find_missing_outputs.py
 
 
generate_readme_summary.py
generate_readme_summary.py
 
 
generate_shapefiles.py
generate_shapefiles.py
 
 
list_no_hcd.py
list_no_hcd.py
 
 
main.py
main.py
 
 
requirements.txt
requirements.txt
 
 
schema.json
schema.json
 
 
schema_for_doc_metadata.json
schema_for_doc_metadata.json
 
 
sheets.py
sheets.py
 
 
thumbnail.py
thumbnail.py
 
 
update_doc_metadata.py
update_doc_metadata.py
 
 

Repository files navigation

Bounty Project for CA Yimby bounty

Shapefiles from the housing element documents of all cities that are members of ABAG, SACOG and SCAG. (This project satisfies 2 bounties)

(shapefile path: counties/{county name}/cities/{city name}/output/{document name}/misc)

How this project was created:

  1. I generated a list of incorporated cities along with associated planning agency (SACOG, ABAG, etc), county, and downloaded links to housing element PDFs.
  2. Geojson parcel data from each county website along with cleanup, normalization, and transfer to my database.
  3. Download each PDF, use machine learning to extract the data. Then do a second pass to clean up the (very messy) data. Generate metadata (page count, thumbnail, etc). Transfer all data to my database.
  4. Created endpoints to query database from the UI.
  5. Extra: Created web UI viewer using nextjs and react.

Summary explained

👇 Columns explained 
* city - name of city
* documents - number of documents city has publicly available for download
* tables - number of tables found in the documents
* apns - number of APNs (assessor parcel numbers) found in the tables
* parcels - parcel data found for APNs. Ideally, this number should be very close to the previous column.
* agency - regional planning organization this city is a member of
* county - county the city resides in
* link - link to relevant folder in this repo. There you can find shapefiles and other output.

⬇️ START Summary (Google Sheets version here)

ABAG

County Municipality Sources Tables APNs Link
Alameda Berkeley 3 793 837 link
Alameda San Leandro 3 820 19 link
Alameda Dublin 3 821 21 link
Alameda Hayward 4 2604 44 link
Alameda Emeryville 3 1818 56 link
Alameda Alameda 3 385 57 link
Alameda Newark 2 243 69 link
Alameda Piedmont 3 1276 77 link
Alameda Union City 3 606 264 link
Alameda Livermore 4 2198 273 link
Alameda Alameda County 1 0 349 link
Alameda Pleasanton 3 470 890 link
Alameda Albany 3 1103 987 link
Alameda Fremont 3 744 1085 link
Alameda Oakland 3 961 3173 link
Contra Costa Hercules 3 945 0 link
Contra Costa San Ramon 3 957 27 link
Contra Costa Pleasant Hill 2 327 71 link
Contra Costa Clayton 3 336 106 link
Contra Costa Martinez 1 0 110 link
Contra Costa Pittsburg 1 278 120 link
Contra Costa Contra Costa County 1 0 131 link
Contra Costa Pinole 2 1242 138 link
Contra Costa Moraga 4 1138 148 link
Contra Costa El Cerrito 3 448 225 link
Contra Costa Brentwood 3 1662 232 link
Contra Costa San Pablo 2 192 279 link
Contra Costa Richmond 3 234 501 link
Contra Costa Danville 3 900 514 link
Contra Costa Lafayette 2 847 627 link
Contra Costa Oakley 5 1830 655 link
Contra Costa Antioch 3 2790 747 link
Contra Costa Concord 3 3161 750 link
Contra Costa Orinda 4 1271 2007 link
Contra Costa Walnut Creek 5 2288 2322 link
Marin Corte Madera 4 1427 37 link
Marin Novato 2 235 40 link
Marin Belvedere 2 361 49 link
Marin Ross 2 152 54 link
Marin Larkspur 2 351 75 link
Marin Marin County 1 0 149 link
Marin Tiburon 3 507 187 link
Marin Mill Valley 1 519 244 link
Marin Fairfax 2 614 319 link
Marin San Anselmo 2 585 367 link
Marin San Rafael 3 1789 564 link
Marin Sausalito 3 773 2451 link
Napa American Canyon 2 410 0 link
Napa Napa County 1 0 1 link
Napa St. Helena 1 0 5 link
Napa Yountville 3 396 24 link
Napa Napa 2 501 51 link
Napa Calistoga 3 264 57 link
San Francisco San Francisco County 0 0 56 link
San Francisco San Francisco 5 1845 9770 link
San Mateo Broadmoor 0 0 0 link
San Mateo Half Moon Bay 0 0 0 link
San Mateo Portola Valley 1 459 2 link
San Mateo Belmont 3 666 53 link
San Mateo Pacifica 2 42 60 link
San Mateo Colma 3 805 73 link
San Mateo Hillsborough 3 4394 82 link
San Mateo Burlingame 2 304 99 link
San Mateo Foster City 3 1372 111 link
San Mateo Atherton 3 1660 129 link
San Mateo Woodside 3 1104 131 link
San Mateo San Bruno 3 1560 146 link
San Mateo East Palo Alto 3 729 152 link
San Mateo Redwood City 3 707 153 link
San Mateo Daly City 2 266 182 link
San Mateo Millbrae 3 3068 319 link
San Mateo San Mateo County 1 0 465 link
San Mateo South San Francisco 3 224 483 link
San Mateo Menlo Park 3 2748 696 link
San Mateo San Mateo 3 2599 777 link
San Mateo San Carlos 3 748 1073 link
San Mateo Brisbane 5 2801 1262 link
Santa Clara Santa Clara County 1 0 22 link
Santa Clara Morgan Hill 3 1108 25 link
Santa Clara Milpitas 3 129 39 link
Santa Clara Cupertino 2 342 57 link
Santa Clara Gilroy 3 1802 164 link
Santa Clara Santa Clara 3 407 209 link
Santa Clara Los Altos Hills 4 884 308 link
Santa Clara Monte Sereno 3 394 336 link
Santa Clara Sunnyvale 2 309 378 link
Santa Clara Palo Alto 2 341 480 link
Santa Clara Los Gatos 4 2056 576 link
Santa Clara San Jose 2 383 610 link
Santa Clara Los Altos 3 322 706 link
Santa Clara Saratoga 3 866 752 link
Santa Clara Mountain View 4 637 754 link
Santa Clara Campbell 3 1 941 link
Solano Dixon 3 574 19 link
Solano California Forever 0 0 51 link
Solano Suisun City 3 568 84 link
Solano Solano County 1 0 102 link
Solano Vallejo 0 0 102 link
Solano Fairfield 3 555 166 link
Solano Rio Vista 3 549 224 link
Solano Vacaville 3 4287 491 link
Solano Benicia 3 1391 993 link
Sonoma Healdsburg 2 690 9 link
Sonoma Cloverdale 3 425 49 link
Sonoma Sonoma County 1 0 49 link
Sonoma Cotati 3 781 67 link
Sonoma Rohnert Park 3 573 85 link
Sonoma Petaluma 3 754 118 link
Sonoma Sonoma 3 598 119 link
Sonoma Windsor 3 283 166 link
Sonoma Sebastopol 3 412 184 link
Sonoma Santa Rosa 3 899 3379 link

SACOG

County Municipality Sources Tables APNs Link
Sacramento Citrus Heights 4 1021 141 link
Sacramento Elk Grove 4 922 470 link
Sacramento Folsom 4 620 1215 link
Sacramento Galt 5 771 395 link
Sacramento Isleton 4 585 23 link
Sacramento Rancho Cordova 4 612 185 link
Sacramento Sacramento 4 1749 7071 link
Sacramento Sacramento County 1 0 2471 link
Sutter Live Oak 2 292 339 link
Sutter Sutter County 1 0 41 link
Sutter Yuba City 3 1188 450 link
Yolo Davis 4 0 259 link
Yolo West Sacramento 4 499 576 link
Yolo Winters 3 1282 347 link
Yolo Woodland 6 1389 801 link
Yolo Yolo County 1 0 172 link
Yuba Marysville 4 1874 596 link
Yuba Wheatland 2 131 106 link
Yuba Yuba County 1 0 334 link

SCAG

County Municipality Sources Tables APNs Link
Imperial Brawley 5 1169 60 link
Imperial Calexico 2 805 29 link
Imperial Calipatria 6 1015 144 link
Imperial El Centro 3 821 194 link
Imperial Holtville 6 1123 55 link
Imperial Imperial 6 1275 367 link
Imperial Imperial County 1 0 112 link
Imperial Westmorland 1 50 3 link
Los Angeles Agoura Hills 3 897 89 link
Los Angeles Alhambra 5 1490 80 link
Los Angeles Arcadia 5 2573 3211 link
Los Angeles Artesia 2 205 334 link
Los Angeles Avalon 5 1340 1 link
Los Angeles Azusa 6 1385 724 link
Los Angeles Baldwin Park 1 0 131 link
Los Angeles Bell 5 2179 14 link
Los Angeles Bell Gardens 3 796 135 link
Los Angeles Bellflower 4 1557 0 link
Los Angeles Beverly Hills 5 1713 3289 link
Los Angeles Bradbury 3 455 68 link
Los Angeles Burbank 6 2656 3172 link
Los Angeles Calabasas 4 1015 32 link
Los Angeles Carson 4 1503 794 link
Los Angeles Cerritos 4 1245 55 link
Los Angeles Claremont 4 1424 288 link
Los Angeles Commerce 1 0 13 link
Los Angeles Compton 0 0 3 link
Los Angeles Covina 3 211 163 link
Los Angeles Cudahy 4 1417 79 link
Los Angeles Culver City 3 3223 1652 link
Los Angeles Diamond Bar 3 541 623 link
Los Angeles Downey 4 1161 2385 link
Los Angeles Duarte 4 970 177 link
Los Angeles El Monte 4 1979 4900 link
Los Angeles El Segundo 3 1145 62 link
Los Angeles Gardena 6 2468 4523 link
Los Angeles Glendale 5 2517 4692 link
Los Angeles Glendora 4 758 385 link
Los Angeles Hawaiian Gardens 3 1209 130 link
Los Angeles Hawthorne 1 0 60 link
Los Angeles Hermosa Beach 3 314 122 link
Los Angeles Hidden Hills 4 350 18 link
Los Angeles Huntington Park 5 1616 271 link
Los Angeles Industry 4 363 1 link
Los Angeles Inglewood 2 845 0 link
Los Angeles Irwindale 1 412 21 link
Los Angeles La Canada Flintridge 3 3629 1180 link
Los Angeles La Habra Heights 2 236 320 link
Los Angeles La Mirada 3 292 42 link
Los Angeles La Puente 4 1205 650 link
Los Angeles La Verne 4 1611 412 link
Los Angeles Lakewood 4 591 584 link
Los Angeles Lancaster 5 1290 3148 link
Los Angeles Lawndale 2 539 252 link
Los Angeles Lomita 4 1117 593 link
Los Angeles Long Beach 4 786 2123 link
Los Angeles Los Angeles 6 3831 304163 link
Los Angeles Los Angeles County 0 0 187 link
Los Angeles Lynwood 4 1138 351 link
Los Angeles Malibu 3 251 59 link
Los Angeles Manhattan Beach 5 1960 50 link
Los Angeles Maywood 4 271 269 link
Los Angeles Monrovia 5 1124 387 link
Los Angeles Montebello 3 554 22 link
Los Angeles Monterey Park 5 1052 1558 link
Los Angeles Norwalk 3 562 414 link
Los Angeles Palmdale 5 2171 570 link
Los Angeles Palos Verdes Estates 1 0 41 link
Los Angeles Paramount 4 556 74 link
Los Angeles Pasadena 5 2561 1600 link
Los Angeles Pico Rivera 3 1992 77 link
Los Angeles Pomona 3 2110 56 link
Los Angeles Rancho Palos Verdes 2 496 28 link
Los Angeles Redondo Beach 5 1234 4571 link
Los Angeles Rolling Hills 5 1311 144 link
Los Angeles Rolling Hills Estates 7 1402 106 link
Los Angeles Rosemead 4 1398 1226 link
Los Angeles San Dimas 3 553 66 link
Los Angeles San Fernando 3 627 346 link
Los Angeles San Gabriel 4 843 144 link
Los Angeles San Marino 2 497 23 link
Los Angeles Santa Clarita 4 1493 426 link
Los Angeles Santa Fe Springs 3 482 132 link
Los Angeles Santa Monica 5 4565 2277 link
Los Angeles Sierra Madre 6 1388 316 link
Los Angeles Signal Hill 5 881 80 link
Los Angeles South El Monte 4 1214 133 link
Los Angeles South Gate 3 504 457 link
Los Angeles South Pasadena 6 6903 1533 link
Los Angeles Temple City 5 1141 246 link
Los Angeles Torrance 4 2407 1451 link
Los Angeles Vernon 3 332 3 link
Los Angeles Walnut 5 1101 171 link
Los Angeles West Covina 3 346 1214 link
Los Angeles West Hollywood 4 1598 339 link
Los Angeles Westlake Village 4 443 67 link
Los Angeles Whittier 4 1337 507 link
Orange Aliso Viejo 3 301 9 link
Orange Anaheim 4 2429 925 link
Orange Brea 5 1841 352 link
Orange Buena Park 4 1317 2219 link
Orange Costa Mesa 4 5367 38 link
Orange Cypress 4 805 650 link
Orange Dana Point 6 1582 18 link
Orange Fountain Valley 4 583 9 link
Orange Fullerton 2 179 479 link
Orange Garden Grove 4 1313 2089 link
Orange Huntington Beach 0 0 42 link
Orange Irvine 5 4670 9563 link
Orange La Habra 6 1074 66 link
Orange La Palma 4 391 172 link
Orange Laguna Beach 5 1823 71 link
Orange Laguna Hills 5 2812 4 link
Orange Laguna Niguel 3 348 67 link
Orange Laguna Woods 4 1840 56 link
Orange Lake Forest 5 1694 821 link
Orange Los Alamitos 5 1194 379 link
Orange Mission Viejo 2 95 35 link
Orange Newport Beach 6 3481 113 link
Orange Orange 4 1675 56 link
Orange Orange County 0 0 0 link
Orange Placentia 3 384 709 link
Orange Rancho Santa Margarita 0 0 28 link
Orange San Clemente 5 1256 149 link
Orange San Juan Capistrano 4 1639 407 link
Orange Santa Ana 4 1282 96 link
Orange Seal Beach 3 612 334 link
Orange Stanton 4 1453 564 link
Orange Tustin 5 4131 330 link
Orange Villa Park 6 712 15 link
Orange Westminster 5 2453 2862 link
Orange Yorba Linda 4 1035 374 link
Riverside Banning 3 747 1051 link
Riverside Beaumont 3 740 89 link
Riverside Blythe 3 364 1643 link
Riverside Calimesa 4 1103 530 link
Riverside Canyon Lake 4 978 481 link
Riverside Cathedral City 2 305 41 link
Riverside Coachella 2 791 1459 link
Riverside Corona 4 1683 2638 link
Riverside Desert Hot Springs 5 1243 1537 link
Riverside Eastvale 3 615 208 link
Riverside Hemet 3 804 3674 link
Riverside Indian Wells 3 458 26 link
Riverside Indio 4 1606 1269 link
Riverside Jurupa Valley 4 1329 466 link
Riverside La Quinta 6 1102 1882 link
Riverside Lake Elsinore 4 1183 1080 link
Riverside Menifee 8 3906 4147 link
Riverside Moreno Valley 4 2347 13676 link
Riverside Murrieta 5 1913 4446 link
Riverside Norco 6 3029 503 link
Riverside Palm Desert 6 1069 221 link
Riverside Palm Springs 4 886 65 link
Riverside Perris 5 2298 683 link
Riverside Rancho Mirage 6 467 465 link
Riverside Riverside 5 1955 3040 link
Riverside Riverside County 1 0 295 link
Riverside San Jacinto 3 2697 799 link
Riverside Temecula 4 2050 525 link
Riverside Wildomar 3 508 445 link
San Bernardino Adelanto 2 171 564 link
San Bernardino Apple Valley 7 1681 38 link
San Bernardino Barstow 3 602 25 link
San Bernardino Big Bear Lake 5 498 38 link
San Bernardino Chino 4 2000 130 link
San Bernardino Chino Hills 6 1340 9 link
San Bernardino Colton 4 868 251 link
San Bernardino Fontana 5 2354 473 link
San Bernardino Grand Terrace 1 81 186 link
San Bernardino Hesperia 6 1260 1289 link
San Bernardino Highland 3 1165 238 link
San Bernardino Loma Linda 5 1240 408 link
San Bernardino Montclair 3 836 3 link
San Bernardino Needles 3 550 13 link
San Bernardino Ontario 4 982 61 link
San Bernardino Rancho Cucamonga 4 1142 6 link
San Bernardino Redlands 5 1022 137 link
San Bernardino Rialto 5 2828 68 link
San Bernardino San Bernardino 1 0 555 link
San Bernardino San Bernardino County 1 0 54 link
San Bernardino Twentynine Palms 4 786 23 link
San Bernardino Upland 3 477 20 link
San Bernardino Victorville 5 2631 378 link
San Bernardino Yucaipa 5 1028 10 link
San Bernardino Yucca Valley 4 962 265 link
Ventura Camarillo 4 1289 218 link
Ventura Fillmore 1 292 218 link
Ventura Moorpark 4 985 218 link
Ventura Ojai 3 548 218 link
Ventura Oxnard 5 2657 2754 link
Ventura Port Hueneme 3 648 274 link
Ventura Santa Paula 3 519 218 link
Ventura Simi Valley 5 1429 767 link
Ventura Thousand Oaks 3 1193 59 link
Ventura Ventura 5 1738 1300 link
Ventura Ventura County 1 0 276 link

END Summary

About

Shapefiles generated from the housing element documents of ABAG, SACOG, and SCAG cities. Project for CA Yimby.

Resources

Readme
Activity

Stars

1 star

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages