The addon I'm using with the library tool talks to a backend server that rate limits its queries (to approximately one listing every 3-4 seconds). This means that when I add a directory with several shows every show has many seasons) I end up missing lots of seasons or entire shows.
When I manually add xbmc.sleep(5000) after each directory listing in resources/lib/utils.py everything works, obviously. Another option would be back off for 5 seconds and retry listing when it results in error...