Try to fix CI problems

.travis.yml

@@ -32,11 +32,17 @@ before_install:
   - conda install --yes -n condaenv pip
   - source activate condaenv
   - conda install --yes -c conda-forge obspy psycopg2 markdown flake8 gdal pyyaml pip geopy
-  - pip install codecov "django>=1.9,<1.10" djangorestframework djangorestframework-gis djangorestframework-jsonp djangorestframework-xml djangorestframework-yaml django-cors-headers django-debug-toolbar django-plugins defusedxml geojson markdown mkdocs mkdocs-bootswatch
+  - pip install codecov "django>=1.9,<1.10" 'djangorestframework<3.7' djangorestframework-gis djangorestframework-jsonp djangorestframework-xml djangorestframework-yaml django-cors-headers django-debug-toolbar django-plugins defusedxml geojson markdown mkdocs mkdocs-bootswatch
   # Copy local_settings template.
   - cp $TRAVIS_BUILD_DIR/src/jane/local_settings.py.example $TRAVIS_BUILD_DIR/src/jane/local_settings.py
   # Make sure django picks the correct geos library.
   - echo -e "\n\nGDAL_LIBRARY_PATH = '/home/travis/miniconda/envs/condaenv/lib/libgdal.so'" >> $TRAVIS_BUILD_DIR/src/jane/local_settings.py
+  # fix django resolving the geos version for conda-installed geos packages
+  # django 1.9 is parsing e.g. '3.4.0dev-CAPI-1.8.0 r0'
+  # but we're encounering e.g. '3.6.2-CAPI-1.10.2 0'
+  # not sure if that's due to version number being changed in conda packaging..
+  # if not maybe restricting the geos version to something lower might help
+  - sed --in-place '/?P<capi_version>/s#( r\\d+)#( r?\\d+)#' `python -c 'import os; from django import contrib; print(os.path.dirname(contrib.__file__) + "/gis/geos/libgeos.py")'`
 
 install:
   - git version

docs/docs/setup.md

@@ -12,14 +12,14 @@ This page details how to setup and install `Jane`.
 
 and furthermore on
 
-* `Python 3.4 or 3.5`
+* `Python 3.4 or 3.5 or 3.6`
 
 with the following Python modules
 
 * `obspy`
 * `django==1.9.x`
 * `psycopg2`
-* `djangorestframework`
+* `djangorestframework<3.7`
 * `djangorestframework-gis`
 * `djangorestframework-jsonp`
 * `djangorestframework-xml`
@@ -45,12 +45,10 @@ used to setup a new dedicated and separate environment to run `Jane`:
 
 ```bash
 $ conda config --add channels conda-forge
-$ conda create -n jane python=3.5
-$ source activate jane
-(jane)$ conda install obspy psycopg2 markdown flake8 gdal pyyaml pip geopy
-# Install the latest 1.9.x release.
-(jane)$ pip install "django>=1.9,<1.10"
-(jane)$ pip install djangorestframework djangorestframework-gis djangorestframework-jsonp djangorestframework-xml djangorestframework-yaml django-cors-headers django-debug-toolbar django-plugins defusedxml geojson markdown mkdocs mkdocs-bootswatch
+$ conda create -n jane python=3.6
+$ conda activate jane
+(jane)$ conda install django=1.9 obspy psycopg2 markdown flake8 gdal pyyaml pip geopy 'djangorestframework<3.7' djangorestframework-gis djangorestframework-xml djangorestframework-yaml django-debug-toolbar defusedxml geojson markdown mkdocs mkdocs-bootswatch
+(jane)$ pip install djangorestframework-jsonp django-cors-headers django-plugins
 ```
 
 Alternatively, the following Anaconda environment description file ...
@@ -69,7 +67,7 @@ dependencies:
 - pip
 - pip:
   - django>=1.9,<1.10
-  - djangorestframework
+  - djangorestframework<3.7
   - djangorestframework-gis
   - djangorestframework-jsonp
   - djangorestframework-xml
@@ -99,8 +97,8 @@ content in a new file `/path/to/anaconda/envs/jane/conda-meta/pinned` (see
 [http://conda.pydata.org/docs/faq.html#pinning-packages](http://conda.pydata.org/docs/faq.html#pinning-packages)):
 
 ```bash
-python 3.5.*
-obspy 1.0.*
+python 3.6.*
+obspy 1.1.*
 ```
 
 After everything has been installed, make sure to run the tests as explained
@@ -226,8 +224,8 @@ psql --command="CREATE EXTENSION postgis;" jane
 ### Python and Dependencies
 
 We (as much as possible) rely on Debian packages. Sometimes we have to use 
-`pip` if the Python module is not packaged. Make sure to install Python 3.4 or 
-3.5!
+`pip` if the Python module is not packaged. Make sure to install Python 3.4,
+3.5 or 3.6!
 
 First, add the ObsPy sources as written 
 [here](https://github.com/obspy/obspy/wiki/Installation-on-Linux-via-Apt-Repository).
@@ -257,7 +255,7 @@ to use `pip3` instead of `pip`):
 # AS USER!
 pip3 install --user \
     "django>=1.9,<1.10" \
-    djangorestframework \
+    "djangorestframework<3.7" \
     djangorestframework-gis \
     djangorestframework-jsonp \
     djangorestframework-xml \
@@ -332,3 +330,29 @@ will of course differ per installation):
 ```
 
 Make sure to choose sensible `processes` and `threads` options.
+
+### Troubleshooting
+
+When running `python3 manage.py runserver` the following error traceback shows up:
+
+```
+  [...]
+  File ".../lib/python3.6/site-packages/django/contrib/gis/geos/libgeos.py", line 171, in geos_version_info
+    raise GEOSException('Could not parse version info string "%s"' % ver)
+django.contrib.gis.geos.error.GEOSException: Could not parse version info string "3.6.2-CAPI-1.10.2 0"
+```
+
+See [#78](https://github.com/krischer/jane/pull/78) and monkey patch the above file `libgeos.py` by adding a `?` after the `r` in the regex pattern like shown [here](https://github.com/krischer/jane/blob/45862359dc385a3518b4d080ac7336ad78ec85c5/.travis.yml#L45).
+
+---
+
+When running `python3 manage.py runserver` the following error traceback shows up:
+
+```
+  [...]
+  File ".../lib/python3.6/site-packages/rest_framework/compat.py", line 21, in <module>
+    from django.urls import (  # noqa
+ModuleNotFoundError: No module named 'django.urls'
+```
+
+Check version of `django-restframework`. `django-restframework`>= 3.7 does not work together with Django 1.9, so downgrade `django-restframework` if it is a higher version. See [#78](https://github.com/krischer/jane/pull/78).

src/jane/documents/plugins.py

@@ -114,7 +114,7 @@ def initialize_plugins():
         # case.
         try:
             [_i for _i in obj.get_plugins()]
-        except:
+        except Exception:
             pass
 
         for plugin in obj.plugins:

src/jane/documents/signals.py

@@ -67,13 +67,13 @@ def index_document(sender, instance, created, **kwargs):  # @UnusedVariable
             attachments = index.get('attachments')
             try:
                 del index['attachments']
-            except:
+            except Exception:
                 pass
             # geometry
             geometry = index.get('geometry')
             try:
                 del index['geometry']
-            except:
+            except Exception:
                 pass
             # add index
             obj = models.DocumentIndex(document=instance, json=index)

src/jane/jane/middleware.py

@@ -15,7 +15,7 @@ def process_request(self, request):
         try:
             if not settings.AUTO_LOGOUT_MINUTES:
                 return
-        except:
+        except AttributeError:
             return
 
         try:

src/jane/quakeml/plugins.py

@@ -44,7 +44,7 @@ def validate(self, document):
         from obspy.io.quakeml.core import _validate as validate_quakeml  # NOQA
         try:
             is_valid = validate_quakeml(document)
-        except:
+        except Exception:
             is_valid = False
         return is_valid
 

src/jane/stationxml/plugins.py

@@ -162,7 +162,7 @@ def index(self, document):
 
                     try:
                         plt.close()
-                    except:
+                    except Exception:
                         pass
 
                     # Sometimes fails. Wrap in try/except.
@@ -179,7 +179,7 @@ def index(self, document):
                     finally:
                         try:
                             plt.close()
-                        except:
+                        except Exception:
                             pass
 
                     indices.append(index)

src/jane/waveforms/management/commands/add_mappings.py

@@ -37,23 +37,23 @@ def handle(self, *args, **kwargs):
 
                 try:
                     from_net, from_sta, from_loc, from_cha = from_id.split(".")
-                except:
+                except Exception:
                     msg = "From id '%s' is not acceptable SEED id." % from_id
                     raise Exception(msg)
                 try:
                     to_net, to_sta, to_loc, to_cha = to_id.split(".")
-                except:
+                except Exception:
                     msg = "From id '%s' is not acceptable SEED id." % to_id
                     raise Exception(msg)
 
                 try:
                     start = obspy.UTCDateTime(start)
-                except:
+                except Exception:
                     msg = "Start time '%s' cannot be parsed." % start
                     raise Exception(msg)
                 try:
                     end = obspy.UTCDateTime(end)
-                except:
+                except Exception:
                     msg = "End time '%s' cannot be parsed." % end
                     raise Exception(msg)
 

src/jane/waveforms/management/commands/index_waveforms.py

@@ -72,7 +72,7 @@ def _select(self, path=None):
     def _process_log_queue(self):
         try:
             msg = self.log_queue.pop(0)
-        except:
+        except Exception:
             pass
         else:
             if msg.startswith('['):
@@ -255,7 +255,7 @@ def iterate(self):
             if time.time() - mtime > 60 * 60 * self.options["recent"]:
                 try:
                     db_file_mtime = self._db_files.pop(file)
-                except:
+                except Exception:
                     pass
                 return
 
@@ -273,7 +273,7 @@ def iterate(self):
         # -> remove from file list so it won't be deleted on database cleanup
         try:
             db_file_mtime = self._db_files.pop(file)
-        except:
+        except Exception:
             return
         # -> compare modification times of current file with database entry
         if datetime.datetime.fromtimestamp(mtime) == db_file_mtime:
@@ -289,7 +289,7 @@ def worker(_i, input_queue, work_queue, log_queue):
             # fetch a unprocessed item
             try:
                 filepath = input_queue.popitem()[0]
-            except:
+            except Exception:
                 continue
             # skip item if already in work queue
             if filepath in work_queue:
@@ -305,7 +305,7 @@ def worker(_i, input_queue, work_queue, log_queue):
             # ALways attempt to remove it from the worker queue.
             try:
                 work_queue.remove(filepath)
-            except:
+            except Exception:
                 pass
     except KeyboardInterrupt:
         return

src/jane/waveforms/process_waveforms.py

@@ -51,7 +51,7 @@ def process_file(filename):
     #         eventually existing file.
     try:
         stream = read(filename)
-    except:
+    except Exception:
         # Delete if invalid file.
         if file is not None:
             file.delete()
@@ -142,7 +142,7 @@ def process_file(filename):
                 # does not work.
                 try:
                     preview_trace = create_preview(trace, 60)
-                except:
+                except Exception:
                     preview_trace = None
                 else:
                     preview_trace = list(map(float, preview_trace.data))