diff --git a/doc/en/user/source/community/index.rst b/doc/en/user/source/community/index.rst index 10430c8850c..9f9b639c386 100644 --- a/doc/en/user/source/community/index.rst +++ b/doc/en/user/source/community/index.rst @@ -45,5 +45,4 @@ officially part of the GeoServer releases. They are however built along with the onelogin/index wmts-multidimensional/index notification/index - opensearch-eo/index - s3-geotiff/index + nsg-profile/index diff --git a/doc/en/user/source/community/nsg-profile/index.rst b/doc/en/user/source/community/nsg-profile/index.rst new file mode 100644 index 00000000000..9c8af4798d2 --- /dev/null +++ b/doc/en/user/source/community/nsg-profile/index.rst @@ -0,0 +1,212 @@ +.. _community_nsg_profile: + +NSG Profile +=========== +NSG Profile introduces a new operation for WFS 2.0.2 named PageResults. This operation will allow clients to access paginated results using random positions. + +The current WFS 2.0.2 OGC specification defines a basic pagination support that can been used to navigate through features responses results. + +Pagination is activated when parameters count and startIndex are used in the query, for example: + + :: + + http:///geoserver/ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=topp%3Atasmania_roads&count=5&startIndex=0 + + + +In this case each page will contain five features. +The returned feature collection will have the next and previous attributes which will contain an URL that will allow clients to navigate through the results pages, i.e. previous page and next page: + + :: + + + + +This means that this type of navigation will always be sequential, if the client is showing page two and the user wants to see page five the client will have to: + +#. request page three and use the provided next URL to retrieve page four +#. request page four and use the provided next URI to retrieve page five + +This is not an ideal solution to access random pages, which is common action. +PageResults operation will improve this by allowing clients to request random pages directly. + +Installing the extension +------------------------ + +#. Download the NSG Profile extension from the nightly GeoServer community module builds. + +#. Place the JARs into the ``WEB-INF/lib`` directory of the GeoServer installation. + +Configure the extension +----------------------- + +The root directory inside the GeoServer data directory for the nsg-profile community module is named nsg-profile and all the configurations properties are stored in a file named **configuration.properties**. + +All configuration properties are changeable at runtime, which means that if a property is updated the module take it into account. + +When the application starts if no configuration file exists one with the default values is created. + +The GetFeature requests representations associated with an index result type is serialized and stored in the file system in a location that is configurable. + +The default location, relative to the GeoServer data directory, is nsg-profie/resultSets. + +The GetFeature request to resultSetID mapping is stored by default in an H2 DB in nsg-profie/resultSets folder; for details on database configuration see `GeoTools JDBCDataStore syntax `_ + +The configuration properties are the follows: + + +.. list-table:: + :widths: 20 30 50 + :header-rows: 1 + + * - Name + - Default Value + - Description + * - resultSets.storage.path + - ${GEOSERVER_DATA_DIR}/nsg-profile/resultSets + - Path where to store GetFeature requests representations + * - resultSets.timeToLive + - 600 + - How long a GetFeature request should be maintained by the server (in seconds) + * - resultSets.db.dbtype + - h2 + - DB type used to store GetFeature request to resultSetID mapping + * - resultSets.db.database + - ${GEOSERVER_DATA_DIR}/nsg-profile/db/resultSets + - path where to store GetFeature request to resultSetID mapping + * - resultSets.db.user + - sa + - database user username + * - resultSets.db.password + - sa + - database user password + * - resultSets.db.port + - + - database port to connect to + * - resultSets.db.schema + - + - database schema + * - resultSets.db.host + - + - server to connect to + + +Index Result Type +----------------- +The **index result type** extends the WFS **hits result type** by adding an extra attribute named **resultSetID** to the response. +The **resultSetID** attribute can then be used by the **PageResults operation** to navigate randomly through the results. + +A GetFeature request that uses the index result type should look like this: + + :: + + http:///geoserver/ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=topp%3Atasmania_roads&resultType=index + + +The response of a GetFeature operation when the index result type is used should look like this: + + :: + + + + +The **resultSetID** is an unique identifier that identifies the original request. + +Clients will use the **resultSetID** with the PageResults operation to reference the original request. + +If pagination is used, the previous and next attributes should appear as in hits result type request. + +PageResults Operation +--------------------- + +The **PageResults operation** allows clients to query random positions of an existing result set (stored GetFeature request) that was previously created using the **index result type** request. + +The available parameters are this ones: + +.. list-table:: + :widths: 40 20 40 + :header-rows: 1 + + * - Name + - Mandotry + - Default Value + * - service + - YES + - WFS + * - version + - YES + - 2.0.2 + * - request + - YES + - PageResults + * - resultSetID + - YES + - + * - startIndex + - NO + - 0 + * - count + - NO + - 10 + * - outputFormat + - NO + - application/gml+xml; version=3.2 + * - resultType + - NO + - results + * - timeout + - NO + - 300 + + +The two parameters that are not already supported by the GetFeature operation are the **resultSetID** parameter and the **timeout** parameter. + +#. The **resultSetID** parameter should reference an existing result set (stored GetFeature request). + + A typical PageResults request will look like this: + + :: + + http:///geoserver/ows?service=WFS&version=2.0.2&request=PageResults&resultSetID=ef35292477a011e7b5a5be2e44b06b34&startIndex=5&count=10&outputFormat=application/gml+xml; version=3.2&resultType=results + + + This looks like a GetFeature request where the **query expression was substituted by the resultSetID parameter**. + +#. The **timeout** parameter is not implemented yet. + +The following parameters of index request are override using the ones provided with the PageResults operation or the default values: + +#. startIndex +#. count +#. outputFormat +#. resultType + +and finally the GetFeature response is returned. diff --git a/src/community/nsg-profile/pom.xml b/src/community/nsg-profile/pom.xml new file mode 100644 index 00000000000..4fcfea4c77c --- /dev/null +++ b/src/community/nsg-profile/pom.xml @@ -0,0 +1,92 @@ + + + 4.0.0 + + org.geoserver + community + 2.12-SNAPSHOT + + org.geoserver.community + gs-nsg-profile + jar + NSG Profile + + + + org.geoserver + gs-wfs + + + org.geoserver.web + gs-web-core + + + + org.geotools.jdbc + gt-jdbc-h2 + + + com.google.code.gson + gson + + + + org.geoserver + gs-main + tests + test + + + org.geoserver + gs-wfs + tests + ${project.version} + + + org.geoserver.web + gs-web-core + ${project.version} + tests + test + + + + org.hamcrest + hamcrest-library + test + + + org.springframework + spring-test + test + + + junit + junit + test + + + javax.servlet + javax.servlet-api + test + + + + + + ${basedir}/src/main/java + + **/*.html + + + + ${basedir}/src/main/resources + + **/* + + + + + + diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexConfiguration.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexConfiguration.java new file mode 100644 index 00000000000..d7c33bb703a --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexConfiguration.java @@ -0,0 +1,78 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.util.Map; +import java.util.concurrent.TimeUnit; + +import org.geoserver.platform.resource.Resource; +import org.geotools.data.DataStore; + +/** + * + * Class used to store the index result type configuration managed by {@link IndexInitializer} + * + * @author sandr + * + */ +public class IndexConfiguration { + + private DataStore currentDataStore; + + private Resource storageResource; + + private Long timeToLiveInSec = 600l; + + private Map currentDataStoreParams; + + /** + * Store the DB parameters and the relative {@link DataStore} + * + * @param currentDataStoreParams + * @param currentDataStore + */ + public void setCurrentDataStore(Map currentDataStoreParams, + DataStore currentDataStore) { + this.currentDataStoreParams = currentDataStoreParams; + this.currentDataStore = currentDataStore; + } + + /** + * Store the reference to resource used to archive the serialized GetFeatureRequest + * + * @param storageResource + */ + public void setStorageResource(Resource storageResource) { + this.storageResource = storageResource; + } + + /** + * Store the value of time to live of stored GetFeatureRequest + * + * @param timeToLive + * @param timeUnit + */ + public void setTimeToLive(Long timeToLive, TimeUnit timeUnit) { + this.timeToLiveInSec = timeUnit.toSeconds(timeToLive); + } + + public DataStore getCurrentDataStore() { + return currentDataStore; + } + + public Map getCurrentDataStoreParams() { + return currentDataStoreParams; + } + + public Resource getStorageResource() { + return storageResource; + } + + public Long getTimeToLiveInSec() { + return timeToLiveInSec; + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexInitializer.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexInitializer.java new file mode 100644 index 00000000000..035d1520f90 --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexInitializer.java @@ -0,0 +1,389 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.io.File; +import java.io.InputStream; +import java.io.OutputStream; +import java.text.SimpleDateFormat; +import java.util.Date; +import java.util.HashMap; +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.locks.ReadWriteLock; +import java.util.concurrent.locks.ReentrantReadWriteLock; +import java.util.logging.Level; +import java.util.logging.Logger; + +import org.geoserver.config.GeoServer; +import org.geoserver.config.GeoServerDataDirectory; +import org.geoserver.config.GeoServerInitializer; +import org.geoserver.platform.GeoServerExtensions; +import org.geoserver.platform.GeoServerResourceLoader; +import org.geoserver.platform.resource.FileSystemResourceStore; +import org.geoserver.platform.resource.Resource; +import org.geoserver.platform.resource.ResourceNotification.Kind; +import org.geotools.data.DataStore; +import org.geotools.data.DataStoreFinder; +import org.geotools.data.DataUtilities; +import org.geotools.data.DefaultTransaction; +import org.geotools.data.Transaction; +import org.geotools.data.simple.SimpleFeatureCollection; +import org.geotools.data.simple.SimpleFeatureIterator; +import org.geotools.data.simple.SimpleFeatureSource; +import org.geotools.data.simple.SimpleFeatureStore; +import org.geotools.feature.NameImpl; +import org.geotools.filter.text.cql2.CQL; +import org.geotools.jdbc.JDBCDataStoreFactory; +import org.geotools.util.logging.Logging; +import org.opengis.feature.simple.SimpleFeature; +import org.opengis.feature.simple.SimpleFeatureType; +import org.opengis.filter.Filter; + +/** + * + * Class used to parse the configuration properties stored in nsg-profile module folder: + *
    + *
  • resultSets.storage.path path where to store the serialized GetFeatureRequest with name + * of random UUID. + *
  • resultSets.timeToLive time to live value, all the stored requests that have not been + * used for a period of time bigger than this will be deleted. + *
  • resultSets.db.{@link JDBCDataStoreFactory#DBTYPE} + *
  • resultSets.db.{@link JDBCDataStoreFactory#DATABASE} + *
  • resultSets.db.{@link JDBCDataStoreFactory#HOST} + *
  • resultSets.db.{@link JDBCDataStoreFactory#PORT} + *
  • resultSets.db.{@link JDBCDataStoreFactory#SCHEMA} + *
  • resultSets.db.{@link JDBCDataStoreFactory#USER} + *
  • resultSets.db.{@link JDBCDataStoreFactory#PASSWD} + *
+ * All configuration properties is changeable at runtime so when this properties is updated the + * module take the appropriate action: + *
    + *
  • When the index DB is changed the new DB should be used and the content of the old table moved + * to the new table. If the new DB already has the index table it should be emptied, + *
  • When the storage path is changed, the new storage path should be used and the old storage + * path content should be moved to the new one, + *
  • When the the time to live is changed the {@link #clean()} procedure will update. + *
+ * + * The class is also responsible to {@link #clean()} the stored requests (result sets) that have not + * been used for a period of time bigger than the configured time to live value + *

+ * + * @author sandr + * + */ + +public final class IndexInitializer implements GeoServerInitializer { + + static Logger LOGGER = Logging.getLogger(IndexInitializer.class); + + static final String PROPERTY_DB_PREFIX = "resultSets.db."; + + static final String PROPERTY_FILENAME = "configuration.properties"; + + static final String MODULE_DIR = "nsg-profile"; + + public static final String STORE_SCHEMA_NAME = "RESULT_SET"; + + public static final String STORE_SCHEMA = "ID:java.lang.String,created:java.lang.Long,updated:java.lang.Long"; + + private IndexConfiguration indexConfiguration; + + /* + * Lock to synchronize activity of clean task with listener that changes the DB and file + * resources + */ + protected static final ReadWriteLock READ_WRITE_LOCK = new ReentrantReadWriteLock(); + + @Override + public void initialize(GeoServer geoServer) throws Exception { + indexConfiguration = GeoServerExtensions.bean(IndexConfiguration.class); + GeoServerResourceLoader loader = GeoServerExtensions.bean(GeoServerResourceLoader.class); + GeoServerDataDirectory dd = new GeoServerDataDirectory(loader); + Resource resource = dd.get(MODULE_DIR + "/" + PROPERTY_FILENAME); + if (loader != null) { + File directory = loader.findOrCreateDirectory(MODULE_DIR); + File file = new File(directory, PROPERTY_FILENAME); + // Create default configuration file + if (!file.exists()) { + InputStream stream = IndexInitializer.class + .getResourceAsStream("/" + PROPERTY_FILENAME); + Properties properties = new Properties(); + properties.load(stream); + // Replace GEOSERVER_DATA_DIR placeholder + properties.replaceAll((k, v) -> ((String) v).replace("${GEOSERVER_DATA_DIR}", + dd.root().getPath())); + stream.close(); + // Create resource and save properties + try { + OutputStream out = resource.out(); + properties.store(out, null); + out.close(); + } catch (Exception exception) { + throw new RuntimeException("Error to initialize configurations.", exception); + } + } + loadConfigurations(resource); + // Listen for changes in configuration file and reload properties + resource.addListener(notify -> { + if (notify.getKind() == Kind.ENTRY_MODIFY) { + try { + loadConfigurations(resource); + } catch (Exception exception) { + throw new RuntimeException("Error reload configurations.", exception); + } + } + }); + } + } + + /** + * Helper method that loads configuration file and changes environment setup + */ + private void loadConfigurations(Resource resource) throws Exception { + try { + IndexInitializer.READ_WRITE_LOCK.writeLock().lock(); + Properties properties = new Properties(); + InputStream is = resource.in(); + properties.load(is); + is.close(); + // Reload database + Map params = new HashMap<>(); + params.put(JDBCDataStoreFactory.DBTYPE.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.DBTYPE.key)); + params.put(JDBCDataStoreFactory.DATABASE.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.DATABASE.key)); + params.put(JDBCDataStoreFactory.HOST.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.HOST.key)); + params.put(JDBCDataStoreFactory.PORT.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.PORT.key)); + params.put(JDBCDataStoreFactory.SCHEMA.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.SCHEMA.key)); + params.put(JDBCDataStoreFactory.USER.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.USER.key)); + params.put(JDBCDataStoreFactory.PASSWD.key, + properties.get(PROPERTY_DB_PREFIX + JDBCDataStoreFactory.PASSWD.key)); + /** + * When the index DB is changed the new DB should be used and the content of the old + * table moved to the new table. If the new DB already has the index table it should be + * emptied + */ + manageDBChange(params); + /* + * If the storage path is changed, the new storage path should be used and the old + * storage path content should be moved to the new one + */ + manageStorageChange(resource, properties.get("resultSets.storage.path")); + /* + * Change time to live + */ + manageTimeToLiveChange(properties.get("resultSets.timeToLive")); + } catch (Exception exception) { + throw new RuntimeException("Error reload configurations.", exception); + } finally { + IndexInitializer.READ_WRITE_LOCK.writeLock().unlock(); + } + } + + /** + * Helper method that store the time to live value + */ + private void manageTimeToLiveChange(Object timneToLive) { + try { + if (timneToLive != null) { + String timneToLiveStr = (String) timneToLive; + indexConfiguration.setTimeToLive(Long.parseLong(timneToLiveStr), TimeUnit.SECONDS); + } + } catch (Exception exception) { + throw new RuntimeException("Error on change time to live", exception); + } + } + + /** + * Helper method that move resources files form current folder to the new one, current storage + * is deleted + */ + private void manageStorageChange(Resource resource, Object newStorage) { + try { + if (newStorage != null) { + String newStorageStr = (String) newStorage; + Resource newResource = new FileSystemResourceStore(new File(newStorageStr)).get(""); + Resource exResource = indexConfiguration.getStorageResource(); + if (exResource != null && !newResource.dir().getAbsolutePath() + .equals(exResource.dir().getAbsolutePath())) { + exResource.delete(); + } + indexConfiguration.setStorageResource(newResource); + } + } catch (Exception exception) { + throw new RuntimeException("Error on change store", exception); + } + } + + /** + * Helper method that move DB data from old store to new one + */ + private void manageDBChange(Map params) { + try { + DataStore exDataStore = indexConfiguration.getCurrentDataStore(); + DataStore newDataStore = DataStoreFinder.getDataStore(params); + if (exDataStore != null) { + // New database is valid and is different from current one + if (newDataStore != null && !isDBTheSame(params)) { + // Create table in new database + createTable(newDataStore, true); + // Move data to new database + moveData(exDataStore, newDataStore); + // Dispose old database + exDataStore.dispose(); + } + } else { + // Create schema + createTable(newDataStore, false); + } + indexConfiguration.setCurrentDataStore(params, newDataStore); + } catch (Exception exception) { + throw new RuntimeException("Error reload DB configurations.", exception); + } + } + + /** + * Helper method that check id the DB is the same, matching the JDBC configurations parameters. + */ + private Boolean isDBTheSame(Map newParams) { + Map currentParams = indexConfiguration.getCurrentDataStoreParams(); + boolean isTheSame = (currentParams.get(JDBCDataStoreFactory.DBTYPE.key) == null + && newParams.get(JDBCDataStoreFactory.DBTYPE.key) == null) + || (currentParams.get(JDBCDataStoreFactory.DBTYPE.key) != null + && newParams.get(JDBCDataStoreFactory.DBTYPE.key) != null + && currentParams.get(JDBCDataStoreFactory.DBTYPE.key) + .equals(newParams.get(JDBCDataStoreFactory.DBTYPE.key))); + isTheSame = isTheSame + && (currentParams.get(JDBCDataStoreFactory.DATABASE.key) == null + && newParams.get(JDBCDataStoreFactory.DATABASE.key) == null) + || (currentParams.get(JDBCDataStoreFactory.DATABASE.key) != null + && newParams.get(JDBCDataStoreFactory.DATABASE.key) != null + && currentParams.get(JDBCDataStoreFactory.DATABASE.key) + .equals(newParams.get(JDBCDataStoreFactory.DATABASE.key))); + isTheSame = isTheSame + && (currentParams.get(JDBCDataStoreFactory.HOST.key) == null + && newParams.get(JDBCDataStoreFactory.HOST.key) == null) + || (currentParams.get(JDBCDataStoreFactory.HOST.key) != null + && newParams.get(JDBCDataStoreFactory.HOST.key) != null + && currentParams.get(JDBCDataStoreFactory.HOST.key) + .equals(newParams.get(JDBCDataStoreFactory.HOST.key))); + isTheSame = isTheSame + && (currentParams.get(JDBCDataStoreFactory.PORT.key) == null + && newParams.get(JDBCDataStoreFactory.PORT.key) == null) + || (currentParams.get(JDBCDataStoreFactory.PORT.key) != null + && newParams.get(JDBCDataStoreFactory.PORT.key) != null + && currentParams.get(JDBCDataStoreFactory.PORT.key) + .equals(newParams.get(JDBCDataStoreFactory.PORT.key))); + isTheSame = isTheSame + && (currentParams.get(JDBCDataStoreFactory.SCHEMA.key) == null + && newParams.get(JDBCDataStoreFactory.SCHEMA.key) == null) + || (currentParams.get(JDBCDataStoreFactory.SCHEMA.key) != null + && newParams.get(JDBCDataStoreFactory.SCHEMA.key) != null + && currentParams.get(JDBCDataStoreFactory.SCHEMA.key) + .equals(newParams.get(JDBCDataStoreFactory.SCHEMA.key))); + return isTheSame; + } + + /** + * Helper method that create a new table on DB to store resource informations + */ + private void createTable(DataStore dataStore, boolean forceDelete) throws Exception { + boolean exists = dataStore.getNames().contains(new NameImpl(STORE_SCHEMA_NAME)); + // Schema exists + if (exists) { + // Delete of exist is required, and then create a new one + if (forceDelete) { + dataStore.removeSchema(STORE_SCHEMA_NAME); + SimpleFeatureType schema = DataUtilities.createType(STORE_SCHEMA_NAME, + STORE_SCHEMA); + dataStore.createSchema(schema); + } + // Schema not exists, create a new one + } else { + SimpleFeatureType schema = DataUtilities.createType(STORE_SCHEMA_NAME, STORE_SCHEMA); + dataStore.createSchema(schema); + } + } + + /** + * Helper method that move resource informations from current DB to the new one + */ + private void moveData(DataStore exDataStore, DataStore newDataStore) throws Exception { + Transaction session = new DefaultTransaction("Moving"); + try { + SimpleFeatureSource exFs = exDataStore.getFeatureSource(STORE_SCHEMA_NAME); + SimpleFeatureStore newFs = (SimpleFeatureStore) newDataStore + .getFeatureSource(STORE_SCHEMA_NAME); + newFs.setTransaction(session); + newFs.addFeatures(exFs.getFeatures()); + session.commit(); + } catch (Throwable t) { + session.rollback(); + throw new RuntimeException("Error on move data", t); + } finally { + session.close(); + } + } + + /** + * Delete all the stored requests (result sets) that have not been used for a period of time + * bigger than the configured time to live value. Clean also related resource files. + *

+ * Executed by scheduler, for details see Spring XML configuration + */ + public void clean() throws Exception { + Transaction session = new DefaultTransaction("RemoveOld"); + try { + IndexInitializer.READ_WRITE_LOCK.writeLock().lock(); + // Remove record + Long timeToLive = indexConfiguration.getTimeToLiveInSec(); + DataStore currentDataStore = indexConfiguration.getCurrentDataStore(); + Long liveTreshold = System.currentTimeMillis() - timeToLive * 1000; + long featureRemoved = 0; + if (currentDataStore != null) { + SimpleFeatureStore store = (SimpleFeatureStore) currentDataStore + .getFeatureSource(STORE_SCHEMA_NAME); + Filter filter = CQL.toFilter("updated < " + liveTreshold); + + SimpleFeatureCollection toRemoved = store.getFeatures(filter); + // Remove file + Resource currentResource = indexConfiguration.getStorageResource(); + SimpleFeatureIterator iterator = toRemoved.features(); + try { + while (iterator.hasNext()) { + SimpleFeature feature = iterator.next(); + currentResource.get(feature.getID()).delete(); + featureRemoved++; + } + } finally { + iterator.close(); + } + store.removeFeatures(filter); + } + if (LOGGER.isLoggable(Level.FINEST)) { + if (featureRemoved > 0) { + LOGGER.finest("CLEAN executed, removed " + featureRemoved + + " stored requests older than " + + new SimpleDateFormat("yyyy-MM-dd HH:mm:ss") + .format(new Date(liveTreshold))); + } + } + } catch (Throwable t) { + session.rollback(); + LOGGER.warning("Error on clean data"); + } finally { + session.close(); + IndexInitializer.READ_WRITE_LOCK.writeLock().unlock(); + } + } +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexOutputFormat.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexOutputFormat.java new file mode 100644 index 00000000000..2ce258f4408 --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexOutputFormat.java @@ -0,0 +1,216 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.io.BufferedOutputStream; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.ObjectOutputStream; +import java.io.OutputStream; +import java.math.BigInteger; +import java.nio.charset.Charset; +import java.util.Arrays; +import java.util.Map; +import java.util.UUID; +import java.util.logging.Logger; + +import javax.xml.transform.Transformer; +import javax.xml.transform.TransformerFactory; +import javax.xml.transform.dom.DOMSource; +import javax.xml.transform.stream.StreamResult; + +import org.geoserver.config.GeoServer; +import org.geoserver.ows.Request; +import org.geoserver.ows.util.OwsUtils; +import org.geoserver.ows.util.ResponseUtils; +import org.geoserver.platform.Operation; +import org.geoserver.platform.ServiceException; +import org.geoserver.platform.resource.Resource; +import org.geoserver.wfs.WFSInfo; +import org.geoserver.wfs.request.FeatureCollectionResponse; +import org.geoserver.wfs.response.v2_0.HitsOutputFormat; +import org.geotools.data.DataStore; +import org.geotools.data.collection.ListFeatureCollection; +import org.geotools.data.simple.SimpleFeatureCollection; +import org.geotools.data.simple.SimpleFeatureStore; +import org.geotools.feature.simple.SimpleFeatureBuilder; +import org.geotools.util.logging.Logging; +import org.geotools.wfs.v2_0.WFS; +import org.geotools.wfs.v2_0.WFSConfiguration; +import org.geotools.xml.Encoder; +import org.opengis.feature.simple.SimpleFeature; +import org.w3c.dom.Document; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + +import net.opengis.wfs20.GetFeatureType; + +/** + * This output format handles requests if the original requested result type was "index"
+ * See {@link IndexResultTypeDispatcherCallback} + * + * @author sandr + * + */ +public class IndexOutputFormat extends HitsOutputFormat { + + private final static Logger LOGGER = Logging.getLogger(IndexOutputFormat.class); + + private String resultSetId; + + private Request request; + + private IndexConfiguration indexConfiguration; + + public IndexOutputFormat(GeoServer gs, IndexConfiguration indexConfiguration) { + super(gs); + this.indexConfiguration = indexConfiguration; + } + + public void setRequest(Request request) { + this.request = request; + } + + @Override + public void write(Object value, OutputStream output, Operation operation) + throws IOException, ServiceException { + // extract GetFeature request + GetFeatureType getFeatureType = OwsUtils.parameter(operation.getParameters(), + GetFeatureType.class); + // generate an UUID (resultSetID) for this request, GeoTools complained about the - in the + // ID + resultSetId = UUID.randomUUID().toString().replaceAll("-", ""); + // store request and associate it to UUID + storeGetFeature(resultSetId, this.request); + super.write(value, output, operation); + } + + @Override + protected void encode(FeatureCollectionResponse hits, OutputStream output, WFSInfo wfs) + throws IOException { + + hits.setNumberOfFeatures(BigInteger.ZERO); + // instantiate the XML encoder + Encoder encoder = new Encoder(new WFSConfiguration()); + encoder.setEncoding(Charset.forName(wfs.getGeoServer().getSettings().getCharset())); + encoder.setSchemaLocation(WFS.NAMESPACE, + ResponseUtils.appendPath(wfs.getSchemaBaseURL(), "wfs/2.0/wfs.xsd")); + Document document; + try { + // encode the HITS result using FeatureCollection as the root XML element + document = encoder.encodeAsDOM(hits.getAdaptee(), WFS.FeatureCollection); + } catch (Exception exception) { + throw new RuntimeException("Error encoding INDEX result.", exception); + } + // add the resultSetID attribute to the result + addResultSetIdElement(document, resultSetId); + // write the XML document to response output stream + writeDocument(document, output); + } + + /** + * Helper method that serialize GetFeature request, store it in the file system and associate it + * with resultSetId + * + * @param resultSetId + * @param getFeatureType + * @throws RuntimeException + */ + private void storeGetFeature(String resultSetId, Request request) throws RuntimeException { + try { + IndexInitializer.READ_WRITE_LOCK.writeLock().lock(); + DataStore dataStore = this.indexConfiguration.getCurrentDataStore(); + // Create and store new feature + SimpleFeatureStore featureStore = (SimpleFeatureStore) dataStore + .getFeatureSource(IndexInitializer.STORE_SCHEMA_NAME); + SimpleFeatureBuilder builder = new SimpleFeatureBuilder(featureStore.getSchema()); + Long now = System.currentTimeMillis(); + // Add ID field value (see IndexInitializer.STORE_SCHEMA) + builder.add(resultSetId); + // Add created field value (see IndexInitializer.STORE_SCHEMA) + builder.add(now); + // Add updated field value (see IndexInitializer.STORE_SCHEMA) + builder.add(now); + SimpleFeature feature = builder.buildFeature(null); + SimpleFeatureCollection collection = new ListFeatureCollection(featureStore.getSchema(), + Arrays.asList(feature)); + featureStore.addFeatures(collection); + // Create and store file + Resource storageResource = this.indexConfiguration.getStorageResource(); + + // Serialize KVP parameters and the POST content + Map kvp = request.getKvp(); + Map rawKvp = request.getRawKvp(); + RequestData data = new RequestData(); + data.setKvp(kvp); + data.setRawKvp(rawKvp); + + FileOutputStream fos = new FileOutputStream( + storageResource.dir().getAbsolutePath() + "\\" + resultSetId + ".feature"); + BufferedOutputStream bos = new BufferedOutputStream(fos); + + ObjectOutputStream oos = new ObjectOutputStream(bos); + oos.writeObject(data); + oos.close(); + + } catch (Exception exception) { + throw new RuntimeException("Error storing feature.", exception); + } finally { + IndexInitializer.READ_WRITE_LOCK.writeLock().unlock(); + } + } + + /** + * Helper method that adds the resultSetID attribute to XML result. If no FeatureCollection + * element can be found nothing will be done. + */ + private static void addResultSetIdElement(Document document, String resultSetId) { + // search FeatureCollection XML nodes + NodeList nodes = document.getElementsByTagName("wfs:FeatureCollection"); + if (nodes.getLength() != 1) { + // only one node should exists, let's log an warning an move on + LOGGER.warning( + "No feature collection element could be found, resultSetID attribute will not be added."); + return; + } + // get the FeatureCollection node + Node node = nodes.item(0); + if (node.getNodeType() == Node.ELEMENT_NODE) { + // the found node is a XML element so let's add the resultSetID attribute + Element element = (Element) node; + element.setAttribute("resultSetID", resultSetId); + } else { + // unlikely but we got a XML node that is not a XML element + LOGGER.warning( + "Feature collection node is not a XML element, resultSetID attribute will not be added."); + } + } + + /** + * Helper method that just writes a XML document to a given output stream. + */ + private static void writeDocument(Document document, OutputStream output) { + // instantiate a new XML transformer + TransformerFactory transformerFactory = TransformerFactory.newInstance(); + Transformer transformer; + try { + transformer = transformerFactory.newTransformer(); + } catch (Exception exception) { + throw new RuntimeException("Error creating XML transformer.", exception); + } + // write the XML document to the provided output stream + DOMSource source = new DOMSource(document); + StreamResult result = new StreamResult(output); + try { + transformer.transform(source, result); + } catch (Exception exception) { + throw new RuntimeException("Error writing INDEX result to the output stream.", + exception); + } + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexResultTypeDispatcherCallback.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexResultTypeDispatcherCallback.java new file mode 100644 index 00000000000..7cd244c55ce --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/IndexResultTypeDispatcherCallback.java @@ -0,0 +1,80 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.util.logging.Logger; + +import org.geoserver.config.GeoServer; +import org.geoserver.ows.AbstractDispatcherCallback; +import org.geoserver.ows.Request; +import org.geoserver.ows.Response; +import org.geoserver.platform.Operation; +import org.geotools.util.logging.Logging; + +import net.opengis.wfs20.ResultTypeType; + +/** + *

+ * When a request that contains the "resultType" parameter arrives, if the parameter value is + * "index" it is substituted by "hits". + *

+ *

+ * A new entry named RESULT_TYPE_INDEX specifying that the original result type was "index" is added + * to KVP maps + *

+ *

+ * The object that manage response of type HitsOutputFormat is replaced with IndexOutputFormat + * before response has been dispatched + *

+ * + * @author sandr + * + */ + +public class IndexResultTypeDispatcherCallback extends AbstractDispatcherCallback { + + private final static Logger LOGGER = Logging.getLogger(IndexResultTypeDispatcherCallback.class); + + private GeoServer gs; + + private IndexConfiguration indexConfiguration; + + private static final String RESULT_TYPE_PARAMETER = "resultType"; + + private static final String RESULT_TYPE_INDEX = "index"; + + static final String RESULT_TYPE_INDEX_PARAMETER = "RESULT_TYPE_INDEX"; + + public IndexResultTypeDispatcherCallback(GeoServer gs, IndexConfiguration indexConfiguration) { + this.gs = gs; + this.indexConfiguration = indexConfiguration; + } + + @Override + @SuppressWarnings("unchecked") + public Request init(Request request) { + Object resultType = request.getKvp().get(RESULT_TYPE_PARAMETER); + if (resultType != null && resultType.toString().equals(RESULT_TYPE_INDEX)) { + request.getKvp().put(RESULT_TYPE_PARAMETER, ResultTypeType.HITS); + request.getKvp().put(RESULT_TYPE_INDEX_PARAMETER, true); + } + return super.init(request); + } + + @Override + public Response responseDispatched(Request request, Operation operation, Object result, + Response response) { + Response newResponse = response; + if (request.getKvp().get(RESULT_TYPE_INDEX_PARAMETER) != null + && (Boolean) request.getKvp().get(RESULT_TYPE_INDEX_PARAMETER)) { + IndexOutputFormat r = new IndexOutputFormat(this.gs, this.indexConfiguration); + r.setRequest(request); + newResponse = r; + } + return super.responseDispatched(request, operation, result, newResponse); + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/PageResultsDispatcherCallback.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/PageResultsDispatcherCallback.java new file mode 100644 index 00000000000..e6a25ce785c --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/PageResultsDispatcherCallback.java @@ -0,0 +1,67 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.util.Collections; +import java.util.logging.Logger; + +import org.geoserver.config.GeoServer; +import org.geoserver.ows.AbstractDispatcherCallback; +import org.geoserver.ows.Request; +import org.geoserver.platform.Operation; +import org.geoserver.platform.Service; +import org.geoserver.platform.ServiceException; +import org.geotools.util.logging.Logging; + +/** + * + * This dispatcher manages service of type {@link PageResultsWebFeatureService} and sets the + * parameter ResultSetID present on KVP map. + *

+ * Dummy featureId value is added to KVP map to allow dispatcher to manage it as usual WFS 2.0 + * request. + * + * @author sandr + * + */ + +public class PageResultsDispatcherCallback extends AbstractDispatcherCallback { + + private final static Logger LOGGER = Logging.getLogger(PageResultsDispatcherCallback.class); + + private GeoServer gs; + + public PageResultsDispatcherCallback(GeoServer gs) { + this.gs = gs; + } + + @SuppressWarnings("unchecked") + @Override + public Service serviceDispatched(Request request, Service service) throws ServiceException { + if (service.getService() instanceof PageResultsWebFeatureService) { + PageResultsWebFeatureService prService = (PageResultsWebFeatureService) service + .getService(); + String resultSetId = (String) request.getKvp().get("resultSetID"); + prService.setResultSetId(resultSetId); + request.getKvp().put("featureId", Collections.singletonList("dummy")); + + } + return super.serviceDispatched(request, service); + } + + @Override + public Operation operationDispatched(Request request, Operation operation) { + Operation newOperation = operation; + // Change operation from PageResults to GetFeature to allow management of request as + // standard get feature + if (operation.getId().equals("PageResults")) { + newOperation = new Operation("GetFeature", operation.getService(), + operation.getMethod(), operation.getParameters()); + } + return super.operationDispatched(request, newOperation); + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/PageResultsWebFeatureService.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/PageResultsWebFeatureService.java new file mode 100644 index 00000000000..db7e5979c3c --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/PageResultsWebFeatureService.java @@ -0,0 +1,157 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.io.BufferedInputStream; +import java.io.FileInputStream; +import java.io.IOException; +import java.io.ObjectInputStream; +import java.lang.reflect.Method; +import java.math.BigInteger; +import java.util.Date; +import java.util.logging.Logger; + +import org.geoserver.config.GeoServer; +import org.geoserver.ows.Dispatcher; +import org.geoserver.ows.KvpRequestReader; +import org.geoserver.ows.util.OwsUtils; +import org.geoserver.platform.resource.Resource; +import org.geoserver.wfs.DefaultWebFeatureService20; +import org.geoserver.wfs.request.FeatureCollectionResponse; +import org.geotools.data.DataStore; +import org.geotools.data.DefaultTransaction; +import org.geotools.data.Transaction; +import org.geotools.data.simple.SimpleFeatureStore; +import org.geotools.filter.text.cql2.CQL; +import org.geotools.util.logging.Logging; +import org.opengis.filter.Filter; + +import net.opengis.wfs20.GetFeatureType; +import net.opengis.wfs20.ResultTypeType; + +/** + * This service supports the PageResults operation and manage it + * + * @author sandr + * + */ + +public class PageResultsWebFeatureService extends DefaultWebFeatureService20 { + + static Logger LOGGER = Logging.getLogger(PageResultsWebFeatureService.class); + + private static final String GML32_FORMAT = "application/gml+xml; version=3.2"; + + private static final BigInteger DEFAULT_START = new BigInteger("0"); + + private static final BigInteger DEFAULT_COUNT = new BigInteger("10"); + + private String resultSetId; + + private IndexConfiguration indexConfiguration; + + public PageResultsWebFeatureService(GeoServer geoServer, + IndexConfiguration indexConfiguration) { + super(geoServer); + this.indexConfiguration = indexConfiguration; + } + + /** + * + * Recovers the stored request with associated {@link #resultSetID} and overrides the parameters + * using the ones provided with current operation or the default values: + *

    + *
  • {@link net.opengis.wfs20.GetFeatureType#getStartIndex StartIndex}
    • + *
  • {@link net.opengis.wfs20.GetFeatureType#getCount Count}
    • + *
  • {@link net.opengis.wfs20.GetFeatureType#getOutputFormat OutputFormat}
    • + *
  • {@link net.opengis.wfs20.GetFeatureType#getResultType ResultType}
    • + *
+ * Then executes the GetFeature operation using the WFS 2.0 service implementation and return is + * result. + * + * @param request + * @return + * @throws Exception + */ + public FeatureCollectionResponse pageResults(GetFeatureType request) throws Exception { + // Retrieve stored request + GetFeatureType gft = getFeature(this.resultSetId); + + // Update with incoming parameters or index request or defaults + Method setBaseUrl = OwsUtils.setter(gft.getClass(), "baseUrl", String.class); + setBaseUrl.invoke(gft, new Object[] { request.getBaseUrl() }); + BigInteger startIndex = request.getStartIndex() != null ? request.getStartIndex() + : gft.getStartIndex() != null ? gft.getStartIndex() : DEFAULT_START; + BigInteger count = request.getCount() != null ? request.getCount() + : gft.getCount() != null ? gft.getCount() : DEFAULT_COUNT; + String outputFormat = request.getOutputFormat() != null ? request.getOutputFormat() + : GML32_FORMAT; + ResultTypeType resultType = request.getResultType() != null ? request.getResultType() + : ResultTypeType.RESULTS; + gft.setStartIndex(startIndex); + gft.setCount(count); + gft.setOutputFormat(outputFormat); + gft.setResultType(resultType); + // Execute as getFeature + return super.getFeature(gft); + } + + /** + * Sets the resultSetId + * + * @param resultSetId + */ + public void setResultSetId(String resultSetId) { + this.resultSetId = resultSetId; + } + + /** + * Helper method that deserializes GetFeature request and updates its last utilization + * + * @param resultSetID + * @return + * @throws Exception + */ + private GetFeatureType getFeature(String resultSetId) throws IOException { + GetFeatureType feature = null; + Transaction transaction = new DefaultTransaction("Update"); + try { + IndexInitializer.READ_WRITE_LOCK.writeLock().lock(); + // Update GetFeature utilization + DataStore currentDataStore = this.indexConfiguration.getCurrentDataStore(); + SimpleFeatureStore store = (SimpleFeatureStore) currentDataStore + .getFeatureSource(IndexInitializer.STORE_SCHEMA_NAME); + store.setTransaction(transaction); + Filter filter = CQL.toFilter("ID = '" + resultSetId + "'"); + store.modifyFeatures("updated", new Date().getTime(), filter); + // Retrieve GetFeature from file + Resource storageResource = this.indexConfiguration.getStorageResource(); + + FileInputStream fis = new FileInputStream( + storageResource.dir().getAbsolutePath() + "\\" + resultSetId + ".feature"); + BufferedInputStream bis = new BufferedInputStream(fis); + + ObjectInputStream objectinputstream = new ObjectInputStream(bis); + RequestData data = (RequestData) objectinputstream.readObject(); + + objectinputstream.close(); + + KvpRequestReader kvpReader = Dispatcher.findKvpRequestReader(GetFeatureType.class); + Object requestBean = kvpReader.createRequest(); + feature = (GetFeatureType) kvpReader.read(requestBean, data.getKvp(), data.getRawKvp()); + + } catch (Exception t) { + transaction.rollback(); + throw new RuntimeException("Error on retrive feature", t); + } finally { + transaction.close(); + IndexInitializer.READ_WRITE_LOCK.writeLock().unlock(); + } + return feature; + + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/RequestData.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/RequestData.java new file mode 100644 index 00000000000..5613d142bf2 --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/RequestData.java @@ -0,0 +1,41 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import java.io.Serializable; +import java.util.Map; + +/** + * This class is used to store the data to serialize to recreate previous get feature request + * + * @author sandr + * + */ +class RequestData implements Serializable { + + private static final long serialVersionUID = 6687946816946977568L; + + private Map kvp; + + private Map rawKvp; + + public Map getKvp() { + return kvp; + } + + public void setKvp(Map kvp) { + this.kvp = kvp; + } + + public Map getRawKvp() { + return rawKvp; + } + + public void setRawKvp(Map rawKvp) { + this.rawKvp = rawKvp; + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/ResultTypeKvpParser.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/ResultTypeKvpParser.java new file mode 100644 index 00000000000..bdc797dea78 --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/pagination/random/ResultTypeKvpParser.java @@ -0,0 +1,23 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import org.geotools.util.Version; + +/** + * + * @author sandr + * + */ + +public class ResultTypeKvpParser extends org.geoserver.wfs.kvp.v2_0.ResultTypeKvpParser { + + public ResultTypeKvpParser() { + super(); + setVersion(new Version("2.0.2")); + } + +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/TimeVersioning.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/TimeVersioning.java new file mode 100644 index 00000000000..6abc73e47a4 --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/TimeVersioning.java @@ -0,0 +1,58 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.versioning; + +import org.geoserver.catalog.FeatureTypeInfo; + +public final class TimeVersioning { + + public static final String ENABLED_KEY = "TIME_VERSIONING_ENABLED"; + public static final String ID_PROPERTY_KEY = "TIME_VERSIONING_ID_PROPERTY"; + public static final String TIME_PROPERTY_KEY = "TIME_VERSIONING_TIME_PROPERTY"; + + public static void enable(FeatureTypeInfo featureTypeInfo, String idProperty, String timeProperty) { + featureTypeInfo.putParameter(ENABLED_KEY, true); + featureTypeInfo.putParameter(ID_PROPERTY_KEY, idProperty); + featureTypeInfo.putParameter(TIME_PROPERTY_KEY, timeProperty); + } + + public static void disable(FeatureTypeInfo featureTypeInfo) { + featureTypeInfo.putParameter(ENABLED_KEY, false); + featureTypeInfo.putParameter(ID_PROPERTY_KEY, null); + featureTypeInfo.putParameter(TIME_PROPERTY_KEY, null); + } + + public static boolean isEnabled(FeatureTypeInfo featureTypeInfo) { + return featureTypeInfo.getParameter(ENABLED_KEY, Boolean.class, false); + } + + public static String getIdPropertyName(FeatureTypeInfo featureTypeInfo) { + String idPropertyName = featureTypeInfo.getParameter(ID_PROPERTY_KEY, String.class, null); + if (idPropertyName == null) { + throw new RuntimeException("No id property name was provided."); + } + return idPropertyName; + } + + public static String getTimePropertyName(FeatureTypeInfo featureTypeInfo) { + String timePropertyName = featureTypeInfo.getParameter(TIME_PROPERTY_KEY, String.class, null); + if (timePropertyName == null) { + throw new RuntimeException("No time property name was provided."); + } + return timePropertyName; + } + + public static void setEnable(FeatureTypeInfo featureTypeInfo, boolean enable) { + featureTypeInfo.putParameter(ENABLED_KEY, enable); + } + + public static void setIdAttribute(FeatureTypeInfo featureTypeInfo, String idAttributeName) { + featureTypeInfo.putParameter(ID_PROPERTY_KEY, idAttributeName); + } + + public static void setTimeAttribute(FeatureTypeInfo featureTypeInfo, String timeAttributeName) { + featureTypeInfo.putParameter(TIME_PROPERTY_KEY, timeAttributeName); + } +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/TimeVersioningCallback.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/TimeVersioningCallback.java new file mode 100644 index 00000000000..8b45559076a --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/TimeVersioningCallback.java @@ -0,0 +1,244 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.versioning; + +import org.geoserver.catalog.Catalog; +import org.geoserver.catalog.FeatureTypeInfo; +import org.geoserver.platform.GeoServerExtensions; +import org.geoserver.wfs.GetFeatureCallback; +import org.geoserver.wfs.GetFeatureContext; +import org.geoserver.wfs.InsertElementHandler; +import org.geoserver.wfs.TransactionCallback; +import org.geoserver.wfs.TransactionContext; +import org.geoserver.wfs.TransactionContextBuilder; +import org.geoserver.wfs.request.Insert; +import org.geoserver.wfs.request.RequestObject; +import org.geoserver.wfs.request.Update; +import org.geotools.data.DataUtilities; +import org.geotools.data.FeatureStore; +import org.geotools.data.Query; +import org.geotools.data.simple.SimpleFeatureCollection; +import org.geotools.data.simple.SimpleFeatureIterator; +import org.geotools.data.simple.SimpleFeatureStore; +import org.geotools.factory.CommonFactoryFinder; +import org.geotools.factory.GeoTools; +import org.geotools.feature.NameImpl; +import org.geotools.feature.simple.SimpleFeatureBuilder; +import org.geotools.util.Converters; +import org.opengis.feature.simple.SimpleFeature; +import org.opengis.feature.type.AttributeDescriptor; +import org.opengis.feature.type.FeatureType; +import org.opengis.feature.type.Name; +import org.opengis.filter.Filter; +import org.opengis.filter.FilterFactory2; +import org.opengis.filter.sort.SortBy; +import org.opengis.filter.sort.SortOrder; + +import javax.xml.namespace.QName; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Comparator; +import java.util.Date; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +final class TimeVersioningCallback implements GetFeatureCallback, TransactionCallback { + + private static final FilterFactory2 FILTER_FACTORY = CommonFactoryFinder.getFilterFactory2(GeoTools.getDefaultHints()); + + private final Catalog catalog; + + TimeVersioningCallback(Catalog catalog) { + this.catalog = catalog; + } + + @Override + public GetFeatureContext beforeQuerying(GetFeatureContext context) { + if (!isWfs20(context.getRequest())) { + return context; + } + FeatureTypeInfo featureTypeInfo = context.getFeatureTypeInfo(); + if (!TimeVersioning.isEnabled(featureTypeInfo)) { + // time versioning is not enabled for this feature type or is not a WFS 2.0 request + return context; + } + VersioningFilterAdapter.adapt(featureTypeInfo, context.getQuery().getFilter()); + SortBy sort = FILTER_FACTORY.sort(TimeVersioning.getTimePropertyName(featureTypeInfo), SortOrder.DESCENDING); + SortBy[] sorts = context.getQuery().getSortBy(); + if (sorts == null) { + sorts = new SortBy[]{sort}; + } else { + sorts = Arrays.copyOf(sorts, sorts.length + 1); + sorts[sorts.length - 1] = sort; + } + context.getQuery().setSortBy(sorts); + return context; + } + + @Override + public TransactionContext beforeHandlerExecution(TransactionContext context) { + if (!isWfs20(context.getRequest())) { + return context; + } + if (context.getElement() instanceof Update) { + Insert insert = buildInsertForUpdate(context); + InsertElementHandler handler = GeoServerExtensions.bean(InsertElementHandler.class); + return new TransactionContextBuilder() + .withContext(context) + .withElement(insert) + .withHandler(handler).build(); + } + if (context.getElement() instanceof Insert) { + Insert insert = (Insert) context.getElement(); + for (Object element : insert.getFeatures()) { + if (element instanceof SimpleFeature) { + setTimeAttribute((SimpleFeature) element); + } + } + } + return context; + } + + @Override + public TransactionContext beforeInsertFeatures(TransactionContext context) { + return context; + } + + @Override + public TransactionContext beforeUpdateFeatures(TransactionContext context) { + return context; + } + + @Override + public TransactionContext beforeDeleteFeatures(TransactionContext context) { + return context; + } + + @Override + public TransactionContext beforeReplaceFeatures(TransactionContext context) { + return context; + } + + private void setTimeAttribute(SimpleFeature feature) { + FeatureType featureType = feature.getFeatureType(); + FeatureTypeInfo featureTypeInfo = getFeatureTypeInfo(featureType); + if (TimeVersioning.isEnabled(featureTypeInfo)) { + String timePropertyName = TimeVersioning.getTimePropertyName(featureTypeInfo); + AttributeDescriptor attributeDescriptor = feature.getType().getDescriptor(timePropertyName); + Object timeValue = Converters.convert(new Date(), attributeDescriptor.getType().getBinding()); + feature.setAttribute(timePropertyName, timeValue); + } + } + + private SimpleFeatureCollection getTransactionFeatures(TransactionContext context) { + QName typeName = context.getElement().getTypeName(); + Filter filter = context.getElement().getFilter(); + FeatureTypeInfo featureTypeInfo = getFeatureTypeInfo(new NameImpl(typeName)); + SimpleFeatureStore store = getTransactionStore(context); + try { + Query query = new Query(); + query.setFilter(VersioningFilterAdapter.adapt(featureTypeInfo, filter)); + SortBy sort = FILTER_FACTORY.sort(TimeVersioning.getTimePropertyName(featureTypeInfo), SortOrder.DESCENDING); + query.setSortBy(new SortBy[]{sort}); + return store.getFeatures(query); + } catch (Exception exception) { + throw new RuntimeException(String.format( + "Error getting features of type '%s'.", typeName), exception); + } + } + + private Comparator buildFeatureTimeComparator(FeatureTypeInfo featureTypeInfo) { + String timePropertyName = TimeVersioning.getTimePropertyName(featureTypeInfo); + return (featureA, featureB) -> { + Date timeA = Converters.convert(featureA.getAttribute(timePropertyName), Date.class); + Date timeB = Converters.convert(featureB.getAttribute(timePropertyName), Date.class); + if (timeA == null) { + return -1; + } + return timeA.compareTo(timeB); + }; + } + + private List getOnlyRecentFeatures(SimpleFeatureCollection features, FeatureTypeInfo featureTypeInfo) { + String idPropertyName = TimeVersioning.getIdPropertyName(featureTypeInfo); + Map> featuresIndexedById = new HashMap<>(); + SimpleFeatureIterator iterator = features.features(); + while (iterator.hasNext()) { + SimpleFeature feature = iterator.next(); + Object id = feature.getAttribute(idPropertyName); + List existing = featuresIndexedById.computeIfAbsent(id, key -> new ArrayList<>()); + existing.add(feature); + } + Comparator comparator = buildFeatureTimeComparator(featureTypeInfo); + List finalFeatures = new ArrayList<>(); + featuresIndexedById.values().forEach(indexed -> { + indexed.sort(comparator); + SimpleFeature feature = indexed.get(0); + SimpleFeatureBuilder builder = new SimpleFeatureBuilder(feature.getFeatureType()); + builder.init(feature); + finalFeatures.add(builder.buildFeature(null)); + }); + return finalFeatures; + } + + private Insert buildInsertForUpdate(TransactionContext context) { + Update update = (Update) context.getElement(); + FeatureTypeInfo featureTypeInfo = getFeatureTypeInfo(new NameImpl(update.getTypeName())); + SimpleFeatureCollection features = getTransactionFeatures(context); + List recent = getOnlyRecentFeatures(features, featureTypeInfo); + List newFeatures = recent.stream() + .map(this::prepareInsertFeature).collect(Collectors.toList()); + return new UpdateInsert(context.getRequest(), newFeatures); + } + + private SimpleFeature prepareInsertFeature(SimpleFeature feature) { + SimpleFeatureBuilder builder = new SimpleFeatureBuilder(feature.getFeatureType()); + builder.init(feature); + SimpleFeature versionedFeature = builder.buildFeature(null); + setTimeAttribute(versionedFeature); + return versionedFeature; + } + + private FeatureTypeInfo getFeatureTypeInfo(FeatureType featureType) { + Name featureTypeName = featureType.getName(); + return getFeatureTypeInfo(featureTypeName); + } + + private FeatureTypeInfo getFeatureTypeInfo(Name featureTypeName) { + FeatureTypeInfo featureTypeInfo = catalog.getFeatureTypeByName(featureTypeName); + if (featureTypeInfo == null) { + throw new RuntimeException(String.format( + "Couldn't find feature type info ''%s.", featureTypeName)); + } + return featureTypeInfo; + } + + private SimpleFeatureStore getTransactionStore(TransactionContext context) { + QName typeName = context.getElement().getTypeName(); + FeatureStore store = (FeatureStore) context.getFeatureStores().get(typeName); + return DataUtilities.simple(store); + } + + private boolean isWfs20(RequestObject request) { + return true; + } + + private static final class UpdateInsert extends Insert { + + private final List features; + + protected UpdateInsert(RequestObject request, List features) { + super(request.getAdaptee()); + this.features = features; + } + + @Override + public List getFeatures() { + return features; + } + } +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/VersioningFilterAdapter.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/VersioningFilterAdapter.java new file mode 100644 index 00000000000..42be444be5d --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/VersioningFilterAdapter.java @@ -0,0 +1,106 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.versioning; + +import org.geoserver.catalog.FeatureTypeInfo; +import org.geotools.filter.visitor.DuplicatingFilterVisitor; +import org.opengis.filter.Filter; +import org.opengis.filter.FilterFactory; +import org.opengis.filter.Id; +import org.opengis.filter.expression.Expression; +import org.opengis.filter.identity.Identifier; +import org.opengis.filter.identity.ResourceId; + +import java.util.Date; +import java.util.HashSet; +import java.util.Set; + +final class VersioningFilterAdapter extends DuplicatingFilterVisitor { + + private final String idPropertyName; + private final String timePropertyName; + + private VersioningFilterAdapter(FeatureTypeInfo featureTypeInfo) { + this.idPropertyName = TimeVersioning.getIdPropertyName(featureTypeInfo); + this.timePropertyName = TimeVersioning.getTimePropertyName(featureTypeInfo); + } + + @Override + public Object visit(Id filter, Object extraData) { + FilterFactory filterFactory = getFactory(extraData); + Set ids = filter.getIdentifiers(); + Set finalIds = new HashSet<>(); + Filter versioningFilter = null; + for (Identifier id : ids) { + if (id instanceof ResourceId) { + Filter newFilter = buildVersioningFilter(filterFactory, (ResourceId) id); + versioningFilter = addFilter(filterFactory, versioningFilter, newFilter); + } else { + finalIds.add(id); + } + } + if (finalIds.isEmpty()) { + return versioningFilter; + } + Filter newIdFilter = getFactory(extraData).id(finalIds); + if (versioningFilter != null) { + return filterFactory.and(newIdFilter, versioningFilter); + } + return newIdFilter; + } + + private Filter buildVersioningFilter(FilterFactory filterFactory, ResourceId resourceId) { + Filter idFilter = buildIdFilter(filterFactory, resourceId.getID()); + Filter timeFilter = buildTimeFilter(filterFactory, resourceId.getStartTime(), resourceId.getEndTime()); + if (idFilter != null && timeFilter != null) { + return filterFactory.and(idFilter, timeFilter); + } + if (idFilter != null) { + return idFilter; + } + if (timeFilter != null) { + return timeFilter; + } + return null; + } + + private Filter buildIdFilter(FilterFactory factory, String id) { + if (id == null) { + return null; + } + return factory.equals(factory.property(idPropertyName), factory.literal(id)); + } + + private Filter buildTimeFilter(FilterFactory filterFactory, Date start, Date end) { + Expression timeProperty = filterFactory.property(timePropertyName); + Expression startLiteral = filterFactory.literal(start); + Expression endLiteral = filterFactory.literal(end); + Filter after = filterFactory.after(timeProperty, startLiteral); + Filter before = filterFactory.before(timeProperty, endLiteral); + if (start != null && end != null) { + return filterFactory.and(after, before); + } + if (start != null) { + return after; + } + if (end != null) { + return before; + } + return null; + } + + private Filter addFilter(FilterFactory filterFactory, Filter versioningFilter, Filter filter) { + if (versioningFilter != null) { + return filterFactory.and(versioningFilter, filter); + } + return filter; + } + + static Filter adapt(FeatureTypeInfo featureTypeInfo, Filter filter) { + String timePropertyName = TimeVersioning.getTimePropertyName(featureTypeInfo); + VersioningFilterAdapter adapter = new VersioningFilterAdapter(featureTypeInfo); + return (Filter) filter.accept(adapter, null); + } +} diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/web/WfsVersioningConfig.html b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/web/WfsVersioningConfig.html new file mode 100644 index 00000000000..b67d24989c7 --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/web/WfsVersioningConfig.html @@ -0,0 +1,32 @@ + + + +
+ +

+ WFS Versioning +

+
    +
  • +
    + + +
    +
    • +
  • + +
    • +
  • + +
    • +
  • + +
    • +
  • + +
    • +
+ +
+ + diff --git a/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/web/WfsVersioningConfig.java b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/web/WfsVersioningConfig.java new file mode 100644 index 00000000000..f575a2c96ec --- /dev/null +++ b/src/community/nsg-profile/src/main/java/org/geoserver/nsg/versioning/web/WfsVersioningConfig.java @@ -0,0 +1,153 @@ +/* (c) 2014 - 2016 Open Source Geospatial Foundation - all rights reserved + * (c) 2001 - 2013 OpenPlans + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.versioning.web; + +import org.apache.wicket.ajax.AjaxRequestTarget; +import org.apache.wicket.ajax.form.AjaxFormComponentUpdatingBehavior; +import org.apache.wicket.ajax.markup.html.form.AjaxCheckBox; +import org.apache.wicket.markup.html.basic.Label; +import org.apache.wicket.markup.html.form.CheckBox; +import org.apache.wicket.markup.html.form.DropDownChoice; +import org.apache.wicket.model.IModel; +import org.apache.wicket.model.Model; +import org.apache.wicket.model.StringResourceModel; +import org.geoserver.catalog.FeatureTypeInfo; +import org.geoserver.catalog.LayerInfo; +import org.geoserver.nsg.versioning.TimeVersioning; +import org.geoserver.web.publish.PublishedConfigurationPanel; + +import java.sql.Timestamp; +import java.util.Date; +import java.util.List; +import java.util.stream.Collectors; + +public class WfsVersioningConfig extends PublishedConfigurationPanel { + + public WfsVersioningConfig(String id, IModel model) { + super(id, model); + // get the needed information from the model + FeatureTypeInfo featureTypeInfo = getFeatureTypeInfo(model); + boolean isVersioningActivated = TimeVersioning.isEnabled(featureTypeInfo); + String idAttributeName = isVersioningActivated ? TimeVersioning.getIdPropertyName(featureTypeInfo) : null; + String timeAttributeName = isVersioningActivated ? TimeVersioning.getTimePropertyName(featureTypeInfo) : null; + List attributesNames = getAttributesNames(featureTypeInfo); + List timeAttributesNames = getTimeAttributesNames(featureTypeInfo); + // create dropdown choice for the id attribute name + DropDownChoice idAttributeChoice = new DropDownChoice<>("idAttributeChoice", + new Model<>(idAttributeName), attributesNames); + idAttributeChoice.add(new AjaxFormComponentUpdatingBehavior("change") { + @Override + protected void onUpdate(AjaxRequestTarget target) { + String selected = idAttributeChoice.getModel().getObject(); + TimeVersioning.setIdAttribute(featureTypeInfo, selected); + } + }); + idAttributeChoice.setOutputMarkupId(true); + idAttributeChoice.setOutputMarkupPlaceholderTag(true); + idAttributeChoice.setRequired(true); + idAttributeChoice.setVisible(isVersioningActivated); + add(idAttributeChoice); + // add label for id attribute name dropdown choice + Label idAttributeChoiceLabel = new Label("idAttributeChoiceLabel", + new StringResourceModel("WfsVersioningConfig.idAttributeChoiceLabel")); + idAttributeChoiceLabel.setOutputMarkupId(true); + idAttributeChoiceLabel.setOutputMarkupPlaceholderTag(true); + idAttributeChoiceLabel.setVisible(isVersioningActivated); + add(idAttributeChoiceLabel); + // create dropdown choice for the time attribute name + DropDownChoice timeAttributeChoice = new DropDownChoice<>("timeAttributeChoice", + new Model<>(timeAttributeName), timeAttributesNames); + timeAttributeChoice.add(new AjaxFormComponentUpdatingBehavior("change") { + @Override + protected void onUpdate(AjaxRequestTarget target) { + String selected = timeAttributeChoice.getModel().getObject(); + TimeVersioning.setTimeAttribute(featureTypeInfo, selected); + } + }); + timeAttributeChoice.setOutputMarkupId(true); + timeAttributeChoice.setOutputMarkupPlaceholderTag(true); + timeAttributeChoice.setRequired(true); + timeAttributeChoice.setVisible(isVersioningActivated); + add(timeAttributeChoice); + // add label for id attribute name dropdown choice + Label timeAttributeChoiceLabel = new Label("timeAttributeChoiceLabel", + new StringResourceModel("WfsVersioningConfig.timeAttributeChoiceLabel")); + timeAttributeChoiceLabel.setOutputMarkupId(true); + timeAttributeChoiceLabel.setOutputMarkupPlaceholderTag(true); + timeAttributeChoiceLabel.setVisible(isVersioningActivated); + add(timeAttributeChoiceLabel); + // checkbox for activating versioning + CheckBox versioningActivateCheckBox = new AjaxCheckBox("versioningActivateCheckBox", + new Model<>(isVersioningActivated)) { + @Override + protected void onUpdate(AjaxRequestTarget target) { + boolean checked = getModelObject(); + if (checked) { + // activate versioning attributes selection + idAttributeChoice.setVisible(true); + idAttributeChoiceLabel.setVisible(true); + timeAttributeChoice.setVisible(true); + timeAttributeChoiceLabel.setVisible(true); + // enable time versioning + TimeVersioning.setEnable(featureTypeInfo, true); + } else { + // deactivate versioning attributes selection + idAttributeChoice.setVisible(false); + idAttributeChoiceLabel.setVisible(false); + timeAttributeChoice.setVisible(false); + timeAttributeChoiceLabel.setVisible(false); + // disable time versioning + TimeVersioning.setEnable(featureTypeInfo, false); + } + // update the dropdown choices and labels + target.add(idAttributeChoice); + target.add(idAttributeChoiceLabel); + target.add(timeAttributeChoice); + target.add(timeAttributeChoiceLabel); + } + }; + if (isVersioningActivated) { + versioningActivateCheckBox.setModelObject(true); + } + versioningActivateCheckBox.setEnabled(!timeAttributesNames.isEmpty()); + add(versioningActivateCheckBox); + // add versioning activating checkbox label + Label versioningActivateCheckBoxLabel = new Label("versioningActivateCheckBoxLabel", + new StringResourceModel("WfsVersioningConfig.versioningActivateCheckBoxLabel")); + add(versioningActivateCheckBoxLabel); + } + + private List getAttributesNames(FeatureTypeInfo featureTypeInfo) { + try { + return featureTypeInfo.getFeatureType().getDescriptors().stream() + .map(attribute -> attribute.getName().getLocalPart()) + .collect(Collectors.toList()); + } catch (Exception exception) { + throw new RuntimeException(String.format( + "Error processing attributes of feature type '%s'.", + featureTypeInfo.getName()), exception); + } + } + + private List getTimeAttributesNames(FeatureTypeInfo featureTypeInfo) { + try { + return featureTypeInfo.getFeatureType().getDescriptors().stream().filter(attribute -> { + Class binding = attribute.getType().getBinding(); + return Long.class.isAssignableFrom(binding) + || Date.class.isAssignableFrom(binding) + || Timestamp.class.isAssignableFrom(binding); + }).map(attribute -> attribute.getName().getLocalPart()).collect(Collectors.toList()); + } catch (Exception exception) { + throw new RuntimeException(String.format( + "Error processing attributes of feature type '%s'.", + featureTypeInfo.getName()), exception); + } + } + + private FeatureTypeInfo getFeatureTypeInfo(IModel model) { + return (FeatureTypeInfo) model.getObject().getResource(); + } +} diff --git a/src/community/nsg-profile/src/main/resources/GeoServerApplication.properties b/src/community/nsg-profile/src/main/resources/GeoServerApplication.properties new file mode 100644 index 00000000000..e8f4ddacbf6 --- /dev/null +++ b/src/community/nsg-profile/src/main/resources/GeoServerApplication.properties @@ -0,0 +1,4 @@ +WfsVersioningConfig.wfsVersioning=WFS Versioning +WfsVersioningConfig.versioningActivateCheckBoxLabel=Activate Versioning +WfsVersioningConfig.idAttributeChoiceLabel=Id Attribute +WfsVersioningConfig.timeAttributeChoiceLabel=Time Attribute \ No newline at end of file diff --git a/src/community/nsg-profile/src/main/resources/applicationContext.xml b/src/community/nsg-profile/src/main/resources/applicationContext.xml new file mode 100644 index 00000000000..b3f815b3a10 --- /dev/null +++ b/src/community/nsg-profile/src/main/resources/applicationContext.xml @@ -0,0 +1,70 @@ + + + + + + + + + + + + org.geoserver.catalog.FeatureTypeInfo + + + + + + + + + When a request using the index result type comes in a + dispatcher callback + this switch the "index" value by the "hits" value + + + + + + + + The PageResults operation will allow clients to query + random positions of an existing result set (stored GetFeature + request) that was previously created using the index result type + + + + + + + + + + + + + + + + PageResults + + + + + + + + + + + + diff --git a/src/community/nsg-profile/src/main/resources/configuration.properties b/src/community/nsg-profile/src/main/resources/configuration.properties new file mode 100644 index 00000000000..a1bf0a47f87 --- /dev/null +++ b/src/community/nsg-profile/src/main/resources/configuration.properties @@ -0,0 +1,6 @@ +resultSets.storage.path=${GEOSERVER_DATA_DIR}/nsg-profile/resultSets +resultSets.timeToLive=600 +resultSets.db.dbtype=h2 +resultSets.db.database=${GEOSERVER_DATA_DIR}/nsg-profile/db/resultSets +resultSets.db.user=sa +resultSets.db.password=sa \ No newline at end of file diff --git a/src/community/nsg-profile/src/test/java/org/geoserver/nsg/TestsUtils.java b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/TestsUtils.java new file mode 100644 index 00000000000..3e64290a4fa --- /dev/null +++ b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/TestsUtils.java @@ -0,0 +1,39 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg; + +import org.geoserver.catalog.Catalog; +import org.geoserver.catalog.FeatureTypeInfo; +import org.geoserver.util.IOUtils; + +import java.io.InputStream; + +import static org.geoserver.nsg.versioning.TimeVersioning.disable; +import static org.geoserver.nsg.versioning.TimeVersioning.enable; + +public final class TestsUtils { + + private TestsUtils() { + } + + public static String readResource(String resourceName) { + try (InputStream input = TestsUtils.class.getResourceAsStream(resourceName)) { + return IOUtils.toString(input); + } catch (Exception exception) { + throw new RuntimeException(String.format("Error reading resource '%s'.", resourceName)); + } + } + + public static void updateFeatureTypeTimeVersioning(Catalog catalog, String featureTypeName, + boolean enabled, String idProperty, String timeProperty) { + FeatureTypeInfo featureType = catalog.getFeatureTypeByName(featureTypeName); + if (enabled) { + enable(featureType, idProperty, timeProperty); + } else { + disable(featureType); + } + catalog.save(featureType); + } +} diff --git a/src/community/nsg-profile/src/test/java/org/geoserver/nsg/pagination/random/GetFeatureRequestStorageTest.java b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/pagination/random/GetFeatureRequestStorageTest.java new file mode 100644 index 00000000000..43a0de6e65c --- /dev/null +++ b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/pagination/random/GetFeatureRequestStorageTest.java @@ -0,0 +1,189 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.pagination.random; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import java.io.InputStream; +import java.io.OutputStream; +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutorCompletionService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; + +import org.apache.wicket.util.file.File; +import org.geoserver.config.GeoServerDataDirectory; +import org.geoserver.data.test.SystemTestData; +import org.geoserver.platform.GeoServerExtensions; +import org.geoserver.platform.GeoServerResourceLoader; +import org.geoserver.platform.resource.Resource; +import org.geoserver.wfs.v2_0.WFS20TestSupport; +import org.geotools.data.DataStore; +import org.geotools.data.simple.SimpleFeatureStore; +import org.geotools.filter.text.cql2.CQL; +import org.geotools.jdbc.JDBCDataStoreFactory; +import org.junit.Test; +import org.opengis.feature.simple.SimpleFeature; +import org.w3c.dom.Document; + +public class GetFeatureRequestStorageTest extends WFS20TestSupport { + + @Override + protected void onTearDown(SystemTestData testData) throws Exception { + IndexConfiguration ic = applicationContext.getBean(IndexConfiguration.class); + DataStore dataStore = ic.getCurrentDataStore(); + dataStore.dispose(); + super.onTearDown(testData); + } + + @Test + public void testCleanOldRequest() throws Exception { + Document doc = getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index"); + String resultSetId = doc.getDocumentElement().getAttribute("resultSetID"); + IndexConfiguration ic = applicationContext.getBean(IndexConfiguration.class); + DataStore dataStore = ic.getCurrentDataStore(); + SimpleFeatureStore featureStore = (SimpleFeatureStore) dataStore + .getFeatureSource(IndexInitializer.STORE_SCHEMA_NAME); + SimpleFeature feature = featureStore.getFeatures(CQL.toFilter("ID='" + resultSetId + "'")) + .features().next(); + assertNotNull(feature); + + // Change timeout from default to 5 seconds + GeoServerResourceLoader loader = GeoServerExtensions.bean(GeoServerResourceLoader.class); + GeoServerDataDirectory dd = new GeoServerDataDirectory(loader); + Properties properties = new Properties(); + Resource resource = dd + .get(IndexInitializer.MODULE_DIR + "/" + IndexInitializer.PROPERTY_FILENAME); + InputStream is = resource.in(); + properties.load(is); + is.close(); + properties.put("resultSets.timeToLive", "5"); + OutputStream out = resource.out(); + properties.store(out, null); + out.close(); + final CountDownLatch done1 = new CountDownLatch(1); + new Thread(new Runnable() { + @Override + public void run() { + while (true) { + try { + Long ttl = ic.getTimeToLiveInSec(); + if (ttl == 5) { + done1.countDown(); + break; + } + Thread.sleep(100); + } catch (Exception e) { + } + } + } + }).start(); + done1.await(100, TimeUnit.SECONDS); + assertEquals(new Long(5), ic.getTimeToLiveInSec()); + // Check that feature not used is deleted + final CountDownLatch done2 = new CountDownLatch(1); + new Thread(new Runnable() { + @Override + public void run() { + while (true) { + try { + IndexInitializer.READ_WRITE_LOCK.readLock().lock(); + boolean exists = featureStore + .getFeatures(CQL.toFilter("ID='" + resultSetId + "'")).features() + .hasNext(); + IndexInitializer.READ_WRITE_LOCK.readLock().unlock(); + if (!exists) { + done2.countDown(); + break; + } + Thread.sleep(100); + } catch (Exception e) { + } + } + } + }).start(); + done2.await(100, TimeUnit.SECONDS); + boolean exists = featureStore.getFeatures(CQL.toFilter("ID='" + resultSetId + "'")) + .features().hasNext(); + assertFalse(exists); + } + + @Test + public void testConcurrentChangeDatabaseParameters() throws Exception { + GeoServerResourceLoader loader = GeoServerExtensions.bean(GeoServerResourceLoader.class); + GeoServerDataDirectory dd = new GeoServerDataDirectory(loader); + Resource resource = dd + .get(IndexInitializer.MODULE_DIR + "/" + IndexInitializer.PROPERTY_FILENAME); + Properties properties = new Properties(); + InputStream is = resource.in(); + properties.load(is); + is.close(); + properties.put(IndexInitializer.PROPERTY_DB_PREFIX + JDBCDataStoreFactory.DATABASE.key, + dd.root().getPath() + "/nsg-profile/db/resultSets2"); + IndexConfiguration ic = applicationContext.getBean(IndexConfiguration.class); + ExecutorCompletionService es = new ExecutorCompletionService<>( + Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors())); + final int REQUESTS = 100; + for (int i = 0; i < REQUESTS; i++) { + int count = i; + es.submit(() -> { + getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index"); + if (count == REQUESTS / 2) { + // Change database + OutputStream out = resource.out(); + properties.store(out, null); + out.close(); + } + return null; + }); + } + // just check there are no exceptions + for (int i = 0; i < REQUESTS; i++) { + es.take().get(); + } + // wait for listener receive database notification changes + File dbDataFile = new File(dd.root().getPath() + "/nsg-profile/db/resultSets2.data.db"); + final CountDownLatch done = new CountDownLatch(1); + new Thread(new Runnable() { + @Override + public void run() { + while (true) { + Map params = ic.getCurrentDataStoreParams(); + boolean condition = (dd.root().getPath() + "/nsg-profile/db/resultSets2") + .equals(params.get(JDBCDataStoreFactory.DATABASE.key)) + && dbDataFile.exists(); + if (condition) { + done.countDown(); + break; + } + try { + Thread.sleep(100); + } catch (Exception e) { + } + } + } + }).start(); + done.await(100, TimeUnit.SECONDS); + + DataStore dataStore = ic.getCurrentDataStore(); + assertTrue(dbDataFile.exists()); + + Map params = ic.getCurrentDataStoreParams(); + assertEquals(dd.root().getPath() + "/nsg-profile/db/resultSets2", + params.get(JDBCDataStoreFactory.DATABASE.key)); + + SimpleFeatureStore featureStore = (SimpleFeatureStore) dataStore + .getFeatureSource(IndexInitializer.STORE_SCHEMA_NAME); + assertEquals(REQUESTS, featureStore.getFeatures().size()); + } + +} diff --git a/src/community/nsg-profile/src/test/java/org/geoserver/nsg/pagination/random/IndexResultTypeTest.java b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/pagination/random/IndexResultTypeTest.java new file mode 100644 index 00000000000..131c939db26 --- /dev/null +++ b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/pagination/random/IndexResultTypeTest.java @@ -0,0 +1,131 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ + +package org.geoserver.nsg.pagination.random; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; + +import org.custommonkey.xmlunit.XMLAssert; +import org.geoserver.data.test.SystemTestData; +import org.geoserver.wfs.v2_0.WFS20TestSupport; +import org.geotools.data.DataStore; +import org.geotools.data.simple.SimpleFeatureStore; +import org.geotools.filter.text.cql2.CQL; +import org.junit.Test; +import org.opengis.feature.simple.SimpleFeature; +import org.w3c.dom.Document; + +public class IndexResultTypeTest extends WFS20TestSupport { + + @Override + protected void onTearDown(SystemTestData testData) throws Exception { + IndexConfiguration ic = applicationContext.getBean(IndexConfiguration.class); + DataStore dataStore = ic.getCurrentDataStore(); + dataStore.dispose(); + super.onTearDown(testData); + } + + @Test + public void testIndexGet() throws Exception { + Document doc = getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index"); + assertGML32(doc); + assertEquals("15", doc.getDocumentElement().getAttribute("numberMatched")); + assertEquals("0", doc.getDocumentElement().getAttribute("numberReturned")); + XMLAssert.assertXpathEvaluatesTo("0", "count(//cdf:Fifteen)", doc); + IndexConfiguration ic = applicationContext.getBean(IndexConfiguration.class); + DataStore dataStore = ic.getCurrentDataStore(); + SimpleFeatureStore featureStore = (SimpleFeatureStore) dataStore + .getFeatureSource(IndexInitializer.STORE_SCHEMA_NAME); + String resultSetId = doc.getDocumentElement().getAttribute("resultSetID"); + SimpleFeature feature = featureStore.getFeatures(CQL.toFilter("ID='" + resultSetId + "'")) + .features().next(); + assertNotNull(feature); + } + + @Test + public void testIndexPaginationGet() throws Exception { + Document doc = getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index&count=2&startIndex=6"); + assertGML32(doc); + assertEquals("15", doc.getDocumentElement().getAttribute("numberMatched")); + assertEquals("0", doc.getDocumentElement().getAttribute("numberReturned")); + assertEquals( + "http://localhost:8080/geoserver/wfs?REQUEST=GetFeature&RESULTTYPE=index&VERSION=2.0.0&TYPENAMES=cdf%3AFifteen&SERVICE=WFS&COUNT=2&STARTINDEX=4", + doc.getDocumentElement().getAttribute("previous")); + assertEquals( + "http://localhost:8080/geoserver/wfs?REQUEST=GetFeature&RESULTTYPE=index&VERSION=2.0.0&TYPENAMES=cdf%3AFifteen&SERVICE=WFS&COUNT=2&STARTINDEX=8", + doc.getDocumentElement().getAttribute("next")); + XMLAssert.assertXpathEvaluatesTo("0", "count(//cdf:Fifteen)", doc); + IndexConfiguration ic = applicationContext.getBean(IndexConfiguration.class); + DataStore dataStore = ic.getCurrentDataStore(); + SimpleFeatureStore featureStore = (SimpleFeatureStore) dataStore + .getFeatureSource(IndexInitializer.STORE_SCHEMA_NAME); + String resultSetId = doc.getDocumentElement().getAttribute("resultSetID"); + SimpleFeature feature = featureStore.getFeatures(CQL.toFilter("ID='" + resultSetId + "'")) + .features().next(); + assertNotNull(feature); + } + + @Test + public void testPageResultIndexPaginationGet() throws Exception { + Document docIndex = getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index&count=2&startIndex=6"); + String resultSetId = docIndex.getDocumentElement().getAttribute("resultSetID"); + Document docResutl = getAsDOM( + "ows?service=WFS&version=2.0.2&request=PageResults&resultSetID=" + resultSetId); + XMLAssert.assertXpathEvaluatesTo("2", "count(//cdf:Fifteen)", docResutl); + assertStartIndexCount(docResutl, "previous", 4, 2); + assertStartIndexCount(docResutl, "next", 8, 2); + } + + @Test + public void testPageResultDefaultPaginationGet() throws Exception { + Document docIndex = getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index"); + String resultSetId = docIndex.getDocumentElement().getAttribute("resultSetID"); + Document docResutl = getAsDOM( + "ows?service=WFS&version=2.0.2&request=PageResults&resultSetID=" + resultSetId); + XMLAssert.assertXpathEvaluatesTo("10", "count(//cdf:Fifteen)", docResutl); + } + + @Test + public void testPageResultOverridePaginationGet() throws Exception { + Document docIndex = getAsDOM( + "ows?service=WFS&version=2.0.0&request=GetFeature&typeNames=cdf:Fifteen&resultType=index&count=2&startIndex=6"); + String resultSetId = docIndex.getDocumentElement().getAttribute("resultSetID"); + Document docResutl = getAsDOM( + "ows?service=WFS&version=2.0.2&request=PageResults&count=3&startIndex=5&resultSetID=" + + resultSetId); + XMLAssert.assertXpathEvaluatesTo("3", "count(//cdf:Fifteen)", docResutl); + assertStartIndexCount(docResutl, "previous", 2, 3); + assertStartIndexCount(docResutl, "next", 8, 3); + } + + void assertStartIndexCount(Document doc, String att, int startIndex, int count) { + assertTrue(doc.getDocumentElement().hasAttribute(att)); + String s = doc.getDocumentElement().getAttribute(att); + String[] kvp = s.split("\\?")[1].split("&"); + int actualStartIndex = -1; + int actualCount = -1; + + for (int i = 0; i < kvp.length; i++) { + String k = kvp[i].split("=")[0]; + String v = kvp[i].split("=")[1]; + if ("startIndex".equalsIgnoreCase(k)) { + actualStartIndex = Integer.parseInt(v); + } + if ("count".equalsIgnoreCase(k)) { + actualCount = Integer.parseInt(v); + } + } + + assertEquals(startIndex, actualStartIndex); + assertEquals(count, actualCount); + } + +} diff --git a/src/community/nsg-profile/src/test/java/org/geoserver/nsg/versioning/TimeVersioningTest.java b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/versioning/TimeVersioningTest.java new file mode 100644 index 00000000000..d69d6fe33e5 --- /dev/null +++ b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/versioning/TimeVersioningTest.java @@ -0,0 +1,100 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.versioning; + +import org.custommonkey.xmlunit.SimpleNamespaceContext; +import org.custommonkey.xmlunit.XMLUnit; +import org.custommonkey.xmlunit.XpathEngine; +import org.geoserver.data.test.MockData; +import org.geoserver.data.test.SystemTestData; +import org.geoserver.nsg.TestsUtils; +import org.geoserver.test.GeoServerSystemTestSupport; +import org.geotools.geometry.jts.ReferencedEnvelope; +import org.geotools.referencing.crs.DefaultGeographicCRS; +import org.junit.Before; +import org.junit.Test; +import org.w3c.dom.Document; + +import javax.xml.namespace.QName; +import java.util.HashMap; +import java.util.Map; + +public final class TimeVersioningTest extends GeoServerSystemTestSupport { + + private XpathEngine WFS20_XPATH_ENGINE; + + @Override + protected void onSetUp(SystemTestData testData) throws Exception { + super.setUpTestData(testData); + // create bounding box definitions + ReferencedEnvelope envelope = new ReferencedEnvelope(-5, -5, 5, 5, DefaultGeographicCRS.WGS84); + Map properties = new HashMap<>(); + properties.put(SystemTestData.LayerProperty.LATLON_ENVELOPE, envelope); + properties.put(SystemTestData.LayerProperty.ENVELOPE, envelope); + properties.put(SystemTestData.LayerProperty.SRS, 4326); + // create versioned layer + QName versionedLayerName = new QName(MockData.DEFAULT_URI, "versioned", MockData.DEFAULT_PREFIX); + testData.addVectorLayer(versionedLayerName, properties, "versioned.properties", getClass(), getCatalog()); + // instantiate xpath engine + buildXpathEngine( + "wfs", "http://www.opengis.net/wfs/2.0", + "gml", "http://www.opengis.net/gml/3.2"); + } + + @Before + public void beforeTest() { + // activate versioning for versioned layer + TestsUtils.updateFeatureTypeTimeVersioning(getCatalog(), "gs:versioned", true, "ID", "TIME"); + } + + @Test + public void testGetFeatureVersioned() throws Exception { + Document result = postAsDOM("wfs", TestsUtils.readResource("/requests/get_request_1.xml")); + System.out.println("aa"); + } + + @Test + public void testInsertVersionedFeature() throws Exception { + Document result = postAsDOM("wfs", TestsUtils.readResource("/requests/insert_request_1.xml")); + System.out.println("aa"); + } + + @Test + public void testUpdateVersionedFeature() throws Exception { + Document result = postAsDOM("wfs", TestsUtils.readResource("/requests/update_request_1.xml")); + System.out.println("aa"); + } + + /** + * Helper method that builds a xpath engine using some predefined + * namespaces and all the catalog namespaces. The provided namespaces + * will be added overriding any existing namespace. + */ + private XpathEngine buildXpathEngine(String... namespaces) { + // build xpath engine + XpathEngine xpathEngine = XMLUnit.newXpathEngine(); + Map finalNamespaces = new HashMap<>(); + // add common namespaces + finalNamespaces.put("ows", "http://www.opengis.net/ows"); + finalNamespaces.put("ogc", "http://www.opengis.net/ogc"); + finalNamespaces.put("xs", "http://www.w3.org/2001/XMLSchema"); + finalNamespaces.put("xsd", "http://www.w3.org/2001/XMLSchema"); + finalNamespaces.put("xlink", "http://www.w3.org/1999/xlink"); + finalNamespaces.put("xsi", "http://www.w3.org/2001/XMLSchema-instance"); + // add al catalog namespaces + getCatalog().getNamespaces().forEach(namespace -> + finalNamespaces.put(namespace.getPrefix(), namespace.getURI())); + // add provided namespaces + if (namespaces.length % 2 != 0) { + throw new RuntimeException("Invalid number of namespaces provided."); + } + for (int i = 0; i < namespaces.length; i += 2) { + finalNamespaces.put(namespaces[i], namespaces[i + 1]); + } + // add namespaces to the xpath engine + xpathEngine.setNamespaceContext(new SimpleNamespaceContext(finalNamespaces)); + return xpathEngine; + } +} diff --git a/src/community/nsg-profile/src/test/java/org/geoserver/nsg/versioning/web/WfsVersioningConfigTest.java b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/versioning/web/WfsVersioningConfigTest.java new file mode 100644 index 00000000000..b580d8e1f6e --- /dev/null +++ b/src/community/nsg-profile/src/test/java/org/geoserver/nsg/versioning/web/WfsVersioningConfigTest.java @@ -0,0 +1,103 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.nsg.versioning.web; + +import org.geoserver.web.GeoServerWicketTestSupport; +import org.geoserver.web.publish.PublishedConfigurationPage; +import org.geoserver.wfs.WFSInfo; +import org.junit.Test; + +import static org.hamcrest.CoreMatchers.is; +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertThat; + +public class WfsVersioningConfigTest extends GeoServerWicketTestSupport { + + @Test + public void testPageStart() throws Exception { + WFSInfo wfs = getGeoServerApplication().getGeoServer().getService(WFSInfo.class); + + login(); + tester.startPage(PublishedConfigurationPage.class); + } + + /*@Test + public void testChangesToValues() throws Exception { + String testValue1 = "100", testValue2 = "0"; + WFSInfo wfs = getGeoServerApplication().getGeoServer().getService(WFSInfo.class); + login(); + tester.startPage(WFSAdminPage.class); + FormTester ft = tester.newFormTester("form"); + ft.setValue("maxNumberOfFeaturesForPreview", (String)testValue1); + ft.submit("submit"); + wfs = getGeoServerApplication().getGeoServer().getService(WFSInfo.class); + assertEquals("testValue1 = 100", 100, (int)wfs.getMaxNumberOfFeaturesForPreview()); + tester.startPage(WFSAdminPage.class); + ft = tester.newFormTester("form"); + ft.setValue("maxNumberOfFeaturesForPreview", (String)testValue2); + ft.submit("submit"); + wfs = getGeoServerApplication().getGeoServer().getService(WFSInfo.class); + assertEquals("testValue2 = 0", 0, (int)wfs.getMaxNumberOfFeaturesForPreview()); + } + + @Test + public void testGML32ForceMimeType() throws Exception { + // make sure GML MIME type overriding is disabled + WFSInfo info = getGeoServer().getService(WFSInfo.class); + GMLInfo gmlInfo = info.getGML().get(WFSInfo.Version.V_20); + gmlInfo.setMimeTypeToForce(null); + getGeoServer().save(info); + // login with administrator privileges + login(); + // start WFS service administration page + tester.startPage(new WFSAdminPage()); + // check that GML MIME type overriding is disabled + tester.assertComponent("form:gml32:forceGmlMimeType", CheckBox.class); + CheckBox checkbox = (CheckBox) tester.getComponentFromLastRenderedPage("form:gml32:forceGmlMimeType"); + assertThat(checkbox.getModelObject(), is(false)); + // MIME type drop down choice should be invisible + tester.assertInvisible("form:gml32:mimeTypeToForce"); + // activate MIME type overriding by clicking in the checkbox + FormTester formTester = tester.newFormTester("form"); + formTester.setValue("gml32:forceGmlMimeType", true); + tester.executeAjaxEvent("form:gml32:forceGmlMimeType", "click"); + formTester = tester.newFormTester("form"); + formTester.submit("submit"); + // GML MIME typing overriding should be activated now + tester.startPage(new WFSAdminPage()); + assertThat(checkbox.getModelObject(), is(true)); + tester.assertVisible("form:gml32:mimeTypeToForce"); + // WFS global service configuration should have been updated too + info = getGeoServer().getService(WFSInfo.class); + gmlInfo = info.getGML().get(WFSInfo.Version.V_20); + assertThat(gmlInfo.getMimeTypeToForce().isPresent(), is(true)); + // select text / xml as MIME type to force + formTester = tester.newFormTester("form"); + formTester.select("gml32:mimeTypeToForce", 2); + tester.executeAjaxEvent("form:gml32:mimeTypeToForce", "change"); + formTester = tester.newFormTester("form"); + formTester.submit("submit"); + // WFS global service configuration should be forcing text / xml + info = getGeoServer().getService(WFSInfo.class); + gmlInfo = info.getGML().get(WFSInfo.Version.V_20); + assertThat(gmlInfo.getMimeTypeToForce().isPresent(), is(true)); + assertThat(gmlInfo.getMimeTypeToForce().get(), is("text/xml")); + // deactivate GML MIME type overriding by clicking in the checkbox + tester.startPage(new WFSAdminPage()); + formTester = tester.newFormTester("form"); + formTester.setValue("gml32:forceGmlMimeType", false); + tester.executeAjaxEvent("form:gml32:forceGmlMimeType", "click"); + formTester = tester.newFormTester("form"); + formTester.submit("submit"); + // GML MIME type overriding should be deactivated now + tester.startPage(new WFSAdminPage()); + assertThat(checkbox.getModelObject(), is(true)); + tester.assertInvisible("form:gml32:mimeTypeToForce"); + // WFS global service configuration should have been updated too + info = getGeoServer().getService(WFSInfo.class); + gmlInfo = info.getGML().get(WFSInfo.Version.V_20); + assertThat(gmlInfo.getMimeTypeToForce().isPresent(), is(false)); + }*/ +} diff --git a/src/community/nsg-profile/src/test/resources/org/geoserver/nsg/versioned.properties b/src/community/nsg-profile/src/test/resources/org/geoserver/nsg/versioned.properties new file mode 100644 index 00000000000..ad32708279a --- /dev/null +++ b/src/community/nsg-profile/src/test/resources/org/geoserver/nsg/versioned.properties @@ -0,0 +1,6 @@ +_=ID:String,NAME:String,TIME:java.sql.Timestamp,GEOMETRY:Geometry:srid=4326 +v.1=1|feature1|2017-06-25 14:30:00.0|POINT(-1 1) +v.2=1|feature1|2017-07-25 14:30:00.0|POINT(-1 -1) +v.3=1|feature1|2017-06-25 14:35:00.0|POINT(1 -1) +v.4=2|feature2|2017-04-10 13:30:00.0|POINT(-2 2) +v.5=2|feature2|2017-08-10 17:10:00.0|POINT(2 -2) \ No newline at end of file diff --git a/src/community/nsg-profile/src/test/resources/requests/get_request_1.xml b/src/community/nsg-profile/src/test/resources/requests/get_request_1.xml new file mode 100644 index 00000000000..bd69e4fdbeb --- /dev/null +++ b/src/community/nsg-profile/src/test/resources/requests/get_request_1.xml @@ -0,0 +1,13 @@ + + + + + + + \ No newline at end of file diff --git a/src/community/nsg-profile/src/test/resources/requests/insert_request_1.xml b/src/community/nsg-profile/src/test/resources/requests/insert_request_1.xml new file mode 100644 index 00000000000..c50429c8a86 --- /dev/null +++ b/src/community/nsg-profile/src/test/resources/requests/insert_request_1.xml @@ -0,0 +1,22 @@ + + + + + 1 + feature1 + 2018-01-25T13:30:00Z + + + 1.5 -1.5 + + + + + \ No newline at end of file diff --git a/src/community/nsg-profile/src/test/resources/requests/update_request_1.xml b/src/community/nsg-profile/src/test/resources/requests/update_request_1.xml new file mode 100644 index 00000000000..34ed22793ac --- /dev/null +++ b/src/community/nsg-profile/src/test/resources/requests/update_request_1.xml @@ -0,0 +1,18 @@ + + + + + NAME + feature2_updated + + + + + + \ No newline at end of file diff --git a/src/community/pom.xml b/src/community/pom.xml index 8637f77d6ea..327013956df 100644 --- a/src/community/pom.xml +++ b/src/community/pom.xml @@ -62,7 +62,7 @@ release/ext-gwc-distributed.xml release/ext-geofence-server.xml - + release/ext-gwc-s3.xml release/ext-gdal-wcs.xml release/ext-gdal-wps.xml release/ext-wps-jdbc.xml @@ -225,7 +225,7 @@ gwc-distributed jdbcstore gdal - + gwc-s3 ncwms web-resource @@ -242,7 +242,7 @@ ows-simulate jdbc-metrics oseo - s3-geotiff + nsg-profile @@ -533,9 +533,9 @@ - s3-geotiff + nsg-profile - s3-geotiff + nsg-profile diff --git a/src/community/release/ext-nsg-profile.xml b/src/community/release/ext-nsg-profile.xml new file mode 100644 index 00000000000..11f21b27b56 --- /dev/null +++ b/src/community/release/ext-nsg-profile.xml @@ -0,0 +1,16 @@ + + nsg-profile + + zip + + false + + + release/target/dependency + + + gs-nsg-profile*.jar + + + + \ No newline at end of file diff --git a/src/main/src/main/java/org/geoserver/catalog/FeatureTypeInfo.java b/src/main/src/main/java/org/geoserver/catalog/FeatureTypeInfo.java index f780f80b9f5..3a2b0724b65 100644 --- a/src/main/src/main/java/org/geoserver/catalog/FeatureTypeInfo.java +++ b/src/main/src/main/java/org/geoserver/catalog/FeatureTypeInfo.java @@ -193,4 +193,11 @@ public interface FeatureTypeInfo extends ResourceInfo { void setCircularArcPresent(boolean arcsPresent); + default T getParameter(String parameterName, Class expectType, T fallback) { + return null; + } + + default void putParameter(String parameterName, Object parameterValue) { + // nothing to do + } } diff --git a/src/main/src/main/java/org/geoserver/catalog/impl/FeatureTypeInfoImpl.java b/src/main/src/main/java/org/geoserver/catalog/impl/FeatureTypeInfoImpl.java index 8fc00f86217..b1a5cd481c2 100644 --- a/src/main/src/main/java/org/geoserver/catalog/impl/FeatureTypeInfoImpl.java +++ b/src/main/src/main/java/org/geoserver/catalog/impl/FeatureTypeInfoImpl.java @@ -5,10 +5,6 @@ */ package org.geoserver.catalog.impl; -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; - import org.geoserver.catalog.AttributeTypeInfo; import org.geoserver.catalog.Catalog; import org.geoserver.catalog.CatalogVisitor; @@ -20,11 +16,18 @@ import org.geotools.filter.text.cql2.CQLException; import org.geotools.filter.text.ecql.ECQL; import org.geotools.measure.Measure; +import org.geotools.util.Converters; import org.opengis.feature.Feature; import org.opengis.feature.type.FeatureType; import org.opengis.filter.Filter; import org.opengis.util.ProgressListener; +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + @SuppressWarnings("serial") public class FeatureTypeInfoImpl extends ResourceInfoImpl implements FeatureTypeInfo { @@ -42,7 +45,9 @@ public class FeatureTypeInfoImpl extends ResourceInfoImpl implements boolean overridingServiceSRS; boolean skipNumberMatched = false; boolean circularArcPresent; - + + protected Map parameters = new HashMap<>(); + public boolean isCircularArcPresent() { return circularArcPresent; } @@ -236,5 +241,21 @@ public void setCqlFilter(String cqlFilter) { this.cqlFilter = cqlFilter; this.filter = null; } - + + @Override + public T getParameter(String parameterName, Class expectType, T fallback) { + if (parameters == null || parameters.isEmpty()) { + return fallback; + } + Object value = parameters.get(parameterName); + return value == null ? fallback : Converters.convert(value, expectType); + } + + @Override + public void putParameter(String parameterName, Object parameterValue) { + if (parameters == null) { + parameters = new HashMap<>(); + } + parameters.put(parameterName, parameterValue); + } } diff --git a/src/main/src/main/java/org/geoserver/security/decorators/DecoratingFeatureTypeInfo.java b/src/main/src/main/java/org/geoserver/security/decorators/DecoratingFeatureTypeInfo.java index 4f7da4f075c..40b50d647cc 100644 --- a/src/main/src/main/java/org/geoserver/security/decorators/DecoratingFeatureTypeInfo.java +++ b/src/main/src/main/java/org/geoserver/security/decorators/DecoratingFeatureTypeInfo.java @@ -329,4 +329,13 @@ public void setCqlFilter(String cqlFilter) { delegate.setCqlFilter(cqlFilter); } + @Override + public T getParameter(String parameterName, Class expectType, T fallback) { + return delegate.getParameter(parameterName, expectType, fallback); + } + + @Override + public void putParameter(String parameterName, Object parameterValue) { + delegate.putParameter(parameterName, parameterValue); + } } diff --git a/src/main/src/main/java/org/geoserver/security/decorators/SecuredFeatureTypeInfo.java b/src/main/src/main/java/org/geoserver/security/decorators/SecuredFeatureTypeInfo.java index bcc7c5ed884..9198b365cc3 100644 --- a/src/main/src/main/java/org/geoserver/security/decorators/SecuredFeatureTypeInfo.java +++ b/src/main/src/main/java/org/geoserver/security/decorators/SecuredFeatureTypeInfo.java @@ -111,5 +111,6 @@ public FeatureSource getFeatureSource(ProgressListener listener, Hints hints) public DataStoreInfo getStore() { return (DataStoreInfo) SecuredObjects.secure(delegate.getStore(), policy); } - + + } diff --git a/src/web/app/pom.xml b/src/web/app/pom.xml index aee4265be9e..b92ca71d249 100644 --- a/src/web/app/pom.xml +++ b/src/web/app/pom.xml @@ -141,10 +141,6 @@ org.geotools.jdbc gt-jdbc-postgis - - org.geotools - gt-geopkg - org.geotools gt-wfs-ng @@ -1527,11 +1523,11 @@ - s3-geotiff + nsg-profile org.geoserver.community - gs-s3-geotiff + gs-nsg-profile ${project.version} diff --git a/src/web/security/core/src/main/java/org/geoserver/security/web/user/UserPanel.java b/src/web/security/core/src/main/java/org/geoserver/security/web/user/UserPanel.java index c10f9520432..fd5cb632beb 100644 --- a/src/web/security/core/src/main/java/org/geoserver/security/web/user/UserPanel.java +++ b/src/web/security/core/src/main/java/org/geoserver/security/web/user/UserPanel.java @@ -105,7 +105,7 @@ public void onClick() { }); //("addNew", NewUserPage.class)); - //add.setParameter(AbstractSecurityPage.ServiceNameKey, serviceName); + //add.putParameter(AbstractSecurityPage.ServiceNameKey, serviceName); add.setVisible(canCreateStore); // the removal button diff --git a/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java b/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java index 3e66b5a92ca..151444e61d0 100644 --- a/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java +++ b/src/wfs/src/main/java/org/geoserver/wfs/GetFeature.java @@ -34,6 +34,7 @@ import org.geoserver.ows.Request; import org.geoserver.ows.URLMangler.URLType; import org.geoserver.ows.util.KvpMap; +import org.geoserver.platform.GeoServerExtensions; import org.geoserver.wfs.request.FeatureCollectionResponse; import org.geoserver.wfs.request.GetFeatureRequest; import org.geoserver.wfs.request.Lock; @@ -506,6 +507,21 @@ public FeatureCollectionResponse run(GetFeatureRequest request) queryMaxFeatures, source, request, allPropNames.get(0), viewParam, joins, primaryTypeName, primaryAlias); + // extension point + GetFeatureContext context = new GetFeatureContextBuilder() + .withFeatureSource(source) + .withFeatureTypeInfo(meta) + .withQuery(gtQuery) + .withRequest(request).build(); + List callbacks = GeoServerExtensions.extensions(GetFeatureCallback.class); + for (GetFeatureCallback callback : callbacks) { + context = callback.beforeQuerying(context); + } + source = context.getFeatureSource(); + meta = context.getFeatureTypeInfo(); + gtQuery = context.getQuery(); + request = context.getRequest(); + LOGGER.fine("Query is " + query + "\n To gt2: " + gtQuery); FeatureCollection features = getFeatures(request, source, gtQuery); diff --git a/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureCallback.java b/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureCallback.java new file mode 100644 index 00000000000..c0b5c8ed58f --- /dev/null +++ b/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureCallback.java @@ -0,0 +1,20 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.wfs; + +import org.geoserver.platform.ExtensionPriority; + +public interface GetFeatureCallback extends ExtensionPriority { + + default GetFeatureContext beforeQuerying(GetFeatureContext context) { + // by default nothing is done + return context; + } + + @Override + default int getPriority() { + return ExtensionPriority.LOWEST; + } +} diff --git a/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureContext.java b/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureContext.java new file mode 100644 index 00000000000..3f06b7c4325 --- /dev/null +++ b/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureContext.java @@ -0,0 +1,44 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.wfs; + +import org.geoserver.catalog.FeatureTypeInfo; +import org.geoserver.wfs.request.GetFeatureRequest; +import org.geotools.data.FeatureSource; +import org.geotools.data.Query; +import org.opengis.feature.Feature; +import org.opengis.feature.type.FeatureType; + +public final class GetFeatureContext { + + private final GetFeatureRequest request; + private final FeatureSource featureSource; + private final Query query; + private final FeatureTypeInfo featureTypeInfo; + + GetFeatureContext(GetFeatureRequest request, FeatureSource featureSource, + Query query, FeatureTypeInfo featureTypeInfo) { + this.request = request; + this.featureSource = featureSource; + this.query = query; + this.featureTypeInfo = featureTypeInfo; + } + + public GetFeatureRequest getRequest() { + return request; + } + + public FeatureSource getFeatureSource() { + return featureSource; + } + + public Query getQuery() { + return query; + } + + public FeatureTypeInfo getFeatureTypeInfo() { + return featureTypeInfo; + } +} \ No newline at end of file diff --git a/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureContextBuilder.java b/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureContextBuilder.java new file mode 100644 index 00000000000..0c19934cf6c --- /dev/null +++ b/src/wfs/src/main/java/org/geoserver/wfs/GetFeatureContextBuilder.java @@ -0,0 +1,54 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.wfs; + +import org.geoserver.catalog.FeatureTypeInfo; +import org.geoserver.wfs.request.GetFeatureRequest; +import org.geotools.data.FeatureSource; +import org.geotools.data.Query; +import org.opengis.feature.Feature; +import org.opengis.feature.type.FeatureType; + +public final class GetFeatureContextBuilder { + + private GetFeatureRequest request; + private FeatureSource featureSource; + private Query query; + private FeatureTypeInfo featureTypeInfo; + + public GetFeatureContextBuilder() { + } + + public GetFeatureContextBuilder withRequest(GetFeatureRequest request) { + this.request = request; + return this; + } + + public GetFeatureContextBuilder withFeatureSource(FeatureSource featureSource) { + this.featureSource = featureSource; + return this; + } + + public GetFeatureContextBuilder withQuery(Query query) { + this.query = query; + return this; + } + + public GetFeatureContextBuilder withFeatureTypeInfo(FeatureTypeInfo featureTypeInfo) { + this.featureTypeInfo = featureTypeInfo; + return this; + } + + public GetFeatureContextBuilder withContext(GetFeatureContext context) { + return withRequest(context.getRequest()) + .withFeatureSource(context.getFeatureSource()) + .withQuery(context.getQuery()) + .withFeatureTypeInfo(context.getFeatureTypeInfo()); + } + + public GetFeatureContext build() { + return new GetFeatureContext(request, featureSource, query, featureTypeInfo); + } +} diff --git a/src/wfs/src/main/java/org/geoserver/wfs/InsertElementHandler.java b/src/wfs/src/main/java/org/geoserver/wfs/InsertElementHandler.java index 49ae7fa67be..e277dcdb16d 100644 --- a/src/wfs/src/main/java/org/geoserver/wfs/InsertElementHandler.java +++ b/src/wfs/src/main/java/org/geoserver/wfs/InsertElementHandler.java @@ -5,21 +5,11 @@ */ package org.geoserver.wfs; -import java.math.BigInteger; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.HashSet; -import java.util.Iterator; -import java.util.LinkedList; -import java.util.List; -import java.util.Map; -import java.util.logging.Logger; - -import javax.xml.namespace.QName; - +import com.vividsolutions.jts.geom.Geometry; import org.geoserver.catalog.FeatureTypeInfo; import org.geoserver.config.GeoServer; import org.geoserver.feature.ReprojectingFeatureCollection; +import org.geoserver.platform.GeoServerExtensions; import org.geoserver.wfs.request.Insert; import org.geoserver.wfs.request.TransactionElement; import org.geoserver.wfs.request.TransactionRequest; @@ -40,7 +30,16 @@ import org.opengis.filter.identity.FeatureId; import org.opengis.referencing.crs.CoordinateReferenceSystem; -import com.vividsolutions.jts.geom.Geometry; +import javax.xml.namespace.QName; +import java.math.BigInteger; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Iterator; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.logging.Logger; /** @@ -71,7 +70,21 @@ public void checkValidity(TransactionElement element, Map callbacks = GeoServerExtensions.extensions(TransactionCallback.class); + try { for (Iterator it = elementHandlers.entrySet().iterator(); it.hasNext();) { Map.Entry entry = (Map.Entry) it.next(); TransactionElement element = (TransactionElement) entry.getKey(); TransactionElementHandler handler = (TransactionElementHandler) entry.getValue(); + TransactionContext context = new TransactionContextBuilder() + .withElement(element) + .withRequest(request) + .withFeatureStores(stores) + .withResponse(result) + .withHandler(handler).build(); + context = TransactionCallback.executeCallbacks( + context, callbacks, TransactionCallback::beforeHandlerExecution); + + element = context.getElement(); + request = context.getRequest(); + stores = context.getFeatureStores(); + result = context.getResponse(); + handler = context.getHandler(); + handler.execute(element, request, stores, result, multiplexer); } } catch (WFSTransactionException e) { diff --git a/src/wfs/src/main/java/org/geoserver/wfs/TransactionCallback.java b/src/wfs/src/main/java/org/geoserver/wfs/TransactionCallback.java new file mode 100644 index 00000000000..6193ab9dca4 --- /dev/null +++ b/src/wfs/src/main/java/org/geoserver/wfs/TransactionCallback.java @@ -0,0 +1,49 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.wfs; + +import java.util.List; + +public interface TransactionCallback { + + default TransactionContext beforeHandlerExecution(TransactionContext context) { + // by default nothing is done + return context; + } + + default TransactionContext beforeInsertFeatures(TransactionContext context) { + // by default nothing is done + return context; + } + + default TransactionContext beforeUpdateFeatures(TransactionContext context) { + // by default nothing is done + return context; + } + + default TransactionContext beforeDeleteFeatures(TransactionContext context) { + // by default nothing is done + return context; + } + + default TransactionContext beforeReplaceFeatures(TransactionContext context) { + // by default nothing is done + return context; + } + + @FunctionalInterface + interface Executor { + TransactionContext apply(TransactionCallback callback, TransactionContext context); + } + + static TransactionContext executeCallbacks(TransactionContext context, + List callbacks, + Executor executor) { + for (TransactionCallback callback : callbacks) { + context = executor.apply(callback, context); + } + return context; + } +} diff --git a/src/wfs/src/main/java/org/geoserver/wfs/TransactionContext.java b/src/wfs/src/main/java/org/geoserver/wfs/TransactionContext.java new file mode 100644 index 00000000000..469fcc1a558 --- /dev/null +++ b/src/wfs/src/main/java/org/geoserver/wfs/TransactionContext.java @@ -0,0 +1,49 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.wfs; + +import org.geoserver.wfs.request.TransactionElement; +import org.geoserver.wfs.request.TransactionRequest; +import org.geoserver.wfs.request.TransactionResponse; + +import java.util.Map; + +public final class TransactionContext { + + private final TransactionElement element; + private final TransactionRequest request; + private final Map featureStores; + private final TransactionResponse response; + private final TransactionElementHandler handler; + + TransactionContext(TransactionElement element, TransactionRequest request, Map featureStores, + TransactionResponse response, TransactionElementHandler handler) { + this.element = element; + this.request = request; + this.featureStores = featureStores; + this.response = response; + this.handler = handler; + } + + public TransactionElement getElement() { + return element; + } + + public TransactionRequest getRequest() { + return request; + } + + public Map getFeatureStores() { + return featureStores; + } + + public TransactionResponse getResponse() { + return response; + } + + public TransactionElementHandler getHandler() { + return handler; + } +} \ No newline at end of file diff --git a/src/wfs/src/main/java/org/geoserver/wfs/TransactionContextBuilder.java b/src/wfs/src/main/java/org/geoserver/wfs/TransactionContextBuilder.java new file mode 100644 index 00000000000..a1a2f018c96 --- /dev/null +++ b/src/wfs/src/main/java/org/geoserver/wfs/TransactionContextBuilder.java @@ -0,0 +1,61 @@ +/* (c) 2017 Open Source Geospatial Foundation - all rights reserved + * This code is licensed under the GPL 2.0 license, available at the root + * application directory. + */ +package org.geoserver.wfs; + +import org.geoserver.wfs.request.TransactionElement; +import org.geoserver.wfs.request.TransactionRequest; +import org.geoserver.wfs.request.TransactionResponse; + +import java.util.Map; + +public final class TransactionContextBuilder { + + private TransactionElement element; + private TransactionRequest request; + private Map featureStores; + private TransactionResponse response; + private TransactionElementHandler handler; + + public TransactionContextBuilder() { + } + + public TransactionContextBuilder withElement(TransactionElement element) { + this.element = element; + return this; + } + + public TransactionContextBuilder withRequest(TransactionRequest request) { + this.request = request; + return this; + } + + public TransactionContextBuilder withFeatureStores(Map featureStores) { + this.featureStores = featureStores; + return this; + } + + public TransactionContextBuilder withResponse(TransactionResponse response) { + this.response = response; + return this; + } + + public TransactionContextBuilder withHandler(TransactionElementHandler handler) { + this.handler = handler; + return this; + } + + public TransactionContextBuilder withContext(TransactionContext context) { + this.element = context.getElement(); + this.featureStores = context.getFeatureStores(); + this.handler = context.getHandler(); + this.request = context.getRequest(); + this.response = context.getResponse(); + return this; + } + + public TransactionContext build() { + return new TransactionContext(element, request, featureStores, response, handler); + } +}