Bug in expanding cluster?

[carlos-verdes]

Hi, in GDBScan, the last line of the expand method says "if new point is a cluster"... but in the "if clause" the point is evaluated with the current neighbourhood... not with the new neighbours.

    neighbours.foldLeft(neighbours) {
      case (neighbourhood, neighbour @ Point(row)) =>
        // if not visited yet, create a new neighbourhood
        val newNeighbours = if (!(visited contains neighbour)) {
          visited add neighbour
          getNeighbours(neighbour, points.filterNot(_.row == neighbour.row))
        } else {
          Seq.empty
        }
        // Add to cluster if neighbour point isn't assigned to a  cluster yet
        if (!(clustered contains neighbour)) {
          cluster add neighbour
          clustered add neighbour
        }
        // if the neighbour point is a cluster, join the neighbourhood
        if (isCorePoint(neighbour, neighbourhood)) neighbourhood ++ newNeighbours else neighbourhood
    }

I think the last line should be:

 // if the neighbour point is a cluster, join the neighbourhood
   if (isCorePoint(neighbour, newNeighbours)) neighbourhood ++ newNeighbours else neighbourhood

If you agree I can do a PR.

[carlos-verdes]

@jasonbaldridge @gabeos please could you take a look into this?

Any doubt don't hesitate to ask please.

[muuki88]

It's been a very long time since I wrote this. If you could provide a failing test for this to verify your assumption then accepting a PR would be a lot easier.

[btafel]

hey @carlos-verdes ! I know this is quite old, but... did you manage to verify that issue?

I'm running into another issue now and it might be related.

At some point, the append neighbourhood ++ newNeighbours throws

Java.lang.IllegalArgumentException
            at scala.collection.immutable.VectorPointer$class.gotoFreshPosWritable0(Vector.scala:1195)
            at scala.collection.immutable.Vector.gotoFreshPosWritable0(Vector.scala:62)
            at scala.collection.immutable.VectorPointer$class.gotoFreshPosWritable1(Vector.scala:1204)
......

Scala: 2.11

For what I could tell at looking at the Vector code it is running out of 'levels'

Any clues?? @muuki88

Thanks!

[muuki88]

Thanks for reporting this @btafel 😄 . Unfortunately I can't help in any regards as I haven't used this code for 7 years now. Also I mostly work on Scala 2.13 projects.

[btafel]

lol it makes sense! I know it is old - very old - but I have to say that it works :) at least in most cases. I was inspired by this article https://www.oreilly.com/content/clustering-geolocated-data-using-spark-and-dbscan/ and got it running smoothly.

I'm also trying another approach using UserDefinedAggregateFunction and Smile

[muuki88]

Wow! I'm really happy that this is still in use and useful to others 😍
Thanks for the hints to other libraries 👍