Point detection workflow

[bw4sz]

As part of DeepForest 2.0 we need a model for point prediction.

A couple examples we can work from

https://github.com/xingyizhou/CenterNet.
https://github.com/FateScript/CenterNet-better
https://arxiv.org/abs/1904.07850

We have the data from

https://doi.org/10.1016/j.jag.2024.103848

Roadmap

• Add small point dataset example to get_data. We should use the existing box dataset (OSBS_029) and just get the centroids
• Add a point dataloader to datasets.py similar to

  DeepForest/deepforest/dataset.py

  Line 204 in 7e0aa43

  class BoundingBoxDataset(Dataset):

• Add a point model to models.
• Example training script in examples/
• Confirm visualization
• Confirm can be extended to multi-class

@Mu-Magdy might be interested, since its isolated from the code base, doesn't have a time frame and fits objectives.

Bonus

• Train a tree point detector on the MillionTrees point datasets and/or the center points of the boxes/polygons datasets.

[bw4sz]

Also check https://github.com/airctic/icevision/tree/master/icevision/models/torchvision/keypoint_rcnn

[bw4sz]

and https://link.springer.com/chapter/10.1007/978-3-031-92387-6_12

[bw4sz]

The question is whether we use segment anything to just convert these points into box structures to train.

[musaqlain]

hi @bw4sz

I've been researching the point detection workflow outlined in this issue over the past few days, and I'd like to contribute to implementing this feature. After thoroughly analyzing all the options discussed here and conducting additional research, I believe a custom CenterNet implementation is the best path forward because: (1) it is purpose-built for point detection unlike adapting box-based models, (2) it is fast (140+ FPS) with no NMS post-processing needed, and (3) it can integrate into DeepForest's existing BaseModel wrapper pattern.
I spent significant time studying the DeepForest codebase to understand the existing patterns , particularly how RetinaNet implements the BaseModel interface and how BoxDataset handles annotations. I also reviewed PR #590 (multi-geometry support) which provides the geopandas foundation for handling point annotations.
Here's my planned roadmap, and I need your suggestions/guidelines along the way:

1. Create PointDataset class that generates Gaussian heatmap targets from point annotations
2. Implement CenterNetHub model with ResNet backbone + heatmap/offset heads
3. Add focal loss for heatmap + L1 loss for offset regression
4. Implement peak extraction via 3x3 max pooling (built-in NMS)
5. Add point visualization and evaluation metrics (distance-based instead of IoU)
6. Create training script in examples/train_centernet.py
7. Add comprehensive unit tests

I read the original 'Objects as Points' paper, it includes size regression (for height/width), but I think the point coordinates might be sufficient, right?

[ethanwhite]

@jveitchmichaelis has been working on this so pinging him to engage with @musaqlain