VirtualizedFX table with big dataset

[stefanofornari]

Hello,

I need to attach a TableView to a potentially big dataset (e.g. a database table). I have started with MaterialFX TableView but I can't go further because it is not designed for such use case. I tried to build a new one on my own, but the current version of MFX uses an old version of VirtualizedFX with a SimpleVirtualFlow object that is not any more in VirtualizedFX. I need something I can comfortably look into the code and debug.

I then had a look at VirtualizedFX and as far as I understand, it provides a much better implementation of a TableView with virtual flow. (great stuff, BTW!). Therefore I am now playing with VirtualizedFX.

The question: what's the best way to use a datasource that can provide ondemand data? I am thinking about something like that when I scroll down to the last row, the data source is fetched for more data. Is this use case supported already?

Many thanks in advance.

[palexdev]

Hello @stefanofornari

1. Thanks, I'm happy with the recent work on VirtualizedFX, my brain is still recovering though haha
2. The old implementation is still there, in the 'unused' package for comparison and retro-compatibility
3. Happy to know you're experimenting with it. As usual, examples are in the demo/tests. The good thing about the 'new' VirtualizedFX is that now it also offers a component to wrap the flows and adds scroll bars, creating in fact a full and working virtualized component. The bad thing maybe is that they are not styled, I didn't do it because I will do it in MaterialFX when using it, and yet there's a good thing too, since it doesn't use the nefarious 'getUserAgentStylesheet()' method you should have no issues in styling them
4. No, on demand load is not implemented. I don't know if I'm gonna do it someday. In the near future no for sure, As I said before I'm still tired from the recent work done on VirtualizedFX. Also, the paginated version in a way accomplishes the same thing, maybe you can arrange something with it

[stefanofornari]

Thanks, I'm happy with the recent work on VirtualizedFX, my brain is still recovering though haha

Take some rest matey!!! :)

The old implementation is still there, in the 'unused' package for comparison and retro-compatibility

Yeah, I saw that, but MaterialFX is not updated (import io.github.palexdev.virtualizedfx.flow.simple.SimpleVirtualFlow;)

Happy to know you're experimenting with it. As usual, examples are in the demo/tests. The good thing about the 'new' VirtualizedFX is that now it also offers a component to wrap the flows and adds scroll bars, creating in fact a full and working virtualized component. The bad thing maybe is that they are not styled, I didn't do it because I will do it in MaterialFX when using it, and yet there's a good thing too, since it doesn't use the nefarious 'getUserAgentStylesheet()' method you should have no issues in styling them

Not a big deal, it is good enough for now and have anyway to re-style it.

No, on demand load is not implemented. I don't know if I'm gonna do it someday. In the near future no for sure, As I said before I'm still tired from the recent work done on VirtualizedFX. Also, the paginated version in a way accomplishes the same thing, maybe you can arrange something with it

Mhhh... using the paginated version is actually a good idea. To start I would just need to switch page when I hit the top/bottom. I'll check how to get notified of those events.
The other design difference is that sorting or applying filters should trigger the underlying data source more than then table's rows.
I'll keep you posted.

[stefanofornari]

Good morning.
This is what I am thinking about, what do you think?

Table

The TableView has controls to set the sorting column(s) and filters. Plus indeed the items in the table rows. These are exposed through the properties items, filters, sorting.

TableSource

A TableSource object provides the data to show in the table by binding the table's items property with source's page property. It supports filtering and sorting by binding filters and sorting. page acts as source for the table object while filters and sorting are sinks.
TableSource provides also the way to move the page backwards and forwards with the callbacks backward() and forward()

Controller

The Controller glues together the table view and the table source:

• Creates/holds the source
• Binds table source's page -> table's item, table source's sorting <- table's sorting and table source's filters <- table's filters
• Listens for table view's scrollbar events to trigger source's backwards() when the top is hit and forwards() when the bottom is hit

Controller can creates the table source or the table source can be provided with a setter. In the former case it can be used directly in the FXML, in the latter, FXMLLoader.load() needs be used with the controller factory. A builder can be provided to make it easier.

SimpleTableSource and DBTableSource

We can think of two basic implementations of TableSource. A simple one which basically reproduces the current behaviour and a version that uses a database table to get the data.

SimpleTableSource

• page <- all items in a collection
• sorting -> sort page
• filters ->
  • filter the collection
  • update page
  • sort page
• backwards -> do nothing
• forwards -> do nothing

DBTableSource

• page <- result set
• sorting ->
  • store the sorting columns
  • query the database
  • update page
• filters ->
  • store the predicates
  • query the database
  • update page
• backwards()
  • query the database using sorting and filters
  • update page
• forwards()
  • query the database using sorting and filters
  • update page

[palexdev]

Seems like a good plan
Let me know how it turns out and if you need help

[stefanofornari]

As a first step I added sort order functionality so that when I change the order data are fetched taking that into account.
I now want to implement the infinite scrolling. Do you have any suggestion on the best way to catch scrollbar events?

At the moment I am planning to do something like:

table.onScrollProperty().set((event) -> {
    if (table.getLastRowsRange().getMax().longValue() == table.getItems().size()-1) {
        // DO THE STUFF
    }
});

[palexdev]

With the new VirtualizedFX?

[palexdev]

@stefanofornari
There are a lot of ways to do this, more than I thought

1. The one you described above
2. You can attach a listener on the position property of the table
3. You can attach listeners on the position properties of the VirtualScrollPane (if using it to wrap)
4. I didn't test this, nor tried to implement it, but I'm pretty sure you can also use the experimental behavior API, you can change the behavior of both scroll bars and intercept scrolling there. This is the most elaborate, but also more precise, as this is the where the scrolling starts

[stefanofornari]

Thanks! the 1st works pretty well for me 👍

[stefanofornari]

yep Ste

…

On Wed, Jan 11, 2023 at 9:08 PM Alessadro Parisi ***@***.***> wrote: With the new VirtualizedFX? — Reply to this email directly, view it on GitHub <#282 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAKRAXZJ75DVHVYWJX5ON3WR4HK7ANCNFSM6AAAAAATT76XX4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[stefanofornari]

It looks like I ran into the first bug :) palexdev/VirtualizedFX#4 (comment)

[stefanofornari]

OK, I have done a first step ahead. The table works pretty well. The design came out simpler than anticipated, as it basically binds the table items to the data set. Controlling the scroll as in #282 (comment) triggers fetching more data. I also easily added column sorting.
It would be great to have arrows key navigation our of the box.

For who interested here the relevant code:
https://github.com/stefanofornari/EasyWallet/blob/main/src/main/java/ste/w3/easywallet/ui/LedgerController.java
https://github.com/stefanofornari/EasyWallet/blob/main/src/main/java/ste/w3/easywallet/ui/LedgerDialog.java

[palexdev]

Well done @stefanofornari , very well done