geonetwork datadir checker useless ressources

[jeanmi151]

following #11 ...

The aim of this is to add a checker to spot files that are no longer needed because geonetwork forget to delete them

it checks the database records (metadata table) and search in the /mnt/geonetwork_datadir/data/metadata_data/
(value here https://github.com/georchestra/datadir/blob/docker-master/geonetwork/geonetwork.properties#L2)

[jeanmi151]

Look like that now

[landryb]

I've tested it again and it looks better, there a still a bunch of cleanups to do..

now i know why didn't like the size to be 'humanized' in the display (versus raw bytes).. now you cant easily sort to figure out which leftover dir takes the most space. filtering on 'MB' and sorting doesnt help either because 44 is between 1 and 9, as those are strings..

[landryb]

ah, and can you also use black on all files ? i try to keep the code black-formatted..

[landryb]

now that we use a bootstrap table to display the values, i think we can use a formatter to humanize the numbers on the fly, like the ones in

gaia/geordash/static/js/script.js

Line 561 in c70ae82

function urlFormatter(value, row) {

- and some js like https://gist.github.com/lanqy/5193417

this way, no need for jinja filesizeformat filter in the backend, raw numbers are stored in the task results (which make it easier to reuse if you build something that parses the json from the task result), and eventually that might allow to sort based on the raw value ?

another note.. i dont think we need to store the path/url twice, nor the problem type in the results:

path	"101932200-101932299/101932275"
problem	"5.0 MB"
type	"UnusedFileRes"
url	"101932200-101932299/101932275"

unless you plan to tackle the other cases i had listed in #11 (comment) - for now the type is always the same value, and i dont think we call GetPbStr with the current problem, since we display it straight in a table ?

[jeanmi151]

now that we use a bootstrap table to display the values, i think we can use a formatter to humanize the numbers on the fly, like the ones in

gaia/geordash/static/js/script.js

Line 561 in c70ae82

function urlFormatter(value, row) {

• and some js like https://gist.github.com/lanqy/5193417

this way, no need for jinja filesizeformat filter in the backend, raw numbers are stored in the task results (which make it easier to reuse if you build something that parses the json from the task result), and eventually that might allow to sort based on the raw value ?

another note.. i dont think we need to store the path/url twice, nor the problem type in the results:

path	"101932200-101932299/101932275"
problem	"5.0 MB"
type	"UnusedFileRes"
url	"101932200-101932299/101932275"

unless you plan to tackle the other cases i had listed in #11 (comment) - for now the type is always the same value, and i dont think we call GetPbStr with the current problem, since we display it straight in a table ?

done in my last commit 0b9e744