change request to data model (in admin)

[cccs-ip]

PLEASE NOTE: In this discussion, I refer to google spreadsheet that I started for plotting different design elements of our data model. I find google spreadsheets to be a more flexible system of mapping out ideas that using the GitHub wiki and in the issues file.

---

I plotted out the fields you have in the document's admin interface into the spreadsheet tab called 'current structure'. I then compared the fields you were using to the fields we had identified in our different bibliographic tracking file. From this comparison, I finalized my desired organization, (presented in my 'target restructuring' spreadsheet tab , which identies the names, grouping and sequence for all data-entry fields we'll use to reference a given document / file object. [Please also note that I have changed around your "sections" (the open to show entry fields).]

Please help me to add in all these new fields identified in the spreadsheet's 'backend organization - TARGET' tab. Please do this first, then help to re-deploy to Abadi.

Also relevant to our discussion of the data model, please allow me to address few more places where we encounter confusion:

1. Use of the term "title"

• object name as distinct from title / title as bibliographic field for articles
  • we currently have 'title' as the first value shown in the admin section; I would prefer to make this a distinct field value, and to reserve 'title' for the bibliography (or introduce 'article_title' in the bibliography section and map that to 'title' for BibTeX outputs)
• booktitle as bibliographic field for books
  • please introduce this; it's a standard BibTeX field

2. Use of the field URL

• currently URL only specified in naming data; we wish to store multiple URL records as part of a file's "provenience" record

3. Use of 'lookups' and validations

• countries - please link to our list of countries
• bibtex entities - links to our BibTeX lookup
• cccs entities - links to our CCCS entities (which we can leave, for now)

4. Records of dates created and modified

• I would like this value to be taken from the file metadata, but I would also like to be able to override the value pulled from the metadata. Please note, files in Windows have "data created" values in their metadata; this is what I would like to capture on automated / scripted import, if possible.

5. meanings of "filename"

• It is unclear to me why this list of "filenames" is presented as a lookup table. All that I was looking to do with the "file names" field was to be able to maintain a few different values for how the document was originally named, or how that name changed (like if a doc is called 'download.pdf' as well as 'aaron-test.pdf'). We don't need all the path information in this context; just the actual file name. What do the names in this list link to? Do they link to other "source files" or to another database record for a file object? I don't currently see the value in this.

6. meanings of "categories"

• Our current "categories" are just lists of folder paths from the original directory structure. We do not need / want such long "category" names. The original file path can be saved as just a basic text record. To save that information as some sort of organizational "category", best would be to cut out each path section, so that the files /home/aaron/test1/doc.txt. adn /home/aaron/test2/doc.txt would yeild 4 different categories: 'home', 'aaron' 'test1' and 'test2'.

[pwhipp]

I've created fixtures for the bibtex and cccs types. Countries is rather more problematic. The model for countries is held in the cccs core project within the projects.models module.

There is no Country table in the abadi project and no country table in the docmeta for countries.

We've discussed this in the past and recall that we decided not to use django-countries because of concerns where cccs might deviate from the 3166 standard in order to cover countries that no longer exist or that are named differently. Having the countries in the database gives you control over them.

So there is quite a lot of work in sorting out the Countries data referencing:

• create cccs_countries repository and project
• add cccs_countries as a requirement to docmeta and to cccs (for projects even if docmeta removed).
• add cccs migration to add countries_text field to Project (comma delim 3166 codes)
• add cccs datamigration to populate countries_text field in Project using countries field
• add cccs migration to drop Project's countries m2m field
• add cccs migration to add Project's countries m2m field linking to cccs_countries.country
• add cccs datamigration to populate Project countries field with cccs_countries (using countries_text field)
• add cccs migration to drop countries_text field and project countries model
• fix cccs admin
• dump cccs_countries data into docmeta fixture using cccs db.
• add docmeta migration to rename Document countries field to countries_text
• add docmeta migration to convert countries_text entries into comma delimited 3166 codes
• add docmeta migration to add countries m2m field linking to cccs_countries.country
• add docmeta migration to populate countries field with cccs_countries (using countries_text field)
• run docmeta migrations in cccs (including production and staging)
• load cccs_countries data into abadi
• run docmeta migrations in abadi (including production)

As a bonus, the cccs_countries module could be enhanced by cherry picking django-countries for its flags to get you the flag icons you have been wanting, making it possible to use these wherever necessary on the sites.

Given the need to get this stuff up and running over the next week, I've documented the above and will do the countries last.

[pwhipp]

I've made these changes and am raising separate issues on our discussion points to move forwards.