Clean up updater metadata update stack

[awwad]

tuf.client.updater has Updater objects. These contain a variety of client information, and also use some client information in settings.py. Though some client information lives in settings.py, an Updater object is, basically, a client. You update metadata using an Updater object.

There are a few issues with the stack:

Verification procedure is buried

The stack looks like this, where all functions are methods of Updater except where a module is noted:

• You call refresh() on an updater object.
• refresh() calls, for each role, _update_metadata(...)
• _update_metadata(...) calls _get_metadata_file(...)
• _get_metadata_file() loops over mirrors and:
  • calls (misleadingly named) tuf.download.unsafe_download(...) to get a temp file (max length limit)
  • calls _verify_uncompressed_metadata_file(...) to check the file
• _get_metadata_file() stops when it has a successfully verified file, and returns it up the stack.

This stack limits the interface so that verification is hidden more deeply: it means that it looks like the only way to obtain a file in the API results in that file being already verified. I do not think this is reasonable here, as it's important for understandability and modularity to avoid this burying of verification.

The stack should, I think, instead look like this:

• You call refresh() on an updater object.
• refresh() calls, for each role, update_metadata(...) (now public)
• update_metadata(...) loops over mirrors and:
  • calls get_unverified_metadata to get a temp file (max length limit)
  • calls verify_metadata (now public)
• update_metadata(...) stops when it has a successfully verified file, and returns it up the stack.

You should be able to look at the public refresh() and the public update_metadata(...) and have an immediate understanding of what is happening, and naturally know where to make changes to metadata verification. These are high-level functions that are central to an implementation's TUF's specification conformance.

Redundant code in _update_metadata, _update_metadata_if_changed

[MatthewBosek]

Hello, can I work on this issue?

[awwad]

There's work on this already in #868, as part of #846. That's been left sitting, since I've been pretty busy. When I get TUF time, it's on the top of my stack. That said, you can always provide a PR, suggestions, or talk about what you need / think is best. Work can be integrated or replace existing work.

[lukpueh]

Note to myself: @awwad points out in #868 that the "updater shouldn't use roledb at all".