content scan feature

[liangchenye]

Snapshot : scan backend

updateservice/snapshot is an interface, developers could add different plugins to do their scan work.
Now I just adding a simple 'appv1' plugin which get the md5 of an appv1 package.

why call it 'Snapshot'

In the traditional repositories provider, like SUSE/Redhat distribution host servers, their are not only provide simple metadata, but also provide detail information. It helps user to 'search' required packages and download them.

why under updateservice

In TUF framework, there is a similar snapshot concept. If a package admin want to provide a secure snapshot service to his user, he/she can signature/encrypt snapshot.

How does it work -- Plugin mode

Just like other services under updateservice, we can develop/use Snapshot by a single interface. The interface will choose the suitable scan plugin.

Plugin Developer

There are two key functions:

• Supported(proto string) bool

  This is to tell which protocol this plugin supports. For example, if a plugin could scan 'appv1',
  Supported("appv1") should be 'true'
  We also have Description function which is easier for user to understand what this plugin do.

• Process (url, id, callback)

  • URL

    URL is a local url of an appv1 package or a local dir of a docker image. A plugin developer could read its content and do what ever he want.

  • ID

    ID is the callbackID (scan taskID), but was encrypted in order not to expose the inner database details. Plugin developer should use this to tell 'Plugin user' the scan result.

  • callback

  If the Plugin User provides this 'callback' function to plugin developer, the developer MUST call it in his/her plugin.
  Or the developer should post the scan result to Dockyard by RestAPI which will mentioned later.

Plugin User (or we can call them dockyard handler developer)

Plugin User provides should choose the right plugin and calling it. He/she can query the scan status by callbackID and collected the scan result by this ID too.

I added four APIs:

• RegistScanHook (POST)

  Dockyard will have lots plugins. A dockyard user could choose one (or several in the future) suitable plugin for his/her repository. I call this action 'regist'.

• ActiveScanHook (POST)
  A dockyard user could scan a package. Dockyard will find the registered scan plugin and use that to scan the package. I call this action 'active'

• CallbackScanHook (POST)
  This is used for a plugin developer, he/she can post his/her scan result to dockyard by this ID.

• QueryResultScanHook (GET)
  This is not done. It is also for dockyard user, he/she can query the scan result by calling this API.

[liangchenye]

I'll keep update this document and add to README once it become mature.
Any suggest are welcomed.

[liangchenye]

Update.

Add two types of Snapshot: Simple Snapshot and Group Snapshot

• Simple Snapshot

the scan logic is controlled by 'dockyard' developer, just like 'simpleappv1'
Other integration such like 'Clair' will be a part of 'simple snapshot'.

• Group Snapshot

  the scan logic is controlled by 'dockyard' repo admin, just like 'bycontainer'.
  He/she could use his/her own method, for example, he/she could provide his/her own
  scan container image and ask Dockyard to scan by that image.
  TODO: we need to add some demo images and list of such image on the README