Expose more data over the API

[sophiabits]

Exposes additional API endpoints (e.g. GET /problems/:id.xml), adds additional data to some endpoints (e.g. <groups> node when retrieving a ProblemSet), and hides some sensitive info such as the judge log when retrieving submissions.

Serializers

Serialization logic has been implemented by overriding the to_xml method of the relevant models which avoids the need to repeat :only and :include all over the place. Ideally we'd want this logic separated out into serializer classes, but I couldn't figure out an elegant way of doing that.

Some things I tried:

• The active_model_serializers gem does not support XML, and while the authors are interested in adding support they're only looking into it for version 0.10 which requires Rails 6. Train runs on Rails 4 and I imagine bumping by 2 major versions will break things (see: Added XML support rails-api/active_model_serializers#448 (comment))
• ActiveModel::Serializers::XML is, afaict, built-in to Rails 4 (it was split out in v5?) and so isn't very helpful
• A custom *Serializer class doesn't seem to work the way I'd expect it to (I've tried both returning yielding a Builder::MarkupXml and wind up with strange extraneous elements inserted)
• The serializable_hash method feels a bit nicer to override than to_xml, but doesn't receive all of the builder options (:include is missing, for instance)
• An attributes method results in pretty strange XML, see the total-weighting element here: https://i.imgur.com/w0zsVTj.png

If there's a better place for the serialization logic or a library you know of, I'm happy to rework the PR. Ruby isn't my area of expertise :)

The stock to_xml behavior adds type attributes to nodes, but if you manually add a tag through a builder you don't get that attribute. XmlUtil exists mainly to ergonomically add type attributes to custom nodes in order to maintain consistency with the rest of the XML doc.

Swapping over to JSON could be an idea, as it would let us use active_model_serializers. JSON also handles whitespace perfectly, whereas with XML it's possible to configure your parser to strip out extraneous whitepsace -- which is a problem when working with submissions. This felt like a pretty major change though so I didn't do it.

Didn't do

A few things mentioned on Discord haven't been done:

• Filtering out comments from problem statements; I'm not sure how to accomplish that using the built-in XML serializer.
• Hiding updated-at: updated-at works great as a cache key and I can't see any issue with exposing it -- so it's still visible.
• It was mentioned to hide name on User docs; I've set it up so that if you hit the API from a staff account you get the user's name (and email), but otherwise those fields don't exist. This mirrors the behavior of the HTML page.

Future

• None of these endpoints are paginated; I'm assuming this isn't an issue given Tom's comment re HTTP requests
• Swapping to JSON might be a good idea as it seems like there's more tooling available in the Rails ecosystem

Here are some endpoints which I haven't included in this PR, but would like to have done eventually (eventually probably meaning never -- parsing the HTML feels easier than writing the serializers):

• Including scoreboard data in GET /contests/:id.xml. There are a lot of moving parts to that code and I'm not entirely sure I understand all the acl rules.
• It would also be nice to have the parsed judge data inlined in inside GET /submissions/:id.xml as most of the info is available through the web interface anyway, but I didn't do that either as I think there's room for discussion about the exact shape of the XML and it's also fairly straightforward to parse out judge results from the HTML, anyway. Here's where I got up to before deciding it was getting too big: https://i.imgur.com/ibclHnr.png

XML Samples

Here are some sample XML docs (note that problems in the contest sample is collapsed in the UI; the contest startcode is now censored, and points+max-points are also excluded from submissions now): https://imgur.com/a/UfJs203

[Holmes98]

For stripping comments, could we just do something like this?

to_xml do |xml|
  xml.tag! 'statement', Loofah.fragment(statement).scrub!(Loofah::Scrubbers::NoComment.new)
end

Also, I think :owner_id should be hidden for problems, problem sets, contests, and groups (at least for non-admins).

[Holmes98]

Suggested change

	if policy.scoreboard? then
	if policy.show_details? then

Problem IDs should be hidden to users unless the contest has ended or they are a current/past contestant (we may want to reuse problems). We should probably also restrict access to :problem_set_id in the same way.

[Holmes98]

Suggested change

	opts[:exclude] ||= [:startcode]
	opts[:except] ||= [:startcode]

[Holmes98]

Same as with Problem#to_xml, should we restrict these to admins?

[Holmes98]

Should we restrict these to admins? This will list upcoming/current contests and groups/problem sets that the user doesn't have access to.

[Holmes98]

opts[:except] should work =)

[Holmes98]

Doesn't seem to be used?

Suggested change

require 'builder'