Cedar syntax for entity collections

[benkehoe]

Cedar originally only used JSON for schema, and then added a native syntax with RFC 24. I'm interested in feedback on proposing the same for an entity literal syntax that would be used for defining lists of entities, in particular during testing.

Copying the RFC 24 motivation section here:

Cedar schemas for non-toy applications are hard to read and write. The root cause of the difficulty is the use of JSON:

• JSON has low information density. For three applications we’ve modeled — TinyTodo, Document Cloud, and GitHub — schemas span 4-5 pages.
• JSON does not support comments. This means that any design intent for a schema cannot be expressed inline.

We believe that a custom syntax for Cedar schemas can help. It can be more information-dense, support comments, and leverage familiar syntactic constructs. We think customers will be happier with a custom syntax: we regularly hear favorable reviews of Cedar's custom policy syntax, as compared to a JSON-based syntax.

I think the same applies to a developer's testing loop for the entities in a request. To test out a given request, a cumbersome JSON must be constructed and modified, remembering all the JSON keys for the various parts of an entity definition. I've experienced firsthand developers discussing and building Cedar schema for a use case and then starting to experiment with building policies against that schema but bogging down in getting enough entities defined to make testing the policies useful, and then when adjusting the schema, changing the entities to match is another cumbersome exercise. Unlike schema, where there is value for Cedar syntax being used during development as well as the data format in the implementation itself, I think the value here is primarily just during development.

Actions are already sort of entity literals anyway, so I don't think there would be a lot of new syntax to invent for it. Taking the default list of entities that's populated in the Cedar playground for the photo sharing app, 57 lines of JSON:

[
    {
        "uid": {
            "type": "PhotoApp::User",
            "id": "alice"
        },
        "attrs": {
            "userId": "897345789237492878",
            "personInformation": {
                "age": 25,
                "name": "alice"
            }
        },
        "parents": [
            {
                "type": "PhotoApp::UserGroup",
                "id": "alice_friends"
            },
            {
                "type": "PhotoApp::UserGroup",
                "id": "AVTeam"
            }
        ]
    },
    {
        "uid": {
            "type": "PhotoApp::Photo",
            "id": "vacationPhoto.jpg"
        },
        "attrs": {
            "private": false,
            "account": {
                "__entity": {
                    "type": "PhotoApp::Account",
                    "id": "ahmad"
                }
            }
        },
        "parents": []
    },
    {
        "uid": {
            "type": "PhotoApp::UserGroup",
            "id": "alice_friends"
        },
        "attrs": {},
        "parents": []
    },
    {
        "uid": {
            "type": "PhotoApp::UserGroup",
            "id": "AVTeam"
        },
        "attrs": {},
        "parents": []
    }
]

one could imagine it being instead as something like

PhotoApp::User::"alice" in [PhotoApp::UserGroup::"alice_friends", PhotoApp::UserGroup::"AVTeam"] {
    userId: "897345789237492878",
    personInformation: {
        age: 25,
        name: "alice"
    }
};

PhotoApp::Photo::"vacationPhoto.jpg" {
    private: false,
    account: PhotoApp::Account::"ahmad"
};

PhotoApp::UserGroup::"alice_friends" {};
PhotoApp::UserGroup::"AVTeam" {};

[cdisselkoen]

This sounds very reasonable, and we agree the example syntax is more readable and better information density. We'd be interested in an RFC on this.

[benkehoe]

Great! I'll get to work on that.

[hakanson]

As Ben finds time to write the RFC, I told him I would add notes to this issue. I worked up another example schema so I could include enum, extension functions, tags, etc.

namespace Cloud {
  type IpAddressInfo = {
    ipAddress: ipaddr,
    dnsName: String,
  };
  entity State enum ["pending", "available", "failed"];
  entity Subnet {
    isDefault: Bool,
    cidrBlock: ipaddr,
    availableIpAddressCount: Long,
    state: State,
  } tags String;
  entity Instance in [Subnet] {
    instanceType: String,
    privateIpAddress: IpAddressInfo,
    publicIpAddress?: IpAddressInfo,
  } tags String;
}

these JSON entities represent 2 Subnets and 3 Instances

[
  {
    "uid": { "type": "Cloud::Subnet", "id": "s-0" },
    "attrs": {
      "isDefault": true,
      "cidrBlock": "10.0.0.0/24",
      "availableIpAddressCount": 256,
      "state": { "type": "Cloud::State", "id": "available" }
    },
    "parents": [],
    "tags": {
      "Name": "MySubnet0"
    }
  },
  {
    "uid": { "type": "Cloud::Subnet", "id": "s-2" },
    "attrs": {
      "isDefault": false,
      "cidrBlock": { "fn": "ip", "arg": "10.1.0.0/24" },
      "availableIpAddressCount": 256,
      "state": { "__entity": { "type": "Cloud::State", "id": "pending" } }
    },
    "parents": []
  },
  {
    "uid": { "__entity": { "type": "Cloud::Instance", "id": "i-1" } },
    "attrs": {
      "instanceType": "micro",
      "privateIpAddress": {
        "ipAddress": "10.0.0.1",
        "dnsName": "ip-10-0-0-1.internal"
      }
    },
    "parents": [{ "__entity": { "type": "Cloud::Subnet", "id": "s-0" } }],
    "tags": {
      "Name": "MyInstance0"
    }
  },
  {
    "uid": { "type": "Cloud::Instance", "id": "i-2" },
    "attrs": {
      "instanceType": "large",
      "privateIpAddress": {
        "ipAddress": { "fn": "ip", "arg": "10.0.0.2" },
        "dnsName": "ip-10-0-0-2.internal"
      }
    },
    "parents": [{ "type": "Cloud::Subnet", "id": "s-0" }]
  },
  {
    "uid": { "type": "Cloud::Instance", "id": "i-3" },
    "attrs": {
      "instanceType": "xlarge",
      "privateIpAddress": {
        "ipAddress": { "__extn": { "fn": "ip", "arg": "10.0.0.3" } },
        "dnsName": "ip-10-0-0-3.internal"
      },
      "publicIpAddress": {
        "ipAddress": { "__extn": { "fn": "ip", "arg": "192.168.1.3" } },
        "dnsName": "www.example.com"
      }
    },
    "parents": [{ "type": "Cloud::Subnet", "id": "s-0" }]
  }
]

I "translated" as below (using trailing commas)

Cloud::Subnet::"s-0" {
  isDefault: true,
  cidrBlock: "10.0.0.0/24",
  availableIpAddressCount: 256,
  state: Cloud::State::"available",
} tags {
  Name: "MySubnet0",
};
Cloud::Subnet::"s-2" {
  isDefault: false,
  cidrBlock: ip("10.1.0.0/24"),
  availableIpAddressCount: 256,
  state: Cloud::State::"pending",
};
Cloud::Instance::"i-1" in [Cloud::Subnet::"s-0"] {
  instanceType: "micro",
  privateIpAddress: {
    ipAddress: "10.0.0.1",
    dnsName: "ip-10-0-0-1.internal",
  },
} tags {
  Name: "MyInstance0",
};
Cloud::Instance::"i-2" in [Cloud::Subnet::"s-0"] {
  instanceType: "large",
  privateIpAddress: {
    ipAddress: ip("10.0.0.2"),
    dnsName: "ip-10-0-0-2.internal",
  },
};
Cloud::Instance::"i-3" in [Cloud::Subnet::"s-0"] {
  instanceType: "xlarge",
  privateIpAddress: {
    ipAddress: ip("10.0.0.3"),
    dnsName: "ip-10-0-0-3.internal",
  },
  publicIpAddress: {
    ipAddress: ip("192.168.1.3"),
    dnsName: "www.example.com",
  },
};

I'll post some items as additional comments to make any threaded discussion easier.

[hakanson]

In the Cedar Schema to JSON translation, comments are lost but annotations appear under an "annotations" key at various levels. For Cedar entities JSON, annotations have a natural fit as a top level key on the entity, but harder to graft onto the existing JSON at the attribute level. Maybe annotations are only supported at the top level - this is similar to annotations in Cedar policies.

@doc("entity level")
Cloud::Subnet::"s-0" {
  isDefault: true,
  cidrBlock: "10.0.0.0/24",
  availableIpAddressCount: 256,
  state: Cloud::State::"available",
} tags {
  Name: "MySubnet0",
};

Updated Cedar entities JSON

{
  "annotations": { "doc": "entity level" },
  "uid": { "type": "Cloud::Subnet", "id": "s-0" },
  "attrs": {
    "isDefault": true,
    "cidrBlock": "10.0.0.0/24",
    "availableIpAddressCount": 256,
    "state": { "type": "Cloud::State", "id": "available" }
  },
  "parents": [],
  "tags": {
    "Name": "MySubnet0"
  }
}

[hakanson]

Schema aware parsing of extension functions means all these are valid in JSON

• "ipAddress": "10.0.0.1",
• "ipAddress": { "fn": "ip", "arg": "10.0.0.2" },
• "ipAddress": { "__extn": { "fn": "ip", "arg": "10.0.0.3" } },

Would both these be valid in the native Cedar entities format? Would there be a preference in translation?

• ipAddress: "10.0.0.1"
• ipAddress: ip("10.0.0.2")

[hakanson]

I don't think so, but does a namespace block any value other that less repetition? It seems to have the same problem that schema does with shadow types without a namespace (e.g. State below is assumed to be Cloud::State and could never be without namespace State).

namespace "Cloud" {
  Subnet::"s-2" {
    cidrBlock: ip("10.1.0.0/24"),
    availableIpAddressCount: 256,
    state: State::"pending",
  };
}

[benkehoe]

Opened RFC104. Should this issue be closed?

[cdisselkoen]

Yep, let's close this issue in favor of #104.