Skip to content

Converter to tf.record improvements #118

@iislucas

Description

@iislucas
  • The current CSV to tf.record converter has hard-wired field selectors. These should be specified on the command line.
  • CSV is a bad format: there are multiple usually incompatible and badly supported 'standards'. If people use CSV, print out a warning, that CSVs are fragile, and the user should consider a more robust format like jsonlines, or json.
  • Support jsonlines, and json input, and when we do that, maybe rename script appropriately.
  • Consider: make our dataset class natively support and convert examples inline instead of requiring pre-processing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions