Skip to content

Conversation

@tinkerofwales
Copy link

@tinkerofwales tinkerofwales commented Mar 4, 2017

Adding datamining branch for branching and sentiment analysis. Parsing "libraries" that returns time/location information with LDA assumed sentiment topics

…sys. Parsing "libraries" that returns time/location information with LDA assumed sentiment topics
@newswim
Copy link
Member

newswim commented Mar 4, 2017

This could come in really handy when we're handling more text-based documents. We should definitely think about integrating this with:

  • Proposals that come before council
  • Tax bonds
  • Any type of RFP for the city

Let's think about splitting the off into its own microservice. Doing that would allow us to run background tasks and not worry about eating up limited server resources. Then, we can host both the list of documents that should be processed, along with the results of the analysis, on the denton_data api.

Plus, this allows teams to work on adding features and maintaining the codebase in a more fine-grained way. Let's initialize a new repository under OpenDenton -- tag me or Kyle if you need any help -- but let's also continue to track this (future) service here within this PR.

Here's a few runway items for consideration:

  • What documents should we process and where do we get them from?
  • What is the shape of the output data (schema)?
  • What table names / models will store this, we'll refer to these later to scaffold the API?

This is awesome, y'all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants