added UAST diff algorithm #309

quinor · 2018-09-21T10:38:53Z

still WIP, diffing works, applying diff not implemented yet

uast/diff/changelist.go

uast/diff/diff.go

uast/diff/apply.go

uast/diff/changelist.go

uast/diff/diff.go

dennwc · 2018-09-24T15:21:01Z

uast/diff/diff.go

+		keys := make(map[string]bool)
+		iterate := func(keyset nodes.Object) {
+			for key := range keyset {
+				if in := keys[key]; !in {


This comment still applies

uast/diff/diff.go

dennwc · 2018-09-24T15:30:25Z

Also, it would be nice to have some tests for diff generation as well as for appling the diff.

quinor · 2018-10-27T17:10:11Z

How do we add tests in the repository? I've got some internal ones.

dennwc · 2018-10-30T17:50:52Z

@quinor You can add a test function to a new file ./uast/diff/diff_test.go. CI will call it automatically.

quinor · 2018-10-30T19:01:56Z

@dennwc how to proceed if the test(s) need serialized data or external resources to work? My current testing procedure is to launch a client that runs the diff on two files and tries to apply it afterwards comparing the result with source, additionally having bblfshd docker running for the source file parsing.

dennwc · 2018-10-30T19:41:08Z

@quinor Feel free to add a fixtures or testdata folder and add few YAML UAST files to it. Tests can read and diff those files. No need to run the whole client+bblfshd chain.

To get those YAMLs you can run the bblfsh-cli included in the latest Go client. It can connect to bblfshd, parse files and emit the YAML/binary formats.

quinor · 2018-10-30T20:47:08Z

Is there any documentation/examples for the go+CI we are using?

dennwc · 2018-10-30T22:26:35Z

@quinor Don't think about CI right now, you can use go test ./... to test it locally. The same command will be executed by CI.

quinor · 2018-10-30T22:37:03Z

Did not realize there is an unified "go test" framework. That simplifies things alot :)

uast/diff/apply.go

uast/diff/changelist.go

uast/diff/apply.go

uast/diff/changelist_test.go

uast/diff/diff.go

uast/diff/testdata/create_testdata.py

dennwc · 2018-11-14T12:09:38Z

uast/diff/testdata/create_testdata.py

@@ -0,0 +1,36 @@
+#!/usr/bin/env python3


The script is pretty simple. Can be written in Go as well (it's the only Python file here).

I only threw it in so that one may try to regenerate/generate new test cases. Also, see the answers below.

I still think it makes sense to rewrite it in Go. All the SDK is in Go. I see no reason to require Python installation for such a simple script.

dennwc · 2018-11-14T12:10:53Z

uast/diff/testdata/smalltest.txt

@@ -0,0 +1,148 @@
+720_1


Maybe there is a formal definition for this? For example "source size is less than N bytes/lines" or "the UAST size is less than N bytes/nodes"?

I'm not sure how I'd chosen these. If there was such definition, how to proceed?

The script can check the file size before writing it to the destination.

By the way, if the script was in Go, you can use the Babelfish client directly and count nodes / check the size of the source or the UAST.

The reason I wrote it in Python is I'm not really comfortable with file-processing scripts in Go. Python seemed like a better alternative to bash (that's what it would be written in otherwise).

Also, Python exists on every modern Linux distro so it's still quite portable (see the alternative: bash)

uast/diff/testdata/create_testdata.py

dennwc · 2019-03-06T12:36:34Z

Needs a rebase on top of a current master at least.

Also make sure to run go fmt ./...

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

dennwc · 2019-04-11T13:50:14Z

Have to close this PR - cannot rebase, because it's made from the master branch on the fork. See #396.

quinor force-pushed the master branch from d3f112d to a51a5c5 Compare September 21, 2018 11:02

dennwc suggested changes Sep 21, 2018

View reviewed changes

quinor force-pushed the master branch from a51a5c5 to e02ca6f Compare September 21, 2018 14:15

dennwc suggested changes Sep 24, 2018

View reviewed changes

juanjux suggested changes Oct 31, 2018

View reviewed changes

uast/diff/apply.go Outdated Show resolved Hide resolved

uast/diff/apply.go Outdated Show resolved Hide resolved

uast/diff/changelist.go Show resolved Hide resolved

quinor force-pushed the master branch 2 times, most recently from b2681a8 to c65e770 Compare November 10, 2018 16:01

dennwc suggested changes Nov 14, 2018

View reviewed changes

quinor force-pushed the master branch 2 times, most recently from 025d8bb to 8c1149e Compare December 12, 2018 10:32

Wojciech Jabłoński added 9 commits March 24, 2019 16:09

added AST diff

9866519

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

fixed style/PR notes

230391c

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

added a skeleton of diff apply function

da7c149

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

the diff seems to be appliable

cd6e161

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

Code cleanup and several bugfixes in the diff library

2e69a68

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

fixed a bug when creating/deleting "nil" child

b329d1b

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

basic tests added

32fccbc

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

style fixes

b08b00f

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

simplified decisionType instances

f61f2da

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

quinor force-pushed the master branch from f3f123f to cb33640 Compare March 24, 2019 15:13

style changes

eeffeb8

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

quinor force-pushed the master branch from cb33640 to eeffeb8 Compare March 24, 2019 15:24

Wojciech Jabłoński added 2 commits March 24, 2019 16:51

added dependency to the makefile

5012732

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

moved python script to go

d8d9b25

Signed-off-by: Wojciech Jabłoński <wj359634@students.mimuw.edu.pl>

dennwc mentioned this pull request Apr 11, 2019

UAST diff #396

Open

dennwc closed this Apr 11, 2019

added UAST diff algorithm #309

added UAST diff algorithm #309

Uh oh!

Conversation

quinor commented Sep 21, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dennwc commented Sep 24, 2018

Uh oh!

quinor commented Oct 27, 2018

Uh oh!

dennwc commented Oct 30, 2018

Uh oh!

quinor commented Oct 30, 2018

Uh oh!

dennwc commented Oct 30, 2018

Uh oh!

quinor commented Oct 30, 2018

Uh oh!

dennwc commented Oct 30, 2018

Uh oh!

quinor commented Oct 30, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dennwc commented Mar 6, 2019

Uh oh!

dennwc commented Apr 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development