feat: table overwrite functionality #674
Conversation
| metadata: meta, | ||
| fs: fs, | ||
| projectedSchema: t.meta.CurrentSchema(), | ||
| boundRowFilter: boundFilter, | ||
| caseSensitive: true, | ||
| rowLimit: -1, // No limit | ||
| concurrency: 1, | ||
| } |
There was a problem hiding this comment.
I don't know if this is correct
|
I'll give this a look to start reviewing, in the meantime can you fix the lint issues? |
|
This requires the replacing table to match the schema of the replaced table, right? Asking since I'm interested in something like |
Yes |
|
The spark integration tests for overwrite don't seem to be passing |
I definitely didn't do those correctly, any guidance on whether they're structured fine or if I'm going about it the right way? I couldn't figure out testing it locally |
|
I'll try to find some time next week to take a look at the spark tests and see if I can figure it out. |
3 participants
Resolves #668
There are quite a few things I'm not too sure about with this implementation (like the filtering logic or if I've properly tested it)